9798 2016-06-17 09:58:39 +0000 Files in ~/tmp being deleted during a build 2016-07-01 16:05:08 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.0 x86 Multiple RESOLVED FIXED Medium+ major 2.2 M2 1 bluelightning ross.burton jose.perez.carranza joshuagloe liezhi.yang meta.mr.watcher meta.watcher francisco.j.pedraza.gonzalez --- --- No (bug/feature does not impact docs) oldest_to_newest 63094 0 3238 bluelightning 2016-06-17 09:58:39 +0000 Created attachment 3238 Results of ausearch with audit tracing enabled on ~/tmp So for a while now on my Fedora 23 build machine all the files in my ~/tmp directory were being deleted on an irregular basis (just that directory, not recursively). I had a hunch it was happening at the same time as rebuilding something early in the build process and I'd set up a canary and script to try to track down what was doing it, but it wasn't until yesterday I actually caught it with the audit system - see attached. There were a number of these events, one per entry in my ~/tmp so it was clearly looping through all of the items in that directory and calling unlink() on them. There are a bunch of interesting things in the log entries but the most significant is that the apparent executable running is /usr/bin/ccache. Richard dug into the ccache source and found that as part of cleanup it was deleting CCACHE_DIR/tmp, and then found that CCACHE_DIR is being set by us in bitbake.conf to $HOME if it's not set (which it isn't in my environment): -------- snip -------- # Assign CCACHE_DIR a default value to fix a bug of ccache 3.1.7, # since it would always create CCACHE_DIR/.ccache even if # CCACHE_DISABLE = 1. export CCACHE_DIR ??= "${@os.getenv('HOME')}" -------- snip -------- Given the results I really don't think this is a good idea. We need to investigate whether we really still need this and if we do, default it to something much more sensible. Luckily I don't think I've lost anything valuable with these deletions, but someone else might not be so fortunate. 63144 1 ross.burton 2016-06-20 12:04:01 +0000 I'm not sure why the assignment of CCACHE_DIR was made, as the default value for CCACHE_DIR if it isn't set is $HOME/.ccache/. I don't see what harm ccache touching that would cause compared to $HOME. Note that if ccache.bbclass is actually inherited then CCACHE_DIR is assigned there: export CCACHE_DIR = "${TMPDIR}/ccache/${MULTIMACH_HOST_SYS}/${PN}" Robert, you wrote the patch that introduced this back in 2012 (oe-core d2bab9b), do you remember why you set CCACHE_DIR to just $HOME in bitbake.conf? If ccache still has the bug where autogen sets HOME=/dev/null then I'd be inclined to either patch autogen to remove that, or set CCACHE to $HOME/.ccache in bitbake.conf (which will expand to a full path before autogen can destroy it). 63145 2 ross.burton 2016-06-20 12:12:06 +0000 The ccache bug mentioned was fixed in ccache 3.1.10, released 2014-10-19 (even Debian Stable has this). I think we should just remove that line. 63285 3 ross.burton 2016-06-23 14:41:58 +0000 Fixed in oe-core 15eaf9cb1fa19036fe4442905876dae94070b04d. 3238 2016-06-17 09:58:39 +0000 2016-06-17 09:58:39 +0000 Results of ausearch with audit tracing enabled on ~/tmp audit_tmp_deletions.txt text/plain 3764 bluelightning LS0tLQp0eXBlPVBST0NUSVRMRSBtc2c9YXVkaXQoMTYvMDYvMTYgMTc6MTc6NTAuMDQ5OjQ0Nykg OiBwcm9jdGl0bGU9ZysrIC1vIGNvbmZ0ZXN0IC1pc3lzdGVtL2hvbWUvcGF1bC9wb2t5L3Bva3kv YnVpbGQvdG1wL3N5c3Jvb3RzL3g4Nl82NC1saW51eC91c3IvaW5jbHVkZSAtcGlwZSAtTC9ob21l L3BhdWwvcG9reS9wb2t5L2J1aWxkL3RtcC8gCnR5cGU9UEFUSCBtc2c9YXVkaXQoMTYvMDYvMTYg MTc6MTc6NTAuMDQ5OjQ0NykgOiBpdGVtPTEgbmFtZT0vaG9tZS9wYXVsL3RtcC92aXJ0X2Vudl9s YXllcmluZGV4X3B5dGhvbjMgaW5vZGU9MTkwMjA4MTggZGV2PWZkOjAzIG1vZGU9ZGlyLDc3NSBv dWlkPXBhdWwgb2dpZD1wYXVsIHJkZXY9MDA6MDAgb2JqPXVuY29uZmluZWRfdTpvYmplY3Rfcjpo b21lX3Jvb3RfdDpzMCBuYW1ldHlwZT1ERUxFVEUgCnR5cGU9UEFUSCBtc2c9YXVkaXQoMTYvMDYv MTYgMTc6MTc6NTAuMDQ5OjQ0NykgOiBpdGVtPTAgbmFtZT0vaG9tZS9wYXVsL3RtcC8gaW5vZGU9 MTcxNzA0ODMgZGV2PWZkOjAzIG1vZGU9ZGlyLDc1NSBvdWlkPXBhdWwgb2dpZD1wYXVsIHJkZXY9 MDA6MDAgb2JqPXVuY29uZmluZWRfdTpvYmplY3Rfcjpob21lX3Jvb3RfdDpzMCBuYW1ldHlwZT1Q QVJFTlQgCnR5cGU9Q1dEIG1zZz1hdWRpdCgxNi8wNi8xNiAxNzoxNzo1MC4wNDk6NDQ3KSA6ICBj d2Q9L2hvbWUvcGF1bC9wb2t5L3Bva3kvYnVpbGQvdG1wL3dvcmsveDg2XzY0LWxpbnV4L2djYy1j cm9zcy1pbml0aWFsLXg4Nl82NC82LjEuMS1yMC9nY2MtNi4xLjEvYnVpbGQueDg2XzY0LWxpbnV4 Lng4Nl82NC1wb2t5LWxpbnV4L2djYyAKdHlwZT1TWVNDQUxMIG1zZz1hdWRpdCgxNi8wNi8xNiAx NzoxNzo1MC4wNDk6NDQ3KSA6IGFyY2g9eDg2XzY0IHN5c2NhbGw9dW5saW5rIHN1Y2Nlc3M9bm8g ZXhpdD1FSVNESVIoSXMgYSBkaXJlY3RvcnkpIGEwPTB4NTY0OTU3ZmE1ZDAwIGExPTB4NTY0OTU3 ZmE1ZDAwIGEyPTB4N2ZmZDVjMjUxYTYwIGEzPTB4NjQ2ZTY5NzI2NTc5NjE2YyBpdGVtcz0yIHBw aWQ9MTI2MzIgcGlkPTEyNjM1IGF1aWQ9cGF1bCB1aWQ9cGF1bCBnaWQ9cGF1bCBldWlkPXBhdWwg c3VpZD1wYXVsIGZzdWlkPXBhdWwgZWdpZD1wYXVsIHNnaWQ9cGF1bCBmc2dpZD1wYXVsIHR0eT0o bm9uZSkgc2VzPTEgY29tbT1nKysgZXhlPS91c3IvYmluL2NjYWNoZSBzdWJqPXVuY29uZmluZWRf dTp1bmNvbmZpbmVkX3I6dW5jb25maW5lZF90OnMwLXMwOmMwLmMxMDIzIGtleT13aG9kZWxldGVk aXQgCi0tLS0KdHlwZT1QUk9DVElUTEUgbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3OjUwLjA1MTo0 NDgpIDogcHJvY3RpdGxlPWcrKyAtbyBjb25mdGVzdCAtaXN5c3RlbS9ob21lL3BhdWwvcG9reS9w b2t5L2J1aWxkL3RtcC9zeXNyb290cy94ODZfNjQtbGludXgvdXNyL2luY2x1ZGUgLXBpcGUgLUwv aG9tZS9wYXVsL3Bva3kvcG9reS9idWlsZC90bXAvIAp0eXBlPVBBVEggbXNnPWF1ZGl0KDE2LzA2 LzE2IDE3OjE3OjUwLjA1MTo0NDgpIDogaXRlbT0xIG5hbWU9L2hvbWUvcGF1bC90bXAvc21hcnQt MS40LjFfb2xkIGlub2RlPTE3NTY3NjYwIGRldj1mZDowMyBtb2RlPWRpciw3NTUgb3VpZD1wYXVs IG9naWQ9cGF1bCByZGV2PTAwOjAwIG9iaj11bmNvbmZpbmVkX3U6b2JqZWN0X3I6aG9tZV9yb290 X3Q6czAgbmFtZXR5cGU9REVMRVRFIAp0eXBlPVBBVEggbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3 OjUwLjA1MTo0NDgpIDogaXRlbT0wIG5hbWU9L2hvbWUvcGF1bC90bXAvIGlub2RlPTE3MTcwNDgz IGRldj1mZDowMyBtb2RlPWRpciw3NTUgb3VpZD1wYXVsIG9naWQ9cGF1bCByZGV2PTAwOjAwIG9i aj11bmNvbmZpbmVkX3U6b2JqZWN0X3I6aG9tZV9yb290X3Q6czAgbmFtZXR5cGU9UEFSRU5UIAp0 eXBlPUNXRCBtc2c9YXVkaXQoMTYvMDYvMTYgMTc6MTc6NTAuMDUxOjQ0OCkgOiAgY3dkPS9ob21l L3BhdWwvcG9reS9wb2t5L2J1aWxkL3RtcC93b3JrL3g4Nl82NC1saW51eC9nY2MtY3Jvc3MtaW5p dGlhbC14ODZfNjQvNi4xLjEtcjAvZ2NjLTYuMS4xL2J1aWxkLng4Nl82NC1saW51eC54ODZfNjQt cG9reS1saW51eC9nY2MgCnR5cGU9U1lTQ0FMTCBtc2c9YXVkaXQoMTYvMDYvMTYgMTc6MTc6NTAu MDUxOjQ0OCkgOiBhcmNoPXg4Nl82NCBzeXNjYWxsPXVubGluayBzdWNjZXNzPW5vIGV4aXQ9RUlT RElSKElzIGEgZGlyZWN0b3J5KSBhMD0weDU2NDk1N2ZhNTNkMCBhMT0weDU2NDk1N2ZhNTNkMCBh Mj0weDdmZmQ1YzI1MWE2MCBhMz0weDU2NDk1N2ZhNTNjMCBpdGVtcz0yIHBwaWQ9MTI2MzIgcGlk PTEyNjM1IGF1aWQ9cGF1bCB1aWQ9cGF1bCBnaWQ9cGF1bCBldWlkPXBhdWwgc3VpZD1wYXVsIGZz dWlkPXBhdWwgZWdpZD1wYXVsIHNnaWQ9cGF1bCBmc2dpZD1wYXVsIHR0eT0obm9uZSkgc2VzPTEg Y29tbT1nKysgZXhlPS91c3IvYmluL2NjYWNoZSBzdWJqPXVuY29uZmluZWRfdTp1bmNvbmZpbmVk X3I6dW5jb25maW5lZF90OnMwLXMwOmMwLmMxMDIzIGtleT13aG9kZWxldGVkaXQgCi0tLS0KdHlw ZT1QUk9DVElUTEUgbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3OjUwLjA1Mjo0NDkpIDogcHJvY3Rp dGxlPWcrKyAtbyBjb25mdGVzdCAtaXN5c3RlbS9ob21lL3BhdWwvcG9reS9wb2t5L2J1aWxkL3Rt cC9zeXNyb290cy94ODZfNjQtbGludXgvdXNyL2luY2x1ZGUgLXBpcGUgLUwvaG9tZS9wYXVsL3Bv a3kvcG9reS9idWlsZC90bXAvIAp0eXBlPVBBVEggbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3OjUw LjA1Mjo0NDkpIDogaXRlbT0xIG5hbWU9L2hvbWUvcGF1bC90bXAvdmlydF9lbnZfbGF5ZXJpbmRl eF9kamFuZ28xOCBpbm9kZT0xODg5NTAyOSBkZXY9ZmQ6MDMgbW9kZT1kaXIsNzc1IG91aWQ9cGF1 bCBvZ2lkPXBhdWwgcmRldj0wMDowMCBvYmo9dW5jb25maW5lZF91Om9iamVjdF9yOmhvbWVfcm9v dF90OnMwIG5hbWV0eXBlPURFTEVURSAKdHlwZT1QQVRIIG1zZz1hdWRpdCgxNi8wNi8xNiAxNzox Nzo1MC4wNTI6NDQ5KSA6IGl0ZW09MCBuYW1lPS9ob21lL3BhdWwvdG1wLyBpbm9kZT0xNzE3MDQ4 MyBkZXY9ZmQ6MDMgbW9kZT1kaXIsNzU1IG91aWQ9cGF1bCBvZ2lkPXBhdWwgcmRldj0wMDowMCBv Ymo9dW5jb25maW5lZF91Om9iamVjdF9yOmhvbWVfcm9vdF90OnMwIG5hbWV0eXBlPVBBUkVOVCAK dHlwZT1DV0QgbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3OjUwLjA1Mjo0NDkpIDogIGN3ZD0vaG9t ZS9wYXVsL3Bva3kvcG9reS9idWlsZC90bXAvd29yay94ODZfNjQtbGludXgvZ2NjLWNyb3NzLWlu aXRpYWwteDg2XzY0LzYuMS4xLXIwL2djYy02LjEuMS9idWlsZC54ODZfNjQtbGludXgueDg2XzY0 LXBva3ktbGludXgvZ2NjIAp0eXBlPVNZU0NBTEwgbXNnPWF1ZGl0KDE2LzA2LzE2IDE3OjE3OjUw LjA1Mjo0NDkpIDogYXJjaD14ODZfNjQgc3lzY2FsbD11bmxpbmsgc3VjY2Vzcz1ubyBleGl0PUVJ U0RJUihJcyBhIGRpcmVjdG9yeSkgYTA9MHg1NjQ5NTdmYTVkMDAgYTE9MHg1NjQ5NTdmYTVkMDAg YTI9MHg3ZmZkNWMyNTFhNjAgYTM9MHg2NDZlNjk3MjY1Nzk2MTZjIGl0ZW1zPTIgcHBpZD0xMjYz MiBwaWQ9MTI2MzUgYXVpZD1wYXVsIHVpZD1wYXVsIGdpZD1wYXVsIGV1aWQ9cGF1bCBzdWlkPXBh dWwgZnN1aWQ9cGF1bCBlZ2lkPXBhdWwgc2dpZD1wYXVsIGZzZ2lkPXBhdWwgdHR5PShub25lKSBz ZXM9MSBjb21tPWcrKyBleGU9L3Vzci9iaW4vY2NhY2hlIHN1Ymo9dW5jb25maW5lZF91OnVuY29u ZmluZWRfcjp1bmNvbmZpbmVkX3Q6czAtczA6YzAuYzEwMjMga2V5PXdob2RlbGV0ZWRpdCAKLS0t LQo=