12228 2017-10-16 11:34:15 +0000 Race between do_make_sccripts() and recipes using the scripts it creates 2018-02-28 01:58:49 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 2.3.2 x86 Multiple RESOLVED FIXED Medium+ normal 2.5 M3 1 peter.kjellerstedt bruce.ashfield joe.slater mark.hatle meta.mr.watcher meta.watcher randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 77605 0 peter.kjellerstedt 2017-10-16 11:34:15 +0000 Back in 2011, a lock (${TMPDIR}/kernel-scripts.lock) was introduced to protect do_make_scripts() from racing with itself when executed from different module recipes. The race was due to some of the kernel scripts always being rebuilt. Unfortunately this lock is not enough, since there is still a race between the scripts being rebuilt for one recipe and them being used by another. I.e., if do_make_script() is being executed for one kernel module recipe while another kernel module recipe is compiling, then there is a tiny chance that, e.g., modpost has been rewritten but not yet made executable by the first task at the same time as the second task tries to execute modpost, resulting in an error like this: /bin/sh: scripts/mod/modpost: Permission denied My suggestion for a solution is to move do_make_scripts() from modules-base.bbclass to a recipe of its own (e.g., kernel-scripts) and then let modules.bbclass add a dependency on that recipe instead of adding the make_scripts task to the task list for the inheriting recipe. That way there should only ever be one recipe and task that creates the kernel scripts, and no race exists any more (which also means the lock file is no longer needed). 77633 1 bruce.ashfield 2017-10-17 12:25:47 +0000 Joe Slater has already sent a patch to do this in the past. I'll try and dig it up from the mailing list. 77638 2 bruce.ashfield 2017-10-17 15:45:35 +0000 Attached is the patch to do this. I tested it here, and it works. I can re-submit to the list shortly, it would be nice to know if it works for Peter has well. I'm sure it'll take a few revisions to get in, but the concept is the same. 77640 3 4086 bruce.ashfield 2017-10-17 15:47:11 +0000 Created attachment 4086 initial implementation patch 77645 4 peter.kjellerstedt 2017-10-17 20:28:17 +0000 The attached patch looks more or less exactly as what I had in mind. The only comment I have is that I believe that the dependency: do_compile[depends] += "virtual/kernel:do_compile_kernelmodules" in the new recipe should be: do_compile[depends] += "virtual/kernel:do_shared_workdir" When it comes to testing, I have no real test case. I have only seen it happen once in all the time I have used OE, and based my report on my analysis of the code after it happened. 77960 5 peter.kjellerstedt 2017-11-03 20:44:36 +0000 The patch creates the make-mod-scripts recipe in meta-skeleton, but that shall of course be in meta... 77961 6 peter.kjellerstedt 2017-11-04 00:01:57 +0000 And the make-mod-scripts recipe needs to inherit pkgconfig. 78240 7 bruce.ashfield 2017-11-16 14:29:51 +0000 Thanks for the comments. I'm back on this now and will do some tweaks before posting a new version 4086 2017-10-17 15:47:11 +0000 2017-10-17 15:47:11 +0000 initial implementation patch 0001-make-mod-scripts-change-how-some-kernel-module-tools.patch text/plain 3537 bruce.ashfield RnJvbSA2YWI4MjliY2FiMmMxNTdiMDM3YWE3NzUwNjY5OTcxYTQ2N2FhYmFlIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKb2UgU2xhdGVyIDxqc2xhdGVyQHdpbmRyaXZlci5jb20+CkRh dGU6IFRodSwgMjcgSnVsIDIwMTcgMTQ6NDg6NDUgLTA3MDAKU3ViamVjdDogW1BBVENIXSBtYWtl LW1vZC1zY3JpcHRzOiBjaGFuZ2UgaG93IHNvbWUga2VybmVsIG1vZHVsZSB0b29scyBhcmUKIGJ1 aWx0CgpSZW1vdmUgZG9fbWFrZV9zY3JpcHRzKCkgZnJvbSBtb2R1bGUtYmFzZS5iYmNsYXNzIGFu ZCBwdXQKZnVuY3Rpb25hbGl0eSBpbiBhIHJlY2lwZS4gIFRoaXMgd2lsbCBidWlsZCB0aGUgc2Ny aXB0cyBvbmx5Cm9uY2UgaW5zdGVhZCBvZiBlYWNoIHRpbWUgYW4gZXh0ZXJuYWwgbW9kdWxlIGlz IGJ1aWx0LgoKU2lnbmVkLW9mZi1ieTogSm9lIFNsYXRlciA8anNsYXRlckB3aW5kcml2ZXIuY29t PgpTaWduZWQtb2ZmLWJ5OiBCcnVjZSBBc2hmaWVsZCA8YnJ1Y2UuYXNoZmllbGRAd2luZHJpdmVy LmNvbT4KLS0tCiAuLi4vbWFrZS1tb2Qtc2NyaXB0cy9tYWtlLW1vZC1zY3JpcHRzXzEuMC5iYiAg ICAgICB8IDI2ICsrKysrKysrKysrKysrKysrKysrKysKIG1ldGEvY2xhc3Nlcy9tb2R1bGUtYmFz ZS5iYmNsYXNzICAgICAgICAgICAgICAgICAgIHwgMTIgKysrLS0tLS0tLQogbWV0YS9jbGFzc2Vz L21vZHVsZS5iYmNsYXNzICAgICAgICAgICAgICAgICAgICAgICAgfCAgNCAtLS0tCiAzIGZpbGVz IGNoYW5nZWQsIDI5IGluc2VydGlvbnMoKyksIDEzIGRlbGV0aW9ucygtKQogY3JlYXRlIG1vZGUg MTAwNjQ0IG1ldGEtc2tlbGV0b24vcmVjaXBlcy1rZXJuZWwvbWFrZS1tb2Qtc2NyaXB0cy9tYWtl LW1vZC1zY3JpcHRzXzEuMC5iYgoKZGlmZiAtLWdpdCBhL21ldGEtc2tlbGV0b24vcmVjaXBlcy1r ZXJuZWwvbWFrZS1tb2Qtc2NyaXB0cy9tYWtlLW1vZC1zY3JpcHRzXzEuMC5iYiBiL21ldGEtc2tl bGV0b24vcmVjaXBlcy1rZXJuZWwvbWFrZS1tb2Qtc2NyaXB0cy9tYWtlLW1vZC1zY3JpcHRzXzEu MC5iYgpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAuLjU5YzU1YTdmOTYw ZQotLS0gL2Rldi9udWxsCisrKyBiL21ldGEtc2tlbGV0b24vcmVjaXBlcy1rZXJuZWwvbWFrZS1t b2Qtc2NyaXB0cy9tYWtlLW1vZC1zY3JpcHRzXzEuMC5iYgpAQCAtMCwwICsxLDI2IEBACitTVU1N QVJZID0gIkJ1aWxkIHRvb2xzIG5lZWRlZCBieSBleHRlcm5hbCBtb2R1bGVzIgorTElDRU5TRSA9 ICJHUEx2MiIKKworaW5oZXJpdCBrZXJuZWwtYXJjaAorCitTID0gIiR7V09SS0RJUn0iCisKKyMg VGhpcyBpcyBpbnN0ZWFkIG9mIERFUEVORFMgPSAidmlydHVhbC9rZXJuZWwiLgorIyBJdCB3aWxs IGZvcmNlIHRoZSBrZXJuZWwgdG8gYnVpbGQuCisjCitkb19jb21waWxlW2RlcGVuZHNdICs9ICJ2 aXJ0dWFsL2tlcm5lbDpkb19jb21waWxlX2tlcm5lbG1vZHVsZXMiCisKKyMgQnVpbGQgc29tZSBo b3N0IHRvb2xzIHVuZGVyIHdvcmstc2hhcmVkLiAgQ0MsIExELCBhbmQgQVIgYXJlIHByb2JhYmx5 CisjIG5vdCB1c2VkLCBidXQgdGhpcyBpcyB0aGUgaGlzdG9yaWNhbCB3YXkgb2YgaW52b2tpbmcg Im1ha2Ugc2NyaXB0cyIuCisjCitkb19jb21waWxlKCkgeworCXVuc2V0IENGTEFHUyBDUFBGTEFH UyBDWFhGTEFHUyBMREZMQUdTIAorCW1ha2UgQ0M9IiR7S0VSTkVMX0NDfSIgTEQ9IiR7S0VSTkVM X0xEfSIgQVI9IiR7S0VSTkVMX0FSfSIgXAorCSAgICAgICAgICAgLUMgJHtTVEFHSU5HX0tFUk5F TF9ESVJ9IE89JHtTVEFHSU5HX0tFUk5FTF9CVUlMRERJUn0gc2NyaXB0cworfQorCisKKyMgVGhl cmUgaXMgbm8gcmVhc29uIHRvIGJ1aWxkIHRoaXMgb24gaXRzIG93bi4KKyMKK0VYQ0xVREVfRlJP TV9XT1JMRCA9ICIxIgorCmRpZmYgLS1naXQgYS9tZXRhL2NsYXNzZXMvbW9kdWxlLWJhc2UuYmJj bGFzcyBiL21ldGEvY2xhc3Nlcy9tb2R1bGUtYmFzZS5iYmNsYXNzCmluZGV4IDZmZTc3YzAxYjdl ZC4uZDY3NjJhN2RmZmQ4IDEwMDY0NAotLS0gYS9tZXRhL2NsYXNzZXMvbW9kdWxlLWJhc2UuYmJj bGFzcworKysgYi9tZXRhL2NsYXNzZXMvbW9kdWxlLWJhc2UuYmJjbGFzcwpAQCAtMSw3ICsxLDgg QEAKIGluaGVyaXQga2VybmVsLWFyY2gKIAotIyBUaGlzIGlzIGluc3RlYWQgb2YgREVQRU5EUyA9 ICJ2aXJ0dWFsL2tlcm5lbCIKLWRvX2NvbmZpZ3VyZVtkZXBlbmRzXSArPSAidmlydHVhbC9rZXJu ZWw6ZG9fY29tcGlsZV9rZXJuZWxtb2R1bGVzIgorIyBXZSBkbyB0aGUgZGVwZW5kZW5jeSB0aGlz IHdheSBiZWNhdXNlIHRoZSBvdXRwdXQgaXMgbm90IHByZXNlcnZlZAorIyBpbiBzc3RhdGUsIHNv IHdlIG11c3QgZm9yY2UgZG9fY29tcGlsZSB0byBydW4gKG9uY2UpLgorZG9fY29uZmlndXJlW2Rl cGVuZHNdICs9ICJtYWtlLW1vZC1zY3JpcHRzOmRvX2NvbXBpbGUiCiAKIGV4cG9ydCBPUyA9ICIk e1RBUkdFVF9PU30iCiBleHBvcnQgQ1JPU1NfQ09NUElMRSA9ICIke1RBUkdFVF9QUkVGSVh9IgpA QCAtMTgsMTAgKzE5LDMgQEAgS0VSTkVMX09CSkVDVF9TVUZGSVggPSAiLmtvIgogIyBrZXJuZWwg bW9kdWxlcyBhcmUgZ2VuZXJhbGx5IG1hY2hpbmUgc3BlY2lmaWMKIFBBQ0tBR0VfQVJDSCA9ICIk e01BQ0hJTkVfQVJDSH0iCiAKLSMgRnVuY3Rpb24gdG8gZW5zdXJlIHRoZSBrZXJuZWwgc2NyaXB0 cyBhcmUgY3JlYXRlZC4gRXhwZWN0ZWQgdG8KLSMgYmUgY2FsbGVkIGJlZm9yZSBkb19jb21waWxl LiBTZWUgbW9kdWxlLmJiY2xhc3MgZm9yIGFuIGV4YW1wbGUuCi1kb19tYWtlX3NjcmlwdHMoKSB7 Ci0JdW5zZXQgQ0ZMQUdTIENQUEZMQUdTIENYWEZMQUdTIExERkxBR1MgCi0JbWFrZSBDQz0iJHtL RVJORUxfQ0N9IiBMRD0iJHtLRVJORUxfTER9IiBBUj0iJHtLRVJORUxfQVJ9IiBcCi0JICAgICAg ICAgICAtQyAke1NUQUdJTkdfS0VSTkVMX0RJUn0gTz0ke1NUQUdJTkdfS0VSTkVMX0JVSUxERElS fSBzY3JpcHRzCi19CmRpZmYgLS1naXQgYS9tZXRhL2NsYXNzZXMvbW9kdWxlLmJiY2xhc3MgYi9t ZXRhL2NsYXNzZXMvbW9kdWxlLmJiY2xhc3MKaW5kZXggNzhkMWIyMWRiZDgzLi5lMzQ0OTYwMjQ0 MzQgMTAwNjQ0Ci0tLSBhL21ldGEvY2xhc3Nlcy9tb2R1bGUuYmJjbGFzcworKysgYi9tZXRhL2Ns YXNzZXMvbW9kdWxlLmJiY2xhc3MKQEAgLTEsOSArMSw1IEBACiBpbmhlcml0IG1vZHVsZS1iYXNl IGtlcm5lbC1tb2R1bGUtc3BsaXQgcGtnY29uZmlnCiAKLWFkZHRhc2sgbWFrZV9zY3JpcHRzIGFm dGVyIGRvX3ByZXBhcmVfcmVjaXBlX3N5c3Jvb3QgYmVmb3JlIGRvX2NvbmZpZ3VyZQotZG9fbWFr ZV9zY3JpcHRzW2xvY2tmaWxlc10gPSAiJHtUTVBESVJ9L2tlcm5lbC1zY3JpcHRzLmxvY2siCi1k b19tYWtlX3NjcmlwdHNbZGVwZW5kc10gKz0gInZpcnR1YWwva2VybmVsOmRvX3NoYXJlZF93b3Jr ZGlyIgotCiBFWFRSQV9PRU1BS0UgKz0gIktFUk5FTF9TUkM9JHtTVEFHSU5HX0tFUk5FTF9ESVJ9 IgogCiBNT0RVTEVTX0lOU1RBTExfVEFSR0VUID89ICJtb2R1bGVzX2luc3RhbGwiCi0tIAoyLjUu MAoK