5470 tinfoil2 2013-11-05 11:09:32 +0000 bitbake-layers cannot be used with memory resident bitbake running 2017-03-28 22:41:11 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple VERIFIED FIXED https://wiki.yoctoproject.org/wiki/Tinfoil2 High normal 2.3 M2 7555 8735 10200 10516 1 andreea.b.proca anibal.limon alexandru.damian anibal.limon benjamin.esquivel bluelightning kergoth leonardo.sandoval.gonzalez poky.bs.watcher poky.watcher richard.purdie sgw oscarx.lopez.arandas --- --- No (bug/feature does not impact docs) oldest_to_newest 38528 0 andreea.b.proca 2013-11-05 11:09:32 +0000 Steps to reproduce: 1. Build an image, for example; 2. Wait until it finishes; 3. run "bitbake-layers show_layers" I receive this error: "Error: Only one copy of bitbake should run against a build." 38530 1 bluelightning 2013-11-05 11:40:56 +0000 This will apply to any tool that uses the tinfoil module. Unfortunately I suspect this means tinfoil will need to be reworked. 51657 2 leonardo.sandoval.gonzalez 2015-06-11 22:32:01 +0000 (In reply to comment #1) > This will apply to any tool that uses the tinfoil module. Unfortunately I > suspect this means tinfoil will need to be reworked. Paul, do you have a design/plan to overcome this? devtool is also affected by tinfoil. One approach would be to consider the app name (bitbake, bitbake-layers, devtool, etc) and just make the bitbake.lock check if app == bitbake. 51660 3 bluelightning 2015-06-12 08:21:06 +0000 Actually I've got more than a design - I have an initial version in a contrib branch here: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=paule/tinfoil2 It still has a lot of rough edges; at the moment I'm mostly waiting for responses on an RFC regarding a proper API for tinfoil before proceeding: http://lists.openembedded.org/pipermail/openembedded-core/2015-June/105804.html When this first came up I did consider workarounds similar to what you are describing, but given the way bitbake works there's not currently a safe way for two servers to access the same set of cache files without a bunch more locking in place. 53251 4 bluelightning 2015-08-20 07:57:24 +0000 FYI work on this has stalled for now - it's turned out to be significantly more complicated than I'd anticipated, not only because there were more places that needed changing, but I also discovered a number of problems with memory resident mode that need fixing at the same time in order to make this of any use. I have sent the patches I have up to this point that are independent of the tinfoil refactoring itself, and these have now been merged. I've rebased the paule/tinfoil2 branch on poky-contrib on top of latest master as well. 56864 5 bluelightning 2015-11-30 09:25:49 +0000 I've rebased the paule/tinfoil2 branch in poky-contrib on top of master and fixed it again so that at least the devtool, recipetool and bblayers oe-selftest tests pass. There's still quite a lot of cleanup and additional testing required though, and memres still needs work - e.g. I'm not actually witnessing any noticeable performance gain from using it at the moment. 56875 6 benjamin.esquivel 2015-11-30 17:01:00 +0000 I guess that the performance gains of using memres over regular setenv not only need to be noticeable but measurable. Leo, it would be nice to identify which performance parameters can be used to measure the improvements of using memres. 67952 7 benjamin.esquivel 2016-11-03 22:18:26 +0000 Waiting on Paul to provide a rebased-to-master branch of tinfoil2. 68452 8 bluelightning 2016-11-21 09:09:16 +0000 I have rebased the branch on top of master and fixed a bunch of issues today - the basics are at least working. I'll keep poking at it. 68514 9 benjamin.esquivel 2016-11-22 21:02:54 +0000 assigning back to Anibal since Paul has already provided a rebased branch. Leo is pending to free up Anibal's workload to see if this can be pulled in for 2.3 M1. 68517 10 bluelightning 2016-11-22 21:23:18 +0000 Note that I am still working on this in the mean time. 68656 11 bluelightning 2016-11-28 22:23:25 +0000 OK, I have just pushed another branch with a whole bunch of issues fixed. I've also written the following wiki page with some background and the current status: https://wiki.yoctoproject.org/wiki/Tinfoil2 68699 12 benjamin.esquivel 2016-11-29 22:59:09 +0000 (In reply to comment #11) > OK, I have just pushed another branch with a whole bunch of issues fixed. > I've also written the following wiki page with some background and the > current status: > > https://wiki.yoctoproject.org/wiki/Tinfoil2 okay Paul, is this good now for taking over? 69642 13 anibal.limon 2017-01-09 22:44:41 +0000 This bug was fixed by Paul with recently tinfoil2 changes hash [1], We need to add more extensive tenting to memres, i'll setup an AB to make periodic runs of memres and discover more bugs. [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=358befb76ede4b111a9ee913ae2586ea30dc23b7 69838 14 benjamin.esquivel 2017-01-16 20:56:40 +0000 Great Anibal, I guess it starts with a buildset that does oe-selftest from a memres environment. 71825 15 oscarx.lopez.arandas 2017-03-28 19:51:17 +0000 BUILD: 2.3.m3.rc1: master:415b72ffcbd26e5f3664370d8b2a9b8105fb6342 ENVIRONMENT: genericx86 Steps to reproduce: 1. Build an image: core-image-sato 2. Wait until it finishes; 3. run "bitbake-layers show_layers" I receive this: scar@oscar-ThinkPad-T540p:~/layers/build$ bitbake-layers show-layers layer path priority ========================================================================== meta /home/oscar/layers/meta 5 meta-poky /home/oscar/layers/meta-poky 5 meta-yocto-bsp /home/oscar/layers/meta-yocto-bsp 5 71828 16 bluelightning 2017-03-28 20:16:30 +0000 (In reply to comment #15) > BUILD: 2.3.m3.rc1: master:415b72ffcbd26e5f3664370d8b2a9b8105fb6342 > ENVIRONMENT: genericx86 > > > Steps to reproduce: > 1. Build an image: core-image-sato > 2. Wait until it finishes; > 3. run "bitbake-layers show_layers" Your steps didn't mention how you initialised your environment, did you use oe-init-build-env or oe-init-build-env-memres? 71835 17 oscarx.lopez.arandas 2017-03-28 21:37:19 +0000 Initialize environment with oe-init-build-env. 71837 18 anibal.limon 2017-03-28 21:50:51 +0000 (In reply to comment #17) > Initialize environment with oe-init-build-env. Hi Oscar, You need to use oe-init-build-env-memres because without it memory resident mode isn't activated. Please run again your tests using memres script. Anibal 71838 19 3677 oscarx.lopez.arandas 2017-03-28 22:24:30 +0000 Created attachment 3677 bitbake-layers show-layers 71839 20 oscarx.lopez.arandas 2017-03-28 22:26:45 +0000 BUILD: 2.3.m3.rc1: master:415b72ffcbd26e5f3664370d8b2a9b8105fb6342 ENVIRONMENT: genericx86 Steps to reproduce: 1. Initialize environment with oe-init-build-env-memres 2. Build an image: core-image-sato 3. Wait until it finishes; 4. I run "bitbake-layers show-layers" 71841 21 anibal.limon 2017-03-28 22:41:11 +0000 (In reply to comment #19) > Created attachment 3677 [details] > bitbake-layers show-layers Good, thanks!. 3677 2017-03-28 22:24:30 +0000 2017-03-28 22:24:30 +0000 bitbake-layers show-layers bitbake-layers show-layers application/octet-stream 1869 oscarx.lopez.arandas b3NjYXJAb3NjYXItVGhpbmtQYWQtVDU0MHA6fi9sYXllcnMkIHNvdXJjZSBvZS1pbml0LWJ1aWxk LWVudi1tZW1yZXMKTm8gcG9ydCBzcGVjaWZpZWQsIHVzaW5nIGR5bmFtaWNhbGx5IHNlbGVjdGVk IHBvcnQKCiMjIyBTaGVsbCBlbnZpcm9ubWVudCBzZXQgdXAgZm9yIGJ1aWxkcy4gIyMjCgpZb3Ug Y2FuIG5vdyBydW4gJ2JpdGJha2UgPHRhcmdldD4nCgpDb21tb24gdGFyZ2V0cyBhcmU6CiAgICBj b3JlLWltYWdlLW1pbmltYWwKICAgIGNvcmUtaW1hZ2Utc2F0bwogICAgbWV0YS10b29sY2hhaW4K ICAgIG1ldGEtaWRlLXN1cHBvcnQKCllvdSBjYW4gYWxzbyBydW4gZ2VuZXJhdGVkIHFlbXUgaW1h Z2VzIHdpdGggYSBjb21tYW5kIGxpa2UgJ3J1bnFlbXUgcWVtdXg4NicKQml0YmFrZSBzZXJ2ZXIg c3RhcnRlZCBvbiBkZW1hbmQgYXMgbmVlZGVkLCB1c2UgYml0YmFrZSAtbSB0byBzaHV0IGl0IGRv d24Kb3NjYXJAb3NjYXItVGhpbmtQYWQtVDU0MHA6fi9sYXllcnMvYnVpbGQkIGJpdGJha2UgY29y ZS1pbWFnZS1zYXRvCkxvYWRpbmcgY2FjaGU6IDEwMCUgfCMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI3wgVGltZTogMDowMDow MQpMb2FkZWQgMTMwMSBlbnRyaWVzIGZyb20gZGVwZW5kZW5jeSBjYWNoZS4KTk9URTogUmVzb2x2 aW5nIGFueSBtaXNzaW5nIHRhc2sgcXVldWUgZGVwZW5kZW5jaWVzCgpCdWlsZCBDb25maWd1cmF0 aW9uOgpCQl9WRVJTSU9OICAgICAgICA9ICIxLjMzLjIiCkJVSUxEX1NZUyAgICAgICAgID0gIng4 Nl82NC1saW51eCIKTkFUSVZFTFNCU1RSSU5HICAgPSAidW5pdmVyc2FsIgpUQVJHRVRfU1lTICAg ICAgICA9ICJpNTg2LXBva3ktbGludXgiCk1BQ0hJTkUgICAgICAgICAgID0gInFlbXV4ODYiCkRJ U1RSTyAgICAgICAgICAgID0gInBva3kiCkRJU1RST19WRVJTSU9OICAgID0gIjIuMitzbmFwc2hv dC0yMDE3MDMyOCIKVFVORV9GRUFUVVJFUyAgICAgPSAibTMyIGk1ODYiClRBUkdFVF9GUFUgICAg ICAgID0gIiIKbWV0YSAgICAgICAgICAgICAgCm1ldGEtcG9reSAgICAgICAgIAptZXRhLXlvY3Rv LWJzcCAgICA9ICJtYXN0ZXI6NDE1YjcyZmZjYmQyNmU1ZjM2NjQzNzBkOGIyYTliODEwNWZiNjM0 MiIKCkluaXRpYWxpc2luZyB0YXNrczogMTAwJSB8IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI3wgVGltZTogMDowMDoxNQpOT1RFOiBF eGVjdXRpbmcgU2V0U2NlbmUgVGFza3MKTk9URTogRXhlY3V0aW5nIFJ1blF1ZXVlIFRhc2tzCk5P VEU6IFRhc2tzIFN1bW1hcnk6IEF0dGVtcHRlZCA2MDUyIHRhc2tzIG9mIHdoaWNoIDYwNTIgZGlk bid0IG5lZWQgdG8gYmUgcmVydW4gYW5kIGFsbCBzdWNjZWVkZWQuCm9zY2FyQG9zY2FyLVRoaW5r UGFkLVQ1NDBwOn4vbGF5ZXJzL2J1aWxkJCBiaXRiYWtlLWxheWVycyBzaG93LWxheWVycwpsYXll ciAgICAgICAgICAgICAgICAgcGF0aCAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgcHJpb3JpdHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KbWV0YSAgICAgICAgICAgICAgICAgIC9ob21l L29zY2FyL2xheWVycy9tZXRhICAgICAgICAgICAgICAgICAgIDUKbWV0YS1wb2t5ICAgICAgICAg ICAgIC9ob21lL29zY2FyL2xheWVycy9tZXRhLXBva3kgICAgICAgICAgICAgIDUKbWV0YS15b2N0 by1ic3AgICAgICAgIC9ob21lL29zY2FyL2xheWVycy9tZXRhLXlvY3RvLWJzcCAgICAgICAgIDUK b3NjYXJAb3NjYXItVGhpbmtQYWQtVDU0MHA6fi9sYXllcnMvYnVpbGQkIAoK