8028 2015-07-17 15:05:58 +0000 Garbage in SDK manifest file 2016-10-03 19:48:22 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.8 x86 Multiple RESOLVED FIXED Medium+ normal 2.0.3 1 bluelightning akuster benjamin.esquivel bogdanx.a.voiculescu mariano.lopez meta.mr.watcher meta.watcher ross.burton yi.zhao bogdanx.a.voiculescu --- --- No (bug/feature does not impact docs) oldest_to_newest 52469 0 bluelightning 2015-07-17 15:05:58 +0000 I just build an SDK (bitbake -c populate_sdk core-image-minimal) and both the host and target manifests seem to be prefixed with some log output that shouldn't be there (presumably from berkeleydb, I'm using RPM packaging): ------------- snip ------------- BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read binutils-cross-canadian-x86-64 x86_64-nativesdk 2.25 gcc-cross-canadian-x86-64 x86_64-nativesdk 4.9.3 ... ------------- snip ------------- 52719 1 mariano.lopez 2015-07-29 22:17:24 +0000 I'm not able to reproduce the bug. I have found these tracebacks are because something crashes while reading the rpmdb or exists without cleaning. I would like to reproduce it in order to start working on this. Are you using a RPM based distro? Can you reproduce this error easily? Can you explain more the steps to reproduce it? 53168 2 mariano.lopez 2015-08-17 21:36:53 +0000 Tried to reproduce it on a Fedora system without any luck. Also tried to reproduce in a Fedora system with three simultaneous builds but couldn't regenerate the issue. 53365 3 2678 mariano.lopez 2015-08-24 17:09:06 +0000 Created attachment 2678 Selftest for manifest files 53366 4 mariano.lopez 2015-08-24 17:11:05 +0000 I'm not able to reproduce the bug yet, nevertheless Paul can reproduce it very easily. The last comment contains a patch for a new selftest to check the manifest files against pkgdata. It needs some work but it's a start 53396 5 benjamin.esquivel 2015-08-25 22:53:29 +0000 Mariano, would you send your patch to the list? I would like to send another one on top of yours. 53442 6 benjamin.esquivel 2015-08-26 20:08:20 +0000 Paul, I'm a little confused with your manifests you're reporting here, when I build SDK (bitbake -c populate_sdk core-image-minimal) like you mention in the first comment, I get no manifests created in the DEPLOY_DIR_IMAGE. Is this the dir where you're seeing it? 53449 7 benjamin.esquivel 2015-08-26 21:18:32 +0000 A test for checking that all the lines of a manifest correspond to a package was proposed in: http://lists.openembedded.org/pipermail/openembedded-core/2015-August/109472.html 53450 8 benjamin.esquivel 2015-08-26 21:18:56 +0000 Waiting on Paul's comments 53457 9 bluelightning 2015-08-26 22:07:51 +0000 ${SDK_DEPLOY} is where you should see the manifests for built SDKs (along with the SDK itself) i.e. tmp/deploy/sdk. 53512 10 benjamin.esquivel 2015-08-27 22:44:51 +0000 Thanks Paul, I checked ${SDK_DEPLOY} for the error you mention and I did not obtain it though. I will proceed to create the test for the manifest of the SDK as well, I expect to send the patch tomorrow. In the meantime, any other pointer as to why you are able to obtain those error messages so easily? 53546 11 benjamin.esquivel 2015-08-28 22:12:06 +0000 Paul, under which context can SDK_DEPLOY be found? I believe oe self-tests execute bitbake -p and -e to get the variables but the get_bb_var('SDK_DEPLOY') fails to get it. 53552 12 bluelightning 2015-08-29 11:32:18 +0000 You'll need to be querying it for an image (e.g. the one you've built the SDK for) i.e. get_bb_var('SDK_DEPLOY', 'core-image-minimal') 53647 13 2702 benjamin.esquivel 2015-09-03 04:26:00 +0000 Created attachment 2702 oeqa.selftest.manifest.VerifyManifest tests log 53648 14 benjamin.esquivel 2015-09-03 04:26:55 +0000 a patch V2 was sent to the list http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109907.html 54401 15 benjamin.esquivel 2015-09-21 16:32:11 +0000 A patch V3 will be required for modifying the test to ignore old manifests in the dir and avoid false negatives 54500 16 benjamin.esquivel 2015-09-22 22:38:01 +0000 A pointer to look in the meta/classes/testimage.bbclass, specifically the following line: manifest = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + ".manifest") 55043 17 benjamin.esquivel 2015-10-07 19:45:34 +0000 patch V3 http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111348.html 55969 18 bluelightning 2015-10-29 10:10:22 +0000 So, FYI this is still happening on my setup; also Benjamin's test does fail here as a result (as it should). 56007 19 benjamin.esquivel 2015-10-29 15:02:06 +0000 I wish I knew why your setup seems to have this failure reproduced often. Also, is there a way to check how many times this test has failed in the autobuilders and if so, access the logs? we never got to see this issue in a number of machines in GDC including autobuilders. 56325 20 bluelightning 2015-11-09 14:48:56 +0000 Setting back to IN PROGRESS IMPLEMENTATION since it's been punted back to me to investigate. 57357 21 mariano.lopez 2015-12-17 17:47:17 +0000 I was able to reproduce this error today using openSuse. I just build a core-image-sato after core-image-full-cmdline. It seems the problem is that the rpm database is dirty after using _list_pkg_deps() from class RpmPkgsList in file package_manager.py. _list_pkg_deps() calls rpmresolve and it seems this command leaves the locks in the database. The actual output of the rpm command is: BDB2053 Freeing read locks for locker 0x98: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x99: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9a: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9b: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9c: 19853/140035173365568 packagegroup-base-usbgadget genericx86_64 1.0 /path/to/tmp/deploy/rpm/genericx86_64/packagegroup-base-usbgadget-1.0-r83.genericx86_64.rpm shadow-securetty genericx86_64 4.2.1 /path/to/tmp/deploy/rpm/genericx86_64/shadow-securetty-4.2.1-r3.genericx86_64.rpm glibc-binary-localedata-en-gb core2_64 2.22 /path/to/tmp/deploy/rpm/core2_64/glibc-binary-localedata-en-gb-2.22-r0.core2_64.rpm linux-firmware-iwlwifi-license all 0.0+git0+bbe4917c05 /path/to/tmp/deploy/rpm/all/linux-firmware-iwlwifi-license-0.0+git0+bbe4917c05-r0.all.rpm 57359 22 mariano.lopez 2015-12-17 18:03:07 +0000 I created a different manifest after calling _list_pkg_deps() and every time I got the dirty output. 57486 23 mariano.lopez 2015-12-29 18:03:52 +0000 The issue is in rpmresolve program, there are some rpmmi pointers never freed when a package have null as requirement. This keeps the database opened after the program terminates. 57487 24 mariano.lopez 2015-12-29 18:31:34 +0000 Sent patch to the mailing list http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114688.html 57830 25 mariano.lopez 2016-01-14 17:12:01 +0000 Paul, patch has been integrated in master, can you confirm this solves the problem? 58186 26 mariano.lopez 2016-01-25 14:49:05 +0000 This has been solved in master with commit: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=eda3947cf06252346dfd89f626b2560092219242 I'll backport the patch to jethro and fido 58763 27 mariano.lopez 2016-02-09 00:34:59 +0000 This has been integrated into jethro waiting for fido and dizzy http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=jethro&id=d632a923dcd4646de827d1d2725d6699730f7feb 59290 28 mariano.lopez 2016-02-19 22:36:48 +0000 This has been integrated in fido: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=fido&id=c94eb07d53c6bc26692dfb30aa7b006129f79d66 Waiting for dizzy 59792 29 mariano.lopez 2016-03-08 14:44:33 +0000 Already talked with Armin, he said he will integrate the patch in the new pull request for dizzy 63951 30 ross.burton 2016-07-13 22:50:20 +0000 This isn't yet integrated into Dizzy. Armin: will you do that, or should we close this as resolved? 63955 31 ross.burton 2016-07-13 23:30:27 +0000 Patch sent for dizzy. 66837 32 akuster 2016-10-03 19:41:43 +0000 I am not planning on fixing this issue. Marking as wontfix. There are several Yocto member's who provide commercial support and may be able to provide assistances on this issue. https://www.yoctoproject.org/ecosystem/member-organizations. 66838 33 bluelightning 2016-10-03 19:48:22 +0000 I'm re-marking this as fixed and changed the target milestone, since it was fixed in master and the newer stable branches. 2678 2015-08-24 17:09:06 +0000 2015-08-24 17:09:06 +0000 Selftest for manifest files 0001-selftest-manifest.py-Test-to-verify-rootfs-manifest.patch text/plain 1756 mariano.lopez RnJvbSBlMTdiYjRiYzkwMmMyZmMxNDkxNWRlOTliZWQzNzk5OTk5MjRlMDM5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBNYXJpYW5vIExvcGV6IDxtYXJpYW5vLmxvcGV6QGxpbnV4Lmlu dGVsLmNvbT4KRGF0ZTogRnJpLCA3IEF1ZyAyMDE1IDEyOjI3OjI2ICswMDAwClN1YmplY3Q6IFtQ QVRDSF0gc2VsZnRlc3QvbWFuaWZlc3QucHk6IFRlc3QgdG8gdmVyaWZ5IHJvb3RmcyBtYW5pZmVz dAoKVGhpcyBhZGQgYSBuZXcgdGVzdCB0byB2ZXJpZnkgaWYgdGhlIHBhY2thZ2VzIGluIHRoZQpt YW5pZmVzdCBmaWxlcyBhY3R1YWxseSBleGlzdHMgaW4gcGtnZGF0YS4KClNpZ25lZC1vZmYtYnk6 IE1hcmlhbm8gTG9wZXogPG1hcmlhbm8ubG9wZXpAbGludXguaW50ZWwuY29tPgotLS0KIG1ldGEv bGliL29lcWEvc2VsZnRlc3QvbWFuaWZlc3QucHkgfCAyNiArKysrKysrKysrKysrKysrKysrKysr KysrKwogMSBmaWxlIGNoYW5nZWQsIDI2IGluc2VydGlvbnMoKykKIGNyZWF0ZSBtb2RlIDEwMDY0 NCBtZXRhL2xpYi9vZXFhL3NlbGZ0ZXN0L21hbmlmZXN0LnB5CgpkaWZmIC0tZ2l0IGEvbWV0YS9s aWIvb2VxYS9zZWxmdGVzdC9tYW5pZmVzdC5weSBiL21ldGEvbGliL29lcWEvc2VsZnRlc3QvbWFu aWZlc3QucHkKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5kZXggMDAwMDAwMC4uMjQ4MzM2YQotLS0g L2Rldi9udWxsCisrKyBiL21ldGEvbGliL29lcWEvc2VsZnRlc3QvbWFuaWZlc3QucHkKQEAgLTAs MCArMSwyNiBAQAoraW1wb3J0IHVuaXR0ZXN0CitpbXBvcnQgb3MKK2Zyb20gZ2xvYiBpbXBvcnQg Z2xvYgorCitmcm9tIG9lcWEuc2VsZnRlc3QuYmFzZSBpbXBvcnQgb2VTZWxmVGVzdAorZnJvbSBv ZXFhLnV0aWxzLmNvbW1hbmRzIGltcG9ydCBnZXRfYmJfdmFyCitmcm9tIG9lcWEudXRpbHMuZGVj b3JhdG9ycyBpbXBvcnQgdGVzdGNhc2UKKworY2xhc3MgVmVyaWZ5TWFuaWZlc3Qob2VTZWxmVGVz dCk6CisKKyAgICBAdGVzdGNhc2UoMzE4NCkKKyAgICBkZWYgdGVzdF9tYW5pZmVzdChzZWxmKToK KyAgICAgICAgZGVwbG95ZGlyID0gZ2V0X2JiX3ZhcigiREVQTE9ZX0RJUl9JTUFHRSIpCisgICAg ICAgIHBrZ2RhdGFkaXIgPSBnZXRfYmJfdmFyKCJQS0dEQVRBX0RJUiIpCisgICAgICAgIHJ1bmRp ciA9IG9zLnBhdGguam9pbihwa2dkYXRhZGlyLCAicnVudGltZS1yZXZlcnNlIikKKyAgICAgICAg ZXJybXNnID0gIkVSUk9SOiBQYWNrYWdlICVzIGlzIGluIG1hbmlmZXN0IGJ1dCBub3QgaW4gcGtn ZGF0YSIKKyAgICAgICAgbWFuaWZlc3RfZmlsZXMgPSBnbG9iKCIlcy8qLm1hbmlmZXN0IiAlIGRl cGxveWRpcikKKyAgICAgICAgc2VsZi5hc3NlcnRUcnVlKG1hbmlmZXN0X2ZpbGVzLAorICAgICAg ICAgICAgICAgICJFUlJPUjogTWFuaWZlc3QgZmlsZXMgbm90IGZvdW5kIikKKyAgICAgICAgZm9y IG1hbmlmZXN0IGluIG1hbmlmZXN0X2ZpbGVzOgorICAgICAgICAgICAgd2l0aCBvcGVuKG1hbmlm ZXN0LCAiciIpIGFzIG1maWxlOgorICAgICAgICAgICAgICAgIGZvciBsaW5lIGluIG1maWxlOgor ICAgICAgICAgICAgICAgICAgICBwa2cgPSBsaW5lLnNwbGl0KClbMF0KKyAgICAgICAgICAgICAg ICAgICAgcGtnZmlsZSA9IG9zLnBhdGguam9pbihydW5kaXIsIHBrZykKKyAgICAgICAgICAgICAg ICAgICAgc2VsZi5hc3NlcnRUcnVlKG9zLnBhdGguaXNmaWxlKHBrZ2ZpbGUpLAorICAgICAgICAg ICAgICAgICAgICAgICAgICAgIGVycm1zZyAlIHBrZykKLS0gCjEuOC40LjUKCg== 2702 2015-09-03 04:26:00 +0000 2015-09-03 04:26:00 +0000 oeqa.selftest.manifest.VerifyManifest tests log file_8028.txt text/plain 1311 benjamin.esquivel JCBvZS1zZWxmdGVzdCAtLXJ1bi10ZXN0cyBtYW5pZmVzdA0KMjAxNS0wOS0wMiAyMDoxMTo0OCwy OTcgLSBzZWxmdGVzdCAtIElORk8gLSBSdW5uaW5nIGJpdGJha2UgLWUgdG8gZ2V0IEJCUEFUSA0K MjAxNS0wOS0wMiAyMDoxMTo0OSwwMjggLSBzZWxmdGVzdCAtIElORk8gLSBDaGVja2luZyB0aGF0 IGV2ZXJ5dGhpbmcgaXMgaW4gb3JkZXIgYmVmb3JlIHJ1bm5pbmcgdGhlIHRlc3RzDQoyMDE1LTA5 LTAyIDIwOjExOjQ5LDc1NyAtIHNlbGZ0ZXN0IC0gSU5GTyAtIFJ1bm5pbmcgYml0YmFrZSAtcA0K MjAxNS0wOS0wMiAyMDoxMTo1MiwwOTMgLSBzZWxmdGVzdCAtIElORk8gLSBMb2FkaW5nIHRlc3Rz IGZyb206IG9lcWEuc2VsZnRlc3QubWFuaWZlc3QNCjIwMTUtMDktMDIgMjA6MTE6NTIsMDk1IC0g c2VsZnRlc3QgLSBJTkZPIC0gQWRkaW5nOiAiaW5jbHVkZSBzZWxmdGVzdC5pbmMiIGluIGxvY2Fs LmNvbmYNCjIwMTUtMDktMDIgMjA6MTE6NTIsMDk1IC0gc2VsZnRlc3QgLSBJTkZPIC0gQWRkaW5n OiAiaW5jbHVkZSBiYmxheWVycy5pbmMiIGluIGJibGF5ZXJzLmNvbmYNCjIwMTUtMDktMDIgMjA6 MTE6NTIsMDk2IC0gc2VsZnRlc3QuYmFzZSAtIElORk8gLSBWZXJpZnlNYW5pZmVzdDogZG9pbmcg Yml0YmFrZSBjb3JlLWltYWdlLW1pbmltYWwgYXMgYSBwcmVyZXF1aXNpdGUgb2YgdGhlIHRlc3QN CjIwMTUtMDktMDIgMjA6MTE6NTkgLSB0ZXN0X1NES19tYW5pZmVzdF9lbnRyaWVzIChvZXFhLnNl bGZ0ZXN0Lm1hbmlmZXN0LlZlcmlmeU1hbmlmZXN0KQ0KVmVyaWZ5aW5nIHRoZSBTREsgbWFuaWZl c3QgZW50cmllcyBleGlzdCwgdGhpcyBtYXkgdGFrZSBhIGJ1aWxkIC4uLiBvaw0KMjAxNS0wOS0w MiAyMDoxMjoxNyAtIHRlc3RfaW1hZ2VfbWFuaWZlc3RfZW50cmllcyAob2VxYS5zZWxmdGVzdC5t YW5pZmVzdC5WZXJpZnlNYW5pZmVzdCkNClZlcmlmeWluZyB0aGUgaW1hZ2UgbWFuaWZlc3QgZW50 cmllcyBleGlzdCAuLi4gb2sNCg0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KUmFuIDIgdGVzdHMgaW4gMzQuMzI4 cw0KDQpPSw0KMjAxNS0wOS0wMiAyMDoxMjoyNiw0MjMgLSBzZWxmdGVzdCAtIElORk8gLSBGaW5p c2hlZA0KMjAxNS0wOS0wMiAyMDoxMjoyNiw0MjMgLSBzZWxmdGVzdCAtIElORk8gLSBSZW1vdmlu ZyB0aGUgaW5jbHVkZSBmcm9tIGxvY2FsLmNvbmYNCjIwMTUtMDktMDIgMjA6MTI6MjYsNDI0IC0g c2VsZnRlc3QgLSBJTkZPIC0gUmVtb3ZpbmcgdGhlIGluY2x1ZGUgZnJvbSBiYmxheWVycy5jb25m