I just build an SDK (bitbake -c populate_sdk core-image-minimal) and both the host and target manifests seem to be prefixed with some log output that shouldn't be there (presumably from berkeleydb, I'm using RPM packaging): ------------- snip ------------- BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read BDB2053 Freeing read binutils-cross-canadian-x86-64 x86_64-nativesdk 2.25 gcc-cross-canadian-x86-64 x86_64-nativesdk 4.9.3 ... ------------- snip -------------
I'm not able to reproduce the bug. I have found these tracebacks are because something crashes while reading the rpmdb or exists without cleaning. I would like to reproduce it in order to start working on this. Are you using a RPM based distro? Can you reproduce this error easily? Can you explain more the steps to reproduce it?
Tried to reproduce it on a Fedora system without any luck. Also tried to reproduce in a Fedora system with three simultaneous builds but couldn't regenerate the issue.
Created attachment 2678 [details] Selftest for manifest files
I'm not able to reproduce the bug yet, nevertheless Paul can reproduce it very easily. The last comment contains a patch for a new selftest to check the manifest files against pkgdata. It needs some work but it's a start
Mariano, would you send your patch to the list? I would like to send another one on top of yours.
Paul, I'm a little confused with your manifests you're reporting here, when I build SDK (bitbake -c populate_sdk core-image-minimal) like you mention in the first comment, I get no manifests created in the DEPLOY_DIR_IMAGE. Is this the dir where you're seeing it?
A test for checking that all the lines of a manifest correspond to a package was proposed in: http://lists.openembedded.org/pipermail/openembedded-core/2015-August/109472.html
Waiting on Paul's comments
${SDK_DEPLOY} is where you should see the manifests for built SDKs (along with the SDK itself) i.e. tmp/deploy/sdk.
Thanks Paul, I checked ${SDK_DEPLOY} for the error you mention and I did not obtain it though. I will proceed to create the test for the manifest of the SDK as well, I expect to send the patch tomorrow. In the meantime, any other pointer as to why you are able to obtain those error messages so easily?
Paul, under which context can SDK_DEPLOY be found? I believe oe self-tests execute bitbake -p and -e to get the variables but the get_bb_var('SDK_DEPLOY') fails to get it.
You'll need to be querying it for an image (e.g. the one you've built the SDK for) i.e. get_bb_var('SDK_DEPLOY', 'core-image-minimal')
Created attachment 2702 [details] oeqa.selftest.manifest.VerifyManifest tests log
a patch V2 was sent to the list http://lists.openembedded.org/pipermail/openembedded-core/2015-September/109907.html
A patch V3 will be required for modifying the test to ignore old manifests in the dir and avoid false negatives
A pointer to look in the meta/classes/testimage.bbclass, specifically the following line: manifest = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + ".manifest")
patch V3 http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111348.html
So, FYI this is still happening on my setup; also Benjamin's test does fail here as a result (as it should).
I wish I knew why your setup seems to have this failure reproduced often. Also, is there a way to check how many times this test has failed in the autobuilders and if so, access the logs? we never got to see this issue in a number of machines in GDC including autobuilders.
Setting back to IN PROGRESS IMPLEMENTATION since it's been punted back to me to investigate.
I was able to reproduce this error today using openSuse. I just build a core-image-sato after core-image-full-cmdline. It seems the problem is that the rpm database is dirty after using _list_pkg_deps() from class RpmPkgsList in file package_manager.py. _list_pkg_deps() calls rpmresolve and it seems this command leaves the locks in the database. The actual output of the rpm command is: BDB2053 Freeing read locks for locker 0x98: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x99: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9a: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9b: 19853/140035173365568 BDB2053 Freeing read locks for locker 0x9c: 19853/140035173365568 packagegroup-base-usbgadget genericx86_64 1.0 /path/to/tmp/deploy/rpm/genericx86_64/packagegroup-base-usbgadget-1.0-r83.genericx86_64.rpm shadow-securetty genericx86_64 4.2.1 /path/to/tmp/deploy/rpm/genericx86_64/shadow-securetty-4.2.1-r3.genericx86_64.rpm glibc-binary-localedata-en-gb core2_64 2.22 /path/to/tmp/deploy/rpm/core2_64/glibc-binary-localedata-en-gb-2.22-r0.core2_64.rpm linux-firmware-iwlwifi-license all 0.0+git0+bbe4917c05 /path/to/tmp/deploy/rpm/all/linux-firmware-iwlwifi-license-0.0+git0+bbe4917c05-r0.all.rpm
I created a different manifest after calling _list_pkg_deps() and every time I got the dirty output.
The issue is in rpmresolve program, there are some rpmmi pointers never freed when a package have null as requirement. This keeps the database opened after the program terminates.
Sent patch to the mailing list http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114688.html
Paul, patch has been integrated in master, can you confirm this solves the problem?
This has been solved in master with commit: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=eda3947cf06252346dfd89f626b2560092219242 I'll backport the patch to jethro and fido
This has been integrated into jethro waiting for fido and dizzy http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=jethro&id=d632a923dcd4646de827d1d2725d6699730f7feb
This has been integrated in fido: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=fido&id=c94eb07d53c6bc26692dfb30aa7b006129f79d66 Waiting for dizzy
Already talked with Armin, he said he will integrate the patch in the new pull request for dizzy
This isn't yet integrated into Dizzy. Armin: will you do that, or should we close this as resolved?
Patch sent for dizzy.
I am not planning on fixing this issue. Marking as wontfix. There are several Yocto member's who provide commercial support and may be able to provide assistances on this issue. https://www.yoctoproject.org/ecosystem/member-organizations.
I'm re-marking this as fixed and changed the target milestone, since it was fixed in master and the newer stable branches.