Bug 9446

Summary:	write_deploy_manifest() assumes all its dependencies have been built, but they are not dependencies of the rootfs task
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Peter Kjellerstedt <peter.kjellerstedt>
Component:	core	Assignee:	Markus Lehtonen <markus.lehtonen>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	joshuagloe, meta.mr.watcher, meta.watcher, mikko.ylinen, richard.purdie
Version:	2.1
Target Milestone:	2.1 M4
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Peter Kjellerstedt 2016-04-12 15:27:15 UTC

The write_deploy_manifest() function is called via ROOTFS_POSTPROCESS_COMMAND, i.e., after the rootfs task. write_deploy_manifest() calculates a list of dependencies via the get_deployed_dependencies() function which takes the dependencies of the rootfs task, EXTRA_IMAGEDEPENDS and some other boot tasks. However, since not all of these dependencies are dependencies of the rootfs task, there is nothing that guarantees that all of them have actually been built at the time write_deploy_manifest() is executed, which results in a race condition...

Comment 1 Richard Purdie 2016-04-14 14:57:49 UTC

I suspect we need to move the license.bbclass:ROOTFS_POSTPROCESS_COMMAND_prepend = "write_package_manifest; write_deploy_manifest; license_create_manifest; " deploY-manifest to something which runs as part of do_image_complete.

Comment 2 Markus Lehtonen 2016-04-15 13:16:57 UTC

Shouldn't IMAGE_POSTPROCESS_COMMAND do the job?

Do you have any instructions how to reproduce this bug?

Comment 3 Markus Lehtonen 2016-04-20 10:34:26 UTC

A fix was merged into master (and krogoth):
http://git.openembedded.org/openembedded-core/commit/?id=1f35b05d60f4e84554c5ee837cb071815e12c91c

Please re-open if you still see issues.

Comment 4 Mikko Ylinen 2016-04-22 06:43:33 UTC

The change triggers an error:

15:54:09 File: '/var/lib/jenkins/ostro-worker-20-slot-0-mLLdv/ostro-os/meta/classes/license.bbclass', lineno: 67, function: write_license_files
15:54:09      0063:    bad_licenses = (d.getVar("INCOMPATIBLE_LICENSE", True) or "").split()
15:54:09      0064:    bad_licenses = map(lambda l: canonical_license(d, l), bad_licenses)
15:54:09      0065:    bad_licenses = expand_wildcard_licenses(d, bad_licenses)
15:54:09      0066:
15:54:09  *** 0067:    with open(license_manifest, "w") as license_file:
15:54:09      0068:        for pkg in sorted(pkg_dic):
15:54:09      0069:            if bad_licenses:
15:54:09      0070:                try:
15:54:09      0071:                    (pkg_dic[pkg]["LICENSE"], pkg_dic[pkg]["LICENSES"]) = \
15:54:09 Exception: IOError: [Errno 2] No such file or directory: '/var/lib/jenkins/ostro-worker-20-slot-0-mLLdv/ostro-os/build/tmp-glibc/deploy/licenses/ostro-image-swupd-all-intel-corei7-64-2016-04-21_10-26-38-build-430/image_license.manifest'

Comment 5 Markus Lehtonen 2016-04-22 06:54:14 UTC

That unfortunate error will be fixed by this:
http://patchwork.openembedded.org/patch/120671/