Bug 13130 - multiconfig fails when depends on artifact from target2
Summary: multiconfig fails when depends on artifact from target2
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.7
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.7 M3
Assignee: Alejandro Hernandez
QA Contact:
URL:
Whiteboard: with backport to 2.6.2
Depends on:
Blocks:
 
Reported: 2019-01-09 20:10 UTC by Armin Kuster
Modified: 2020-01-23 17:41 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
pass all data caches to get_taskhash (2.95 KB, patch)
2020-01-16 12:26 UTC, Frederik Noe-Sdun
no flags Details | Diff
pass data caches to prep_taskhash (8.37 KB, patch)
2020-01-16 12:26 UTC, Frederik Noe-Sdun
no flags Details | Diff
include mcdepend taskhashes (1.57 KB, patch)
2020-01-16 12:27 UTC, Frederik Noe-Sdun
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Armin Kuster 2019-01-09 20:10:25 UTC
When I have a target1 recipe that has a mcdepends against a target2 task, it fails during do_rootfs for target1 on any random package.

ERROR: mc:host:host-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_rootfs(d)
     0003:
File: '/home/build/releases/master/poky/meta/classes/image.bbclass', lineno: 250, function: do_rootfs
     0246:    progress_reporter.next_stage()
     0247:
     0248:    # generate rootfs
     0249:    d.setVarFlag('REPRODUCIBLE_TIMESTAMP_ROOTFS', 'export', '1')
 *** 0250:    create_rootfs(d, progress_reporter=progress_reporter, logcatcher=logcatcher)
     0251:
     0252:    progress_reporter.finish()
     0253:}
     0254:do_rootfs[dirs] = "${TOPDIR}"
File: '/home/build/releases/master/poky/meta/lib/oe/rootfs.py', lineno: 975, function: create_rootfs
     0971:    img_type = d.getVar('IMAGE_PKGTYPE')
     0972:    if img_type == "rpm":
     0973:        RpmRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
     0974:    elif img_type == "ipk":
 *** 0975:        OpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
     0976:    elif img_type == "deb":
     0977:        DpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
     0978:
     0979:    os.environ.clear()
File: '/home/build/releases/master/poky/meta/lib/oe/rootfs.py', lineno: 707, function: __init__
     0703:            bb.utils.remove(self.image_rootfs, True)
     0704:            self.pm = OpkgPM(d,
     0705:                             self.image_rootfs,
     0706:                             self.opkg_conf,
 *** 0707:                             self.pkg_archs)
     0708:        else:
     0709:            self.pm = OpkgPM(d,
     0710:                             self.image_rootfs,
     0711:                             self.opkg_conf,
File: '/home/build/releases/master/poky/meta/lib/oe/package_manager.py', lineno: 1160, function: __init__
     1156:        self.opkg_args = "--volatile-cache -f %s -t %s -o %s " % (self.config_file, self.d.expand('${T}/ipktemp/') ,target_rootfs)
     1157:        self.opkg_args += self.d.getVar("OPKG_ARGS")
     1158:
     1159:        if prepare_index:
 *** 1160:            create_packages_dir(self.d, self.deploy_dir, d.getVar("DEPLOY_DIR_IPK"), "package_write_ipk", filterbydependencies)
     1161:
     1162:        opkg_lib_dir = self.d.getVar('OPKGLIBDIR')
     1163:        if opkg_lib_dir[0] == "/":
     1164:            opkg_lib_dir = opkg_lib_dir[1:]
File: '/home/build/releases/master/poky/meta/lib/oe/package_manager.py', lineno: 711, function: create_packages_dir
     0707:                if destdir not in seendirs:
     0708:                    bb.utils.mkdirhier(destdir)
     0709:                    seendirs.add(destdir)
     0710:                try:
 *** 0711:                    os.link(l, dest)
     0712:                except OSError as err:
     0713:                    if err.errno == errno.EXDEV:
     0714:                        bb.utils.copyfile(l, dest)
     0715:                    else:
Exception: FileExistsError: [Errno 17] File exists: '/home/build/builds/master/tmpmulti-host-glibc/deploy/ipk/aarch64/base-passwd-dev_3.5.29-r0_aarch64.ipk' -> '/home/build/builds/master/tmpmulti-host-glibc/work/odroid_c2-oe-linux/host-image-minimal/1.0-r0/oe-rootfs-repo/aarch64/base-passwd-dev_3.5.29-r0_aarch64.ipk'

ERROR: mc:host:host-image-minimal-1.0-r0 do_rootfs: Function failed: do_rootfs
ERROR: Logfile of failure stored in: /home/build/builds/master/tmpmulti-host-glibc/work/odroid_c2-oe-linux/host-image-minimal/1.0-r0/temp/log.do_rootfs.17024
ERROR: Task (multiconfig:host:/home/build/releases/master/meta-demos/meta-multiconfig/recipes-multiconfig/images/host-image-minimal.bb:do_rootfs) failed with exit code '1'
Comment 1 Armin Kuster 2019-01-09 20:15:04 UTC
conf/multiconfig/host.conf

MACHINE="odroid-c2"
TMPDIR = "${TOPDIR}/tmpmulti-host"

conf/multiconf/container.conf
MACHINE="odroid-c2"
TMPDIR = "${TOPDIR}/tmpmulti-container"

conf/local.conf
MACHINE = "odroid-c2"
BBMULTICONFIG = "host container"


RECIPE:
recipes-multiconfig/apps/container-multiconfig_1.0.bb
SUMMARY = "Package container image"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"

do_fetch[noexec] = "1"
do_compile[noexec] = "1"

do_install[mcdepends] = "multiconfig:host:container:container-image-minimal:do_image_complete"

do_install () {
    install -d ${D}${localstatedir}/lib/machines
    install ${TOPDIR}/tmpmulti-container-${TCLIBC}/deploy/images/${MACHINE}/container-image-minimal-${MACHINE}.tar.bz2 ${D}${localstatedir}/lib/machines
}

IMAGES:
SUMMARY = "A minimal container image"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"

IMAGE_FSTYPES = "container"

inherit image

PREFERRED_PROVIDER_virtual/kernel = "linux-dummy"

IMAGE_FEATURES = ""
IMAGE_LINGUAS = ""

NO_RECOMMENDATIONS = "1"
VIRTUAL-RUNTIME_login_manager ?= "busybox"
VIRTUAL-RUNTIME_dev_manager ?= ""
VIRTUAL-RUNTIME_init_manager ?= ""
VIRTUAL-RUNTIME_initscripts ?= ""
VIRTUAL-RUNTIME_keymaps ?= ""

IMAGE_INSTALL = " \
    base-files \
    base-passwd \
    netbase \
    ${VIRTUAL-RUNTIME_login_manager} \
"

SUMMARY = "A small image just capable of allowing a device to boot."

IMAGE_LINGUAS = " "

LICENSE = "MIT"

inherit core-image

IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL}"
IMAGE_INSTALL_append = " container-multiconfig"

COMMAND RUN:
 bitbake multiconfig:host:host-image-minimal
Comment 2 Armin Kuster 2019-01-09 20:16:12 UTC
 bitbake multiconfig:host:container-multiconfig

does build everything correctly and puts the "container" image artifact into a package. So that depends part appears to work.
Comment 3 Armin Kuster 2019-01-09 20:19:17 UTC
built with poke master-next

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=master-next&id=f20cc42cb02f829f9c36c0d94c6a00c852ed0ccb

bitbake: bitbake: runqueue: Use multiconfig name to fetch unihash

Thud fails with the same error so it should not be related to the "hash equivalence"

I have built this with "qemuarm" as the MACHINE for both the host and container configurations.
Comment 4 Armin Kuster 2019-01-09 20:27:09 UTC
I am now trying a fresh clone of poky master-next just to rule out any weird config cruft that may exist in my local.conf
Comment 5 Armin Kuster 2019-01-09 22:17:09 UTC
nope. clean poky fails too.
Comment 6 Armin Kuster 2019-01-09 22:39:52 UTC
if you are using POKY you need to remove the ${TCLIBC} in the path in the container-multiconfig recipe. So yes, I tried both OE/bitback and Poky combos.
Comment 7 Armin Kuster 2019-01-11 19:11:28 UTC
I tested the v1 patches posted and they allowed me to create the image as configured below.

thanks.
Comment 9 Frederik Noe-Sdun 2020-01-16 12:26:14 UTC
Created attachment 4610 [details]
pass all data caches to get_taskhash
Comment 10 Frederik Noe-Sdun 2020-01-16 12:26:56 UTC
Created attachment 4611 [details]
pass data caches to prep_taskhash
Comment 11 Frederik Noe-Sdun 2020-01-16 12:27:28 UTC
Created attachment 4612 [details]
include mcdepend taskhashes
Comment 12 Frederik Noe-Sdun 2020-01-16 12:32:22 UTC
I had a simliar use-case in a customer based project.

To reproduce the issue I used meta/lib/oeqa/selftest/cases/multiconfig.py MultiConfig.test_multiconfig based setup.
If something is add to core-image-minimal it is now included in multiconfig-image-packager.
Comment 13 Frederik Noe-Sdun 2020-01-23 07:13:46 UTC
Sorry. My patches were supposed to be for #13724