Bug 13724

Summary: task signature not changing when using mcdepends
Product: [Build System, Metadata & Runtime] BitBake Reporter: Jonathan Richardson <jon.richardson>
Component: bitbakeAssignee: Joshua Watt <JPEWhacker>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: frederik.sdun, JPEWhacker, poky.bs.watcher, poky.watcher, randy.macleod, scott.branden
Version: unspecified   
Target Milestone: 3.2 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Jonathan Richardson 2020-01-03 01:29:59 UTC
We have a recipe with a multiconfig dependency on a task from another recipe:

brcm-atf:
do_compile[depends] += "${@bb.utils.contains("USE_UEFI", "yes", "brcm-edk2:do_deploy", bb.utils.contains("USE_UBOOT", "yes", "brcm-u-boot:do_deploy", bb.utils.contains("USE_TINYBOOT", "yes", "brcm-tiny-boot:do_deploy", "", d), d), d)}"
do_compile[depends] += "${@bb.utils.contains("TRUSTED_MODE", "optee", "brcm-optee-os:do_deploy", "" ,d)}"
do_compile[depends] += "${@bb.utils.contains("USE_FRU", "yes", "brcm-fru:do_deploy", "" ,d)}"

MULTICONFIG_FROM = "${@d.getVar('BBMULTICONFIG').split()[0]}"
MULTICONFIG_TO = "${@d.getVar('BBMULTICONFIG').split()[1]}"
do_compile[mcdepends] += "multiconfig:${MULTICONFIG_FROM}:${MULTICONFIG_TO}:brcm-crmu:do_deploy"

If I change source code in the brcm-crmu recipe, brcm_crmu:do_compile runs and the binary is updated. I would then expect the task signature for brcm-atf:do_compile to become invalidated also so that it can also run. The command run is: bitbake mc:stingray:brcm-atf

brcm-crmu has this do_deploy:
do_deploy() {
    install -D -m 0644 ${S}/scp-fw.bin ${DEPLOYDIR}/scp-fw.bin
}
addtask deploy before do_build after do_compile

When I dump the signature it looks like my task doesn't have a dependency from the mcdepends, though it does cause brcm-crmu do_deploy (and subsequently do_compile) to run successfully. brcm-atf has dependencies for the do_compile[depends], just not the do_compile[mcdepends].

$ bitbake-dumpsig -t brcm-atf do_compile

...

Tasks this task depends on: ['/home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/atf/brcm-atf.bb:do_configure', '/home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/atf/brcm-fru.bb:do_deploy', '/home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/edk2/brcm-edk2.bb:do_deploy']
This task depends on the checksums of files: [('oe-devtool-tree-sha1-brcm-atf', '8640fa2da703f7ee8cfd5bca382e2837')]
Hash for dependent task /home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/atf/brcm-atf.bb:do_configure is 17aa9eeff0ea95157868cdbc2748fa52ce09fcefa3d262d3be4a34eb3e840be7
Hash for dependent task /home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/atf/brcm-fru.bb:do_deploy is 1340b31be11e17920e6cf59a6c4bfe750d84a9e9044d62fc797fef72ce3eabec
Hash for dependent task /home/jon/work/yocto/nxs/poky/build/../meta-brcm/common/recipes-bsp/edk2/brcm-edk2.bb:do_deploy is d356bf1184f51363e218aa83e859a1b3225349ed1725ad3ed6acaa1a66470d45
Computed base hash is c98dc7bd249fc3e4887140da6c645fc78fc5e362611674e1d0a05f9b3659902e and from file c98dc7bd249fc3e4887140da6c645fc78fc5e362611674e1d0a05f9b3659902e
Computed task hash is 214645088b423103d12903cb42d87ff7aa9baaa96d7d42f4835608fc7e50a7a3
Comment 1 Frederik Noe-Sdun 2020-01-23 10:56:22 UTC
I had a simliar use-case in a customer based project.

To reproduce the issue I used meta/lib/oeqa/selftest/cases/multiconfig.py MultiConfig.test_multiconfig based setup.
If something is add to core-image-minimal it is now included in multiconfig-image-packager.

to fix the issues in master I attached patches to https://bugzilla.yoctoproject.org/show_bug.cgi?id=13130 (in a false manner)

Will try to send those to ML soonish ...
Comment 2 Joshua Watt 2020-06-15 09:15:11 UTC
Fixed in 749731a420fb905b6af97ce2909f06b2bcd14fe2 of oe-core and f2062c41693f9f684bdaf2df0a2a08b7f3871026 of bitbake
Comment 3 Joshua Watt 2020-06-15 09:16:29 UTC
Also 8ff9203de4fce9c104c2987d86980c9f34036b97 of bitbake
Comment 4 Jonathan Richardson 2020-06-17 17:45:30 UTC
I cherry picked:

commit f78409bb4a6f689953098862c40de580d6d6d4dc (HEAD)
Author: Joshua Watt <JPEWhacker@gmail.com>
Date:   Tue Jun 9 13:34:14 2020 -0500

    bitbake: bitbake: siggen: Pass all data caches to hash functions

but I still see the same behaviour I reported. Is there a commit I missed? I'm not sure what '8ff9203de4fce9c104c2987d86980c9f34036b97  of bitbake' means.

Thanks.
Comment 5 Scott Branden 2020-06-19 10:01:01 UTC
reopened as issue does not appear to be resolved?
Comment 6 Joshua Watt 2020-06-19 11:26:35 UTC
Can you verify if you are still seeing the behavior on the latest master? I'm pretty sure this is fixed there, and I'd like to eliminate the backport as the cause of the problem
Comment 7 Jonathan Richardson 2020-06-19 11:28:11 UTC
I'll do a full merge next week to the latest in case there was some other dependent changes missed by just cherry picking it. Thanks.
Comment 8 Joshua Watt 2020-06-19 11:38:32 UTC
Thanks. Also please make sure to use the "mc:" prefix everywhere. Using "multiconfig:" is spotty at best:

do_compile[mcdepends] += "mc:..."
Comment 9 Joshua Watt 2020-06-25 07:44:35 UTC
Did you get a chance to test this?
Comment 10 Jonathan Richardson 2020-06-25 11:11:53 UTC
Hi Joshua. Yes, I did a full merge and it does work. Thanks for fixing it. There must have been some quirk with just cherry picking that one change. This can be closed.
Comment 11 Joshua Watt 2020-06-25 11:21:07 UTC
Great. Thanks!