Bug 6428

The biggest blocker of this bug at this point is a lack of cases where we need to improve things...

To expand a little on the situation that I think tends to happen - you update the metadata or make some change locally, then run a build, and notice that much more than you expected is getting rebuilt. I know that the last round of improvements in this area focused on bitbake -S printdiff, but I believe that one of the problems with using that in this situation is that if you don't stop the build early enough you've lost the information needed to figure out where the triggering change (the one that caused the first task to re-execute) happened, because sstate has already been written out.

In this instance it's not so much about what the minimal difference from what we could have restored from sstate (which is what bitbake -S printdiff is meant to give you), it's about the difference from what was most recently built, i.e. what was stamped. If I had to suggest what would be ideal it would be a way of printing the difference for stamped tasks that we had to re-run where the trigger for re-running wasn't a cascade from another task or the same change as another task in the dependency chain that had to be re-run. Either we store some context data on each build such that after the build (interrupted or otherwise) you can run some command to print this, or we do this calculation every time and write it to some kind of log that you can read if you want to know what the cause is (assuming it's not too computationally expensive).

I've shared some usability suggestions in https://bugzilla.yoctoproject.org/show_bug.cgi?id=8699

One specific suggestion: fix the cases where "bitbake -t" fails to recurse.

Example:

$ bitbake-diffsigs -t go-cross-powerpc64 do_prepare_recipe_sysroot -s 0db6c0118fdec31604c73b5e8b585f25 d1d7ca0b6b5e7e4cd8002abb37ad534c
Hash for dependent task gcc/gcc-cross_6.3.bb.do_populate_sysroot changed from 7e696e88db395cfc809ed90624ecdbff to 249cf8de3907aae43d270e074641cb9a
Unable to find matching sigdata for /fast/work/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_6.3.bb.do_populate_sysroot with hashes 7e696e88db395cfc809ed90624ecdbff or 249cf8de3907aae43d270e074641cb9a

It fails to recurse because the key passed to recursecb() is /fast/work/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_6.3.bb.do_populate_sysroot (from the 'runtaskhashes' hash in the sigdata file), and that key then isn't "good enough" to locate the sigdata files.

A human has to look at the key and translate it into gcc-cross-powerpc64 do_populate_sysroot to proceed:

$ bitbake-diffsigs -t gcc-cross-powerpc64 do_populate_sysroot -s 7e696e88db395cfc809ed90624ecdbff 249cf8de3907aae43d270e074641cb9a
Hash for dependent task gcc/gcc-cross_6.3.bb.do_install changed from dee590ae5258d9642dc04b9382bcb6f0 to 0d2e43aa746605021e42cf041c0f75ea
Unable to find matching sigdata for /fast/work/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_6.3.bb.do_install with hashes dee590ae5258d9642dc04b9382bcb6f0 or 0d2e43aa746605021e42cf041c0f75ea

These examples come from the bitbake-diffsigs invocations in the new yocto-compat-layer.py script for:
yocto-compat-layer.py --machines b4420qds-64b p5020ds-64b -- .../meta-freescale

Bulk move of 64 bugs to 4.3 M3 after a quick review.
If a bug is actually fixed, please add a commit link and resolve it.
-- Randy for YP bug team.

Moved to M4.

Mistakenly moved this to M4. Since it's an enhancement, it should be 5.0 M1.

Bulk move of 52 5.0-M1 bugs to M2.

Bulk move of bugs owned by "unassigned" from 5.0-M2 to 5.0-M3.

There was a work done by Alex Kanavin with the STF work which significantly improved this and added test cases. I think that fixed the test case here. If there are other specific examples we should improve we should open specific bugs for them.