Bug 1659

Summary: [Usability] Way to determine what is about to happen (similar to a dry-run)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Mark Hatle <mark.hatle>
Component: devtools / tool chainAssignee: Robert Yang <liezhi.yang>
Status: VERIFIED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: corneliux.stoicescu, jessica.zhang, liezhi.yang, meta.mr.watcher, meta.watcher, poky.bs.watcher, poky.watcher, richard.purdie, sgw, song.liu, zhenfeng.zhao
Version: 1.4   
Target Milestone: 1.4 M5   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Mark Hatle 2011-10-05 12:12:12 UTC
When updating a recipe, or doing a git pull from upstream.  It would be nice to be able to run a command that would evaluate the recipes, use the checksumming, stamps, etc and tell me what is about to run, and why.

This way I can do a pull and if eglibc changed, causing every package in the system to rebuild -- I may simply revert and delay my update to later.

If I'm debugging something and trying to understand if I change libnl, why suddenly 30% of the system is rebuilding -- it might give me a list of things I need to verify still work properly after a build.
Comment 1 Saul Wold 2012-05-01 17:19:38 UTC
Is -n what your looking for?
Comment 2 Mark Hatle 2012-05-02 16:29:15 UTC
-n does not appear to be what I am referring to.

-n will simply do a dry run and tell you what is going to happen, but it won't tell you why.

What I'd like to see is an argument that will say:

do_fetch, run because version/pr changed

do_patch, run because SRC_URI references changed

do_compile, run because chksum of do_compile changed


That way when you do a git pull (or bring in a layer, or...) you can run a command and figure out what is going to happen, and why it's going to happen.. this will let you see if you really want to do it -- or if you are trying to figure out why each time you build some task keeps executing, when you don't think it should.
Comment 3 Robert Yang 2012-07-26 08:13:18 UTC
Hi Richard,

I re-routed this feature to the "Product: Bitbake", would you please help me to take a look at the design? (The Queues in bitbake/lib/bb/runqueue.py are so complicated).

Here is the design doc:

1) This is similar to "bitbake -n", I think that we don't have to add a new option, just make an improvement to the "bitbake -n" is OK, the "bitbake -n" goes through the motions, and show a summary in the end currently (e.g.: bitbake -n core-image-sato):

NOTE: Running noexec task 5459 of 5459 (ID: 10, /buildarea2/lyang1/poky/meta/recipes-sato/images/core-image-sato.bb, do_build)
NOTE: Tasks Summary: Attempted 5459 tasks of which 5457 didn't need to be rerun and all succeeded.

We can the print why it needs to re-run, e.g.:
NOTE: (/buildarea2/lyang1/poky/meta/recipes-sato/images/core-image-sato.bb, do_build) needs re-run xxx

2) Edit check_stamp_task or add a similar function to return the reason why need re-run, and print the it when dry-run (-n)

3) The message will be printed by logger.info in lib/bb/runqueue.py
Comment 4 Richard Purdie 2012-07-26 08:55:48 UTC
I'm going to aim a bit higher here and propose a bitbake-whatchanged script. This would basically:

a) run something like STAMPS=/some/tempdir bitbake xxx -S

This would generate the siginfo files for what bitbake xxx would build (without actually building it).

b) Compare the output in /some/tempdir with that in STAMPS and the hashes availble in the sstate cache.

The default output would probably be something like a count of the recipes that would be built, a count of those installed from sstate and count of things that are unchanged.

Extra parameters to bitbake-whatchanged would allow you to dive into the details of why any given thing was getting rebuilt, down to providing the output of bitbake-diffisgs against the appropriate files.
Comment 5 Robert Yang 2012-08-15 15:36:49 UTC
Re-route it to oe-core since it will go into oe-core/scripts/
Comment 6 Robert Yang 2012-09-24 06:50:12 UTC
Move it to 1.4 since the name of the nativesdk changed.
Comment 8 Stoicescu Cornel 2013-03-22 11:06:19 UTC
Verified in master:9ffee9add2d0380f4025e7a678a166f25a7d659e

Test steps:
1) "bitbake man"
2) add "x11vnc" to "DEPENDS" in "meta/recipes-extended/man/man_1.6g.bb"
3) "bitbake-whatchanged -v man"

Result: Output describes what needs to be done/redone over the previous man build.
Comment 9 Stoicescu Cornel 2013-04-30 10:14:27 UTC
test case created: https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=754