Bug 4021 - no recipe changed, script "bitbake-whatchanged" could also list some tasks need to be updated
Summary: no recipe changed, script "bitbake-whatchanged" could also list some tasks ne...
Status: RESOLVED WONTFIX
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: 1.5
Hardware: x86 Multiple
: Medium normal
Target Milestone: Future
Assignee: Robert Yang
QA Contact:
URL:
Whiteboard:
: 5567 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-03-14 03:19 UTC by Ning Zhang
Modified: 2017-08-17 15:23 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ning Zhang 2013-03-14 03:19:22 UTC 
branch: poky/master
git version: 4da406cd50cf02a63ed76810a670aba1eb561c79

test steps
1. create a new project. source poky/oe-init-build-env 20130313
2. modify conf/local.conf file. I choose 'MACHINE ?= "qemuarm"'
3. execute "bitbake core-image-sato" 
4. execute "bitbake-whatchanged core-image-sato". It should show no task need to be update. But I god 12 tasks need to be updated.
5. recall "bitbake core-image-sato", it rebuild 3 tasks

========Test log:============
wrsadmin@pek-nzhang0-d3:/home/yocto/20130313$ bitbake-whatchanged core-image-sato
Figuring out the STAMPS_DIR ...
Generating the new stamps ... (need several minutes)
=== Newly added tasks: (5 tasks)
  glib-2.0-native: do_install_ptest_base
  core-image-sato: do_rootfs do_install do_configure do_compile

=== Dependencies changed: (7 tasks)
  base-files: do_install do_build do_populate_sysroot do_packagedata do_package do_package_write do_package_write_rpm

=== Summary: (12 changed, 4607 unchanged)
Newly added: 5
PV changed: 0
PR changed: 0
Dependencies changed: 7

Removing the newly generated stamps dir ...
wrsadmin@pek-nzhang0-d3:/home/yocto/20130313$ 

==========end================

6. i do a little analyzation. i used "STAMPS_DIR=/home/yocto/20130313/tmp/stamps1.bbs bitbake -S core-image-sato" to generate the tmp stamps dir, and search task "do_install_ptest_base" in it, could find 3 new tasks. And when i find task "do_install_ptest_base" in dir "/home/yocto/20130313/tmp/stamps", it could list 4 tasks. But one new task "./x86_64-linux/glib-2.0-native/1_2.34.3-r2.do_install_ptest_base" is not exists in "stamps" folder. This is why the script "bitbake-whatchanged" lists "do_install_ptest_base" need to be update.


==========analyze log===================
wrsadmin@pek-nzhang0-d3:/home/yocto/20130313/tmp/stamps1.bbs$ find . -name "*do_install_ptest_base*"
./x86_64-linux/glib-2.0-native/1_2.34.3-r2.do_install_ptest_base.sigdata.0b35b4868a26a9918a89902e57af52bf
./armv5te-poky-linux-gnueabi/glib-2.0/1_2.34.3-r2.do_install_ptest_base.sigdata.a82d3ee907ba49ee262dbf61698cb0ea
./armv5te-poky-linux-gnueabi/bash/4.2-r6.do_install_ptest_base.sigdata.fb8d74112244cdb4ad3a4809c56ba872
wrsadmin@pek-nzhang0-d3:/home/yocto/20130313/tmp/stamps1.bbs$ 

wrsadmin@pek-nzhang0-d3:/home/yocto/20130313/tmp/stamps$ find . -name "*do_install_ptest_base*"
./armv5te-poky-linux-gnueabi/glib-2.0/1_2.34.3-r2.do_install_ptest_base.a82d3ee907ba49ee262dbf61698cb0ea
./armv5te-poky-linux-gnueabi/glib-2.0/1_2.34.3-r2.do_install_ptest_base.sigdata.a82d3ee907ba49ee262dbf61698cb0ea
./armv5te-poky-linux-gnueabi/bash/4.2-r6.do_install_ptest_base.sigdata.fb8d74112244cdb4ad3a4809c56ba872
./armv5te-poky-linux-gnueabi/bash/4.2-r6.do_install_ptest_base.fb8d74112244cdb4ad3a4809c56ba872
wrsadmin@pek-nzhang0-d3:/home/yocto/20130313/tmp/stamps$ 

==============end=======================
Comment 1 Chen Qi 2013-03-15 02:34:15 UTC 
I got the same error on Ubuntu 12.04.
And what's worse, on Ubuntu 11.10, I got this error: bitbake -S core-image-minimal' failed with exit code 1.

Here's the log from Ubuntu 11.10.

Execution of 'STAMPS_DIR=/buildarea3/qchen1/poky/build-qa-warn/tmp/stamps.bbs bitbake -S core-image-minimal' failed with exit code 1:
Loading cache...done.
Loaded 1119 entries from dependency cache.

Build Configuration:
BB_VERSION        = "1.17.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Ubuntu-11.10"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130315"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "master:365eb9b5d22208c967dcd91d8ab52faed20f04b5"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Reparsing files to collect dependency data
ERROR: Bitbake's cached basehash does not match the one we just generated (/buildarea3/qchen1/poky/meta/recipes-core/base-files/base-files_3.0.14.bb.do_install)!
ERROR: The mismatched hashes were 6ceb1b76ecee215d179a82789bd44254 and ccb6805d4d89ab4299616bcc88a96a82
ERROR: Bitbake's cached basehash does not match the one we just generated (/buildarea3/qchen1/poky/meta/recipes-core/images/core-image-minimal.bb.do_rootfs)!
ERROR: The mismatched hashes were 9e0067dfede351cf8b6344fe5a4c5148 and 9bd32c696b47315e63fe608141b3aa1a
NOTE: Tasks Summary: Attempted 0 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There were 4 ERROR messages shown, returning a non-zero exit code.

=== Newly added tasks: (5 tasks)
  core-image-minimal: do_install do_rootfs do_compile do_configure
  glib-2.0-native: do_install_ptest_base

=== Dependencies changed: (7 tasks)
  base-files: do_build do_install do_populate_sysroot do_packagedata do_package do_package_write do_package_write_rpm

=== Summary: (12 changed, 1939 unchanged)
Newly added: 5
PV changed: 0
PR changed: 0
Dependencies changed: 7

Removing the newly generated stamps dir ...
Comment 2 Robert Yang 2013-03-15 02:37:30 UTC 
(In reply to comment #1)
> I got the same error on Ubuntu 12.04.
> And what's worse, on Ubuntu 11.10, I got this error: bitbake -S
> core-image-minimal' failed with exit code 1.
> 

I think that if you pull to the up to date master branch, this error will go.

// Robert

> Here's the log from Ubuntu 11.10.
> 
> Execution of
> 'STAMPS_DIR=/buildarea3/qchen1/poky/build-qa-warn/tmp/stamps.bbs bitbake -S
> core-image-minimal' failed with exit code 1:
> Loading cache...done.
> Loaded 1119 entries from dependency cache.
> 
> Build Configuration:
> BB_VERSION        = "1.17.1"
> BUILD_SYS         = "x86_64-linux"
> NATIVELSBSTRING   = "Ubuntu-11.10"
> TARGET_SYS        = "i586-poky-linux"
> MACHINE           = "qemux86"
> DISTRO            = "poky"
> DISTRO_VERSION    = "1.3+snapshot-20130315"
> TUNE_FEATURES     = "m32 i586"
> TARGET_FPU        = ""
> meta              
> meta-yocto        
> meta-yocto-bsp    = "master:365eb9b5d22208c967dcd91d8ab52faed20f04b5"
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Reparsing files to collect dependency data
> ERROR: Bitbake's cached basehash does not match the one we just generated
> (/buildarea3/qchen1/poky/meta/recipes-core/base-files/base-files_3.0.14.bb.
> do_install)!
> ERROR: The mismatched hashes were 6ceb1b76ecee215d179a82789bd44254 and
> ccb6805d4d89ab4299616bcc88a96a82
> ERROR: Bitbake's cached basehash does not match the one we just generated
> (/buildarea3/qchen1/poky/meta/recipes-core/images/core-image-minimal.bb.
> do_rootfs)!
> ERROR: The mismatched hashes were 9e0067dfede351cf8b6344fe5a4c5148 and
> 9bd32c696b47315e63fe608141b3aa1a
> NOTE: Tasks Summary: Attempted 0 tasks of which 0 didn't need to be rerun
> and all succeeded.
> 
> Summary: There were 4 ERROR messages shown, returning a non-zero exit code.
> 
> === Newly added tasks: (5 tasks)
>   core-image-minimal: do_install do_rootfs do_compile do_configure
>   glib-2.0-native: do_install_ptest_base
> 
> === Dependencies changed: (7 tasks)
>   base-files: do_build do_install do_populate_sysroot do_packagedata
> do_package do_package_write do_package_write_rpm
> 
> === Summary: (12 changed, 1939 unchanged)
> Newly added: 5
> PV changed: 0
> PR changed: 0
> Dependencies changed: 7
> 
> Removing the newly generated stamps dir ...
Comment 3 Chen Qi 2013-03-15 03:27:28 UTC 
(In reply to comment #2)
> (In reply to comment #1)
> > I got the same error on Ubuntu 12.04.
> > And what's worse, on Ubuntu 11.10, I got this error: bitbake -S
> > core-image-minimal' failed with exit code 1.
> > 
> 
> I think that if you pull to the up to date master branch, this error will go.
> 
> // Robert
> 

Exactly. This error is gone after updating to the latest master branch.

Here's the log.
bitbake-whatchanged core-image-minimal
Figuring out the STAMPS_DIR ...
Generating the new stamps ... (need several minutes)
=== Newly added tasks: (5 tasks)
  core-image-minimal: do_install do_rootfs do_compile do_configure
  glib-2.0-native: do_install_ptest_base

=== Summary: (5 changed, 1959 unchanged)
Newly added: 5
PV changed: 0
PR changed: 0
Dependencies changed: 0

Removing the newly generated stamps dir ...
Comment 4 Robert Yang 2013-04-16 06:05:04 UTC 
There some issues exists at the moment:

1) The base-files recipe, it depends on DISTRO_VERSION which contains the DATE, but it doesn't rebuild when we run the "bitbake base-file" in another day. (or adjust the date manually), this causes the /etc/issue is incorrect, for example, if we build base-files in 20130410, and we build the core-image-minimal today, then the /etc/issue will still be 20130410.

2) The glib-2.0-native's do_install_ptest_base doesn't run since:

addtask install_ptest_base   after do_install   before do_package

and the do_package doesn't run when building the native package, we may need fix this.

3) The core-image-sato, the  do_rootfs needs re-run, this is correct, but the do_configure, do_compile or do_install doesn't need to run or re-run, this because the "bitbake -S" doesn't handle the do_task[noexec] correctly.

I'd like to fix them one by one in 1.5.
Comment 5 Robert Yang 2013-04-22 06:49:11 UTC 
Another enhancement is that, can the bitbake-whatchanged prints why a recipe need rebuild ?
Comment 6 Robert Yang 2014-03-25 03:26:56 UTC 
*** Bug 5567 has been marked as a duplicate of this bug. ***
Comment 7 Robert Yang 2014-04-02 09:08:50 UTC 
(In reply to comment #6)
> *** Bug 5567 has been marked as a duplicate of this bug. ***

We have done that we can do in YP 1.6, currently, the following 3 tasks are still reported:

core-image-sato: do_rootfs do_install do_configure do_compile

This might be caused by the stamp creating differences between "bitbake -S" and sstate, need more investigations in YP 1.7, and perhaps we can use the "bitbake -S" to replace the bitbake-whatchanged.
Comment 8 Richard Purdie 2017-08-17 15:23:17 UTC 
At this point bitbake-whatechanged is replaced with the functionality of bitbake -S and bitbake-diffsigs -t. I'd much prefer to focus effort on ehnancing them and maybe bitbake-whatchanged becomes a wrapper around those instead if needed.