Bug 6081

Summary: SSTATE_DIR populated from SSTATE_MIRRORS is incomplete
Product: [Build System, Metadata & Runtime] BitBake Reporter: Christopher Larson <kergoth>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher
Version: unspecified   
Target Milestone: 1.6 M5   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
Sstate archives only in build/sstate-cache, not build2/sstate-cache
none
-S printdiff
none
Log from build/
none
Log from build2/
none
Log from build3/ (interrupted once it started compiling things) none

Description Christopher Larson 2014-03-31 18:33:53 UTC
Created attachment 1874 [details]
Sstate archives only in build/sstate-cache, not build2/sstate-cache

It seems that doing a build using an sstate mirror results in an SSTATE_DIR which is missing archives that are necessary to do a build entirely from sstate. How it manages to successfully complete a build without things it seems to think are required, is a question I'd like to find the answer to.

Reproduction steps:
- source poky/oe-init-build-env build
- bitbake core-image-minimal # populate from scratch
- exit

- source poky/oe-init-build-env build2
- echo "SSTATE_MIRRORS = 'file://.* file://${TOPDIR}/../build/sstate-cache \n '" >conf/auto.conf
- bitbake core-image-base
- find tmp*/stamps/ -type f | grep -Ev '(setscene|do_build|do_package_write|do_rootfs)' # note that there's no output here, it built entirely from sstate using the mirror

- sed -i -e '/SSTATE_MIRRORS/s/^/#/' conf/auto.conf
- bitbake -S printdiff core-image-base # note the many differences found, particularly in do_fetch, but no details
- exit

- source poky/oe-init-build-env build3 # to keep the broken build2 sstate-cache isolated
- echo "SSTATE_MIRRORS = 'file://.* file://${TOPDIR}/../build2/sstate-cache \n '" >conf/auto.conf
- bitbake core-image-minimal # Watch it use a ton of setscenes, but still rebuild nearly everything from scratch

If you compare build/sstate-cache to build2/sstate-cache at a file level, you'll see the files listed in the attachment exist only in the from scratch build, but not in build2.
Comment 1 Christopher Larson 2014-03-31 18:49:55 UTC
Created attachment 1875 [details]
-S printdiff
Comment 2 Christopher Larson 2014-03-31 22:18:55 UTC
Created attachment 1876 [details]
Log from build/
Comment 3 Christopher Larson 2014-03-31 22:19:11 UTC
Created attachment 1877 [details]
Log from build2/
Comment 4 Christopher Larson 2014-03-31 22:19:34 UTC
Created attachment 1878 [details]
Log from build3/ (interrupted once it started compiling things)
Comment 5 Richard Purdie 2014-03-31 23:16:30 UTC
Just to put some observations into the notes:

a) the printdiff shows what it does since there is no intermediate task data in build2 and hence it has no comparison reference point.

b) this bug reproduces simply with "bitbake eglibc" instead of "core-image-minimal" which makes the issue a little easier to work with
Comment 6 Richard Purdie 2014-04-01 09:25:28 UTC
Resolved with http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cd8541bbfc750d6e46b901a81bcc5636390f6122


"bitbake -p; bitbake eglibc -DDD" in build2 and build3, then diffing the result (sedding out the build path differences) helped a lot
Comment 7 Christopher Larson 2014-04-01 16:26:05 UTC
Verified, thanks for the quick fix.