Bug 10948

Summary: buildhistory not reporting srcrev's when using sstate cache
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Jonathan Richardson <jon.richardson>
Component: Scripts and ToolsAssignee: Ed Bartosh <eduard.bartosh>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: eduard.bartosh, ross.burton, scott.branden
Version: 2.2   
Target Milestone: 2.4 M1   
Hardware: x86   
OS: x86   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Jonathan Richardson 2017-01-20 19:27:47 UTC 
I enable buildhistory by adding INHERIT += "buildhistory" to local.conf. When we do a build with the sstate cache disabled, the latest_srcrev files are written properly to the buildhistory directory and running buildhistory-collect-srcrevs returns the SRCREV for all our recipes using AUTOREV. When we run the same commands with a populated sstate cache, there is almost never latest_srcrev files written in buildhistory and the buildhistory-collect-srcrevs returns nothing. Sometimes it returns one the SRCREV of one recipe - not sure why.

I tried deleting build/tmp and build/cache but it makes no difference.

sstate cache is enabled with SSTATE_DIR and SSTATE_MIRRORS populated. When disabled, neither are specified.

Thanks.
Comment 1 Jonathan Richardson 2017-01-20 19:50:57 UTC 
A bit more info on what we're compiling. When we do a build we are generating a custom image with many recipes included in the image that use AUTOREV. It also builds several bootloaders such as u-boot, plus a couple of others, built at the same time the image is generated by using EXTRA_IMAGEDEPENDS in our machine conf. Even when doing a bitbake <recipe> -c cleanall and rebuilding I don't get SRCREV info.
Comment 2 Ed Bartosh 2017-03-08 12:21:49 UTC 
Couple of observations and questions

- you can't disable sstate cache by commenting out SSTATE_DIR. This will just make sstate to use default location of sstate-cache, which is "${TOPDIR}/sstate-cache"

- it looks like the difference comes from SSTATE_MIRRORS, i.e. buildhistory-collect-srcrevs doesn't show AUTOREV revisions when SSTATE_MIRRORS are enabled. Please, confirm it.

- Can you show the value of SSTATE_MIRRORS please?
Comment 3 Jonathan Richardson 2017-03-09 01:59:32 UTC 
Hi Ed. Yes, correct on both points. I can confirm that buildhistory-collect-srcrevs doesn't show AUTOREV revisions when SSTATE_MIRRORS is enabled. 

The value of the variable to enable sstate is:
SSTATE_MIRRORS="file://.* file:///projects/sw/yocto/mirror/sstate-cache/PATH"
Comment 4 Stephen K Jolley 2017-03-17 17:25:49 UTC 
It appears the question was answered.
Comment 5 Ed Bartosh 2017-03-29 14:07:15 UTC 
I'm not able to get any output from buildhistory-collect-srcrevs even with commented out SSTATE_MIRRORS line in local.conf

Jonathan, can you suggest some way to reproduce this issue?
Comment 6 Jonathan Richardson 2017-04-01 00:22:13 UTC 
Not sure why you don't get any output if you're using autorev. I put together a simple test. If you need a real recipe I'll find one I can share and modify it exactly, but I'll assume you have some recipe pointing to a git repo using autorev:

my-recipe.bb:
SRCREV = "${AUTOREV}"
SRC_URI = "git://somewhere.com/repo.git;protocol=ssh;branch=master;"
# Force access to git repo every time.
PV="git${SRCPV}"

1) In build/conf/local.conf add:

INHERIT += "buildhistory"

# I have these variables in (conf/distro/include/my-distro-defaults.inc), but local.conf should be fine
SSTATE_DIR ?= "${TOPDIR}/../../../new-yocto-cache/sstate-cache"
SSTATE_MIRRORS = ""

2) bitbake my-recipe

3) buildhistory-collect-srcrevs
SRCREV_pn-my-recipe = "2069d0d124c5d325890a91a6c309693c45143ff9"

4) Populate the sstate cache:
  - mkdir ~/mirror
  - cp -fr ../../../new-yocto-cache/sstate-cache /home/user/mirror/

5) Set SSTATE_MIRRORS to the populated sstate cache:
SSTATE_MIRRORS ?= "file://.* file:///home/user/yocto/mirror/sstate-cache/PATH

6) In your build dir:
   rm -rf buildhistory cache tmp

7) bitbake my-recipe

8) buildhistory-collect-srcrevs
<no output>

9) bitbake my-recipe -c cleanall

10) buildhistory-collect-srcrevs
<no output>

11) Remove SSTATE_MIRRRORS:
SSTATE_MIRRORS = ""

12) bitbake my-recipe -c cleanall; bitbake my-recipe

13) buildhistory-collect-srcrevs
SRCREV_pn-my-recipe = "2069d0d124c5d325890a91a6c309693c45143ff9"
Comment 7 Ed Bartosh 2017-04-04 14:42:44 UTC 
Jonathan, thank you for detailed example.

I was able to reproduce this. The reason for it is that latest_srcrev files are written after sources are fetched, i.e. after do_fetch task is executed. This does not happen when sstate mirror already has fetched sources. So, latest_srcrev files are not created and buildhistory-collect-srcrevs reports nothing.
Comment 8 Ed Bartosh 2017-04-05 11:23:09 UTC 
I've noticed one difference in your example:

with enabled SSTATE_MIRRORS you don't build your recipe again after running cleanall task:
9) bitbake my-recipe -c cleanall

with disabled SSTATE_MIRRORS you do:
12) bitbake my-recipe -c cleanall; bitbake my-recipe

when you build the recipe after cleaning do_fetch task is called and write_srcrev is called too and creates latest_srcrev for your recipe. However, if you don't build a recipe it doesn't happen.

Can you try to run bitbake my-recipe when SSTATE_MIRRORS is enabled?

It worked just fine for me:

$ grep ^SSTATE_MIRRORS conf/local.conf 
SSTATE_MIRRORS = "file://.* file:///home/ed/yocto/poky/sstate-cache.mirror/PATH"

$ rm -rf buildhistory cache tmp
$ bitbake lz4 -c cleanall
$ buildhistory-collect-srcrevs 
# core2-64-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"

I'm using modified lz4 recipe. Here is the diff:
--- a/meta/recipes-support/lz4/lz4.bb
+++ b/meta/recipes-support/lz4/lz4.bb
@@ -2,13 +2,15 @@ SUMMARY = "Extremely Fast Compression algorithm"
 DESCRIPTION = "LZ4 is a very fast lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems."
 
 LICENSE = "BSD"
-LIC_FILES_CHKSUM = "file://lib/LICENSE;md5=0b0d063f37a4477b54af2459477dcafd"
+LIC_FILES_CHKSUM = "file://lib/LICENSE;md5=ebc2ea4814a64de7708f1571904b32cc"
 
-SRCREV = "d86dc916771c126afb797637dda9f6421c0cb998"
+#SRCREV = "d86dc916771c126afb797637dda9f6421c0cb998"
+SRCREV = "${AUTOREV}"
 
-PV = "131+git${SRCPV}"
+#PV = "131+git${SRCPV}"
+PV="git${SRCPV}"
 
-SRC_URI = "git://github.com/Cyan4973/lz4.git"
+SRC_URI = "git://github.com/Cyan4973/lz4.git;branch=master"
Comment 9 Ed Bartosh 2017-04-05 11:42:26 UTC 
Sorry, missed 'bitbake lz4' command in previous comment. Here is how it should loook:

$ grep ^SSTATE_MIRRORS conf/local.conf 
SSTATE_MIRRORS = "file://.* file:///home/ed/yocto/poky/sstate-cache.mirror/PATH"

$ rm -rf buildhistory cache tmp
$ bitbake lz4 -c cleanall
$ bitbake lz4
$ buildhistory-collect-srcrevs 
# core2-64-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"
Comment 10 Jonathan Richardson 2017-04-05 20:23:30 UTC 
Hi Ed. Sorry, there were a couple of typos in my instructions. I still see the failure even with lz4.

Step 5 had the wrong directory for the sstate cache. Correction below.
5) Set SSTATE_MIRRORS to the populated sstate cache:
#SSTATE_MIRRORS ?= "file://.* file:///home/user/yocto/mirror/sstate-cache/PATH
SSTATE_MIRRORS ?= "file://.* file:///home/user/mirror/sstate-cache/PATH

Step 9 was meant to re-compile the recipe again.
9) bitbake my-recipe -c cleanall; bitbake my-recipe

Just want to make sure my instructions didn't mess up your test when the sstate-cache was copied to /home/user/mirror and not /home/user/yocto/mirror. I went back and forth enabling/disabling SSTATE_MIRRORS and it consistently fails.
Comment 11 Ed Bartosh 2017-04-06 15:29:36 UTC 
I'm not able to reproduce it in my environment. Below is the set of steps I performed following your instructions.

I used modified lz4 example on the latest poky master (633ad6c9f436f5d2b6ee1a005b697661a054a394)

1) In build/conf/local.conf I have the following variables:
SSTATE_DIR ?= "${TOPDIR}/../sstate-cache"
SSTATE_MIRRORS = ""
...
INHERIT += "buildhistory"

2) bitbake lz4

3) buildhistory-collect-srcrevs
# core2-64-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"

4) Populate the sstate cache:
  - cp -fr ../sstate-cache ../sstate-cache.mirror

5) Set SSTATE_MIRRORS to the populated sstate cache:
SSTATE_MIRRORS = "file://.* file:///home/ed/yocto/poky/sstate-cache.mirror/PATH"

6) In my build dir:
   rm -rf buildhistory cache tmp

7) bitbake lz4

8) buildhistory-collect-srcrevs
<no output>

9) bitbake lz4 -c cleanall

10) buildhistory-collect-srcrevs
buildhistory-collect-srcrevs
# core2-64-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"


Please, try to repeat this with the same repository and revision that I used. It might be that the issue is fixed only in master.
Comment 12 Jonathan Richardson 2017-04-17 21:12:45 UTC 
Thanks, I'll try to reproduce it as soon as I can. Issues beyond my control are preventing me from accessing some of the git repos required when upgrading to the latest. I do see a commit in there that may have fixed it.
Comment 13 Jonathan Richardson 2017-04-21 18:53:47 UTC 
I upgraded poky to 633ad6c9f436f5d2b6ee1a005b697661a054a394. Unfortunately I can't reproduce your results. buildhistory-collect-srcrevs still doesn't return anything. I tried removing my PREMIRRORS_prepend, DL_DIR, and BB_GENERATE_MIRROR_TARBALLS variables also in case those were affecting it.

I tried using SSTATE_MIRRORS set and doing 'bitbake lz4 -c install -f' and it works fine, but it's not surprising because the sstate cache is bypassed.
Comment 14 Stephen K Jolley 2017-04-28 16:31:37 UTC 
It appears the question has been answered.  Moving back to Ed to work.
Comment 15 Ed Bartosh 2017-05-02 07:31:10 UTC 
Sorry for the delay. I was on vacation.

I'll try to reproduce the issue with absolutely clean build on the latest master.
Comment 16 Ed Bartosh 2017-05-02 08:56:57 UTC 
I'm still not able to reproduce this issue even with a clean build on the latest poky master (381897c64069ea43d595380a3ae913bcc79cf7e1).

Here is a detailed list of what I did:

1. Modify lz4 recipe to use AUTOREV:

diff --git a/meta/recipes-support/lz4/lz4.bb b/meta/recipes-support/lz4/lz4.bb
index 03c5a7a..9a8e612 100644
--- a/meta/recipes-support/lz4/lz4.bb
+++ b/meta/recipes-support/lz4/lz4.bb
@@ -2,13 +2,15 @@ SUMMARY = "Extremely Fast Compression algorithm"
 DESCRIPTION = "LZ4 is a very fast lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems."
 
 LICENSE = "BSD"
-LIC_FILES_CHKSUM = "file://lib/LICENSE;md5=0b0d063f37a4477b54af2459477dcafd"
+LIC_FILES_CHKSUM = "file://lib/LICENSE;md5=ebc2ea4814a64de7708f1571904b32cc"
 
-SRCREV = "d86dc916771c126afb797637dda9f6421c0cb998"
+#SRCREV = "d86dc916771c126afb797637dda9f6421c0cb998"
+SRCREV = "${AUTOREV}"
 
-PV = "131+git${SRCPV}"
+#PV = "131+git${SRCPV}"
+PV="git${SRCPV}"
 
-SRC_URI = "git://github.com/Cyan4973/lz4.git"
+SRC_URI = "git://github.com/Cyan4973/lz4.git;branch=master"
 
 S = "${WORKDIR}/git"

2. Create fresh build directory:

$ . oe-init-build-env build-10948
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

3. Add 2 lines to conf/local.conf:

SSTATE_MIRRORS = ""
INHERIT += "buildhistory"

4. Build lz4:

$ bitbake lz4

WARNING: Host distribution "opensuse-13.2" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:20
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "opensuse-13.2"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a
--2017-05-02 11:04:38--  http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2
Resolving proxy-chain.intel.com (proxy-chain.intel.com)... 163.33.7.140
Connecting to proxy-chain.intel.com (proxy-chain.intel.com)|163.33.7.140|:911... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2535308 (2.4M) [application/octet-stream]
Saving to: ‘/home/ed/git/yocto/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’


2017-05-02 11:04:39 (2.99 MB/s) - ‘/home/ed/git/yocto/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’ saved [2535308/2535308]

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: gmp-native-6.1.2-r0 do_fetch: Failed to fetch URL https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2, attempting MIRRORS if available
NOTE: Tasks Summary: Attempted 470 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages shown.

5. Run buildhistory-collect-srcrevs

$ buildhistory-collect-srcrevs
# i586-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"

6. Populate the sstate cache:

$ cp -fr sstate-cache sstate-cache.mirror

7. Set SSTATE_MIRRORS to the populated sstate cache in conf/local.conf:

SSTATE_MIRRORS = "file://.* file:///home/ed/yocto/poky/build-10948/sstate-cache.mirror/PATH"

8. Remove buildhistory, cache and tmp directories:

$ rm -rf buildhistory cache tmp

9. Build lz4 again:

$ bitbake lz4

WARNING: Host distribution "opensuse-13.2" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:20
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "opensuse-13.2"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 470 tasks of which 469 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.

10. Run buildhistory-collect-srcrevs to ensure it doesn't produce anything:

$ buildhistory-collect-srcrevs

11. Clean all

$ bitbake lz4 -c cleanall

WARNING: Host distribution "opensuse-13.2" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:13
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal-4.8"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 3 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.

12. Build lz4 again:

$ bitbake lz4

WARNING: Host distribution "opensuse-13.2" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |#################################################################################################################################################| Time: 0:00:00
Loaded 1296 entries from dependency cache.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:00
Parsing of 830 .bb files complete (829 cached, 1 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal-4.8"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
Checking sstate mirror object availability: 100% |####################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 470 tasks of which 456 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.

13. Run buildhistory-collect-srcrevs again:

$ buildhistory-collect-srcrevs
# i586-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"

Please, try to repeat above steps. It could be that the issue is only reproducible on your machine and/or your host distribution.

If you're able to reproduce my results on default machine (qemux86) you can try to change machine and let me know which machine and distro you've used to reproduce the issue.
Comment 17 Jonathan Richardson 2017-05-16 21:54:12 UTC 
I tried again using qemux86. It still doesn't work for me. It is reproducible on all of our ubuntu 14.04 machines.
Comment 18 Ed Bartosh 2017-05-18 10:58:16 UTC 
ok, thanks. I'll try to reproduce it on Ubuntu 14.04.
Comment 19 Ed Bartosh 2017-05-18 14:28:29 UTC 
I couldn't reproduce the issue on Ubuntu 16.04
it works the same way as on OpenSUSE.

will try Ubuntu 14.04.
Comment 20 Ed Bartosh 2017-05-19 09:10:41 UTC 
It's the same on Ubuntu 14.04 - not reproducible.

I repeated steps that I mentioned here before. The result is the same for the clean qemux86 build on 3 distros: OpenSUSE 13.2, Ubuntu 16.04 and Ubuntu 14.04

I used Ubuntu 14.04and 16.04 docker images from docker hub.

Here is a full console log from Ubuntu 14.04:

[ed@594ce71a39e0 poky (master)]$ . oe-init-build-env build-10948
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ vi conf/local.conf 
[ed@594ce71a39e0 build-10948 (master)]$ vi /tmp/diff
[ed@594ce71a39e0 build-10948 (master)]$ cd ../
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ patch -p1 < /tmp/diff
patching file meta/recipes-support/lz4/lz4.bb
Reversed (or previously applied) patch detected!  Assume -R? [n] ^C
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ 
[ed@594ce71a39e0 poky (master)]$ cd build-10948/
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ vi conf/local.conf 
[ed@594ce71a39e0 build-10948 (master)]$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:20
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "ubuntu-14.04"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a
--2017-05-19 08:07:51--  http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2
Resolving proxy-chain.intel.com (proxy-chain.intel.com)... 163.33.7.139
Connecting to proxy-chain.intel.com (proxy-chain.intel.com)|163.33.7.139|:911... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2535308 (2.4M) [application/octet-stream]
Saving to: ‘/home/ed/git/yocto/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’


2017-05-19 08:07:52 (3.04 MB/s) - ‘/home/ed/git/yocto/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’ saved [2535308/2535308]

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: elfutils-native-0.168-r0 do_fetch: Failed to fetch URL ftp://sourceware.org/pub/elfutils/0.168/elfutils-0.168.tar.bz2, attempting MIRRORS if available
WARNING: expat-native-2.2.0-r0 do_fetch: Failed to fetch URL http://downloads.sourceforge.net/expat/expat-2.2.0.tar.bz2, attempting MIRRORS if available
NOTE: Tasks Summary: Attempted 470 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There were 3 WARNING messages shown.
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ buildhistory-collect-srcrevs
# i586-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ cp -fr sstate-cache sstate-cache.mirror
[ed@594ce71a39e0 build-10948 (master)]$ vi conf/local.conf 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ rm -rf buildhistory cache tmp
[ed@594ce71a39e0 build-10948 (master)]$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:18
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "ubuntu-14.04"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 470 tasks of which 469 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
[ed@594ce71a39e0 build-10948 (master)]$ buildhistory-collect-srcrevs
[ed@594ce71a39e0 build-10948 (master)]$ bitbake lz4 -c cleanall
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:13
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal-4.8"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 3 tasks of which 0 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
[ed@594ce71a39e0 build-10948 (master)]$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |#################################################################################################################################################| Time: 0:00:00
Loaded 1296 entries from dependency cache.
Parsing recipes: 100% |###############################################################################################################################################| Time: 0:00:00
Parsing of 830 .bb files complete (829 cached, 1 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal-4.8"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:381897c64069ea43d595380a3ae913bcc79cf7e1"

Initialising tasks: 100% |############################################################################################################################################| Time: 0:00:00
Checking sstate mirror object availability: 100% |####################################################################################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 470 tasks of which 456 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
[ed@594ce71a39e0 build-10948 (master)]$ buildhistory-collect-srcrevs
# i586-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"


Jonatan, can you try to repeat what I did? I mean to repeat exactly all steps. At this point I'm pretty sure we do different things. That's the only reason I can see why we have different results.

Please, assign this bug back to me when you're done. Thanks.
Comment 21 Jonathan Richardson 2017-06-02 18:48:06 UTC 
I'm not sure why we're seeing different results. I guess I need to debug it on our machines. Something is different. 

I tried again doing exactly what you did. The only part I can't see is what you changed SSTATE_MIRRORS to after this step:

[ed@594ce71a39e0 build-10948 (master)]$ cp -fr sstate-cache sstate-cache.mirror
[ed@594ce71a39e0 build-10948 (master)]$ vi conf/local.conf 
[ed@594ce71a39e0 build-10948 (master)]$ 
[ed@594ce71a39e0 build-10948 (master)]$ 

I switched to the master branch:

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky$ git branch
  bcm/master
* master
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky$ git log
commit 3dea90e8b7bfe988bc4bb70f0f0c50b4f74416bf
Author: Paul Eggleton <paul.eggleton@linux.intel.com>
Date:   Mon May 15 09:29:56 2017 +1200

    bitbake: lib/bb/fetch2: fix typo in conflicting revision message
    
    spcify -> specify
    
    (Bitbake rev: c46f183b3ea94789dd83cb280a1bf5735189c9bc)
    
    Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

I built everything from scratch, it took a couple of hours. Same results. To give you a log I just used my sstate-cache from the first build and setup a new build dir:

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky$ . oe-init-build-env build-10948
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ ll
total 12
drwxrwxr-x  3 jonathar jonathar 4096 Jun  2 11:43 ./
drwxrwxr-x 19 jonathar jonathar 4096 Jun  2 11:43 ../
drwxrwxr-x  2 jonathar jonathar 4096 Jun  2 11:43 conf/
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes:  45% |########################################                                                  | ETA:  0:00:15
Keyboard Interrupt, closing down...

Parsing recipes: 100% |##########################################################################################| Time: 0:00:13
Parsing of 830 .bb files complete (0 cached, 384 parsed). 586 targets, 28 skipped, 0 masked, 0 errors.

Summary: There was 1 WARNING message shown.
Execution was interrupted, returning a non-zero exit code.
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |##########################################################################################| Time: 0:00:22
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "ubuntu-14.04"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:3dea90e8b7bfe988bc4bb70f0f0c50b4f74416bf"

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a
--2017-06-02 11:45:13--  http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2
Resolving downloads.yoctoproject.org (downloads.yoctoproject.org)... 198.145.20.127
Connecting to downloads.yoctoproject.org (downloads.yoctoproject.org)|198.145.20.127|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2535308 (2.4M) [application/octet-stream]
Saving to: ‘/home/jonathar/work/yocto/buildhistory-issue/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’


2017-06-02 11:45:15 (1.21 MB/s) - ‘/home/jonathar/work/yocto/buildhistory-issue/poky/build-10948/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2’ saved [2535308/2535308]

Initialising tasks: 100% |#######################################################################################| Time: 0:00:01
Checking sstate mirror object availability: 100% |###############################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 857 tasks of which 856 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ buildhistory-collect-srcrevs
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ 

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ tail conf/local.conf
#ASSUME_PROVIDED += "libsdl-native"

# CONF_VERSION is increased each time build/conf/ changes incompatibly and is used to
# track the version of this file when it was generated. This can safely be ignored if
# this doesn't mean anything to you.
CONF_VERSION = "1"

SSTATE_MIRRORS="file://.* file:///home/jonathar/mirror/sstate-cache/PATH"
INHERIT += "buildhistory"
Comment 22 Jonathan Richardson 2017-06-02 19:38:18 UTC 
I added this first line to buildhistory.bbclass to get it to work. If it's cached and do_fetch doesn't run then how is it supposed to write the latest_srcrev file?

do_populate_lic_setscene[postfuncs] += "write_srcrev"
do_fetch[postfuncs] += "write_srcrev"
do_fetch[vardepsexclude] += "write_srcrev"
python write_srcrev() {

When it's cached the do_fetch task isn't run:

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ cat tmp/work/i586-poky-linux/lz4/1_gitAUTOINC+7bb64ff2b6-r0/temp/log.task_order 
do_populate_lic_setscene (12787): log.do_populate_lic_setscene.12787
do_package_write_rpm_setscene (12927): log.do_package_write_rpm_setscene.12927
do_populate_sysroot_setscene (12934): log.do_populate_sysroot_setscene.12934
do_package_qa_setscene (12957): log.do_package_qa_setscene.12957
do_packagedata_setscene (13248): log.do_packagedata_setscene.13248
Comment 23 Ed Bartosh 2017-06-03 21:08:40 UTC 
do_fetch was rerun on all my tests mentioned above after bitbake lz4 -c cleanall. It's still a mistery for me why it's not rerun in your setup.
Comment 24 Ed Bartosh 2017-06-07 14:10:27 UTC 
Can you try this patch, please?

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 3e907fc..057a3cb 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -336,6 +336,7 @@ def write_recipehistory(rcpinfo, d):
         f.write(u"PACKAGES = %s\n" %  rcpinfo.packages)
         f.write(u"LAYER = %s\n" %  rcpinfo.layer)
 
+    write_latest_srcrev(d, pkghistdir)
 
 def write_pkghistory(pkginfo, d):
     bb.debug(2, "Writing package history for package %s" % pkginfo.name)
@@ -853,7 +854,10 @@ def _get_srcrev_values(d):
 do_fetch[postfuncs] += "write_srcrev"
 do_fetch[vardepsexclude] += "write_srcrev"
 python write_srcrev() {
-    pkghistdir = d.getVar('BUILDHISTORY_DIR_PACKAGE')
+    write_latest_srcrev(d, d.getVar('BUILDHISTORY_DIR_PACKAGE'))
+}
+
+def write_latest_srcrev(d, pkghistdir):
     srcrevfile = os.path.join(pkghistdir, 'latest_srcrev')
 
     srcrevs, tag_srcrevs = _get_srcrev_values(d)
@@ -891,4 +895,3 @@ python write_srcrev() {
     else:
         if os.path.exists(srcrevfile):
             os.remove(srcrevfile)
-}


I can't reproduce the issue. Hopefully forcing of srcrev writing can fix it.
Comment 25 Jonathan Richardson 2017-06-08 22:28:49 UTC 
The fix works for me. Thanks.

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ rm -rf buildhistory/ cache/ tmp/
  
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ 
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ bitbake lz4
WARNING: Host distribution "ubuntu-14.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |##########################################################################################| Time: 0:00:28
Parsing of 830 .bb files complete (0 cached, 830 parsed). 1299 targets, 48 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.34.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "ubuntu-14.04"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.3"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:3dea90e8b7bfe988bc4bb70f0f0c50b4f74416bf"

Initialising tasks: 100% |#######################################################################################| Time: 0:00:02
Checking sstate mirror object availability: 100% |###############################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 857 tasks of which 856 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ buildhistory-collect-srcrevs
# i586-poky-linux
SRCREV_pn-lz4 = "7bb64ff2b69a9f8367de9ab483cdadf42b4c1b65"
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ cat tmp/work/
i586-poky-linux/    qemux86-poky-linux/ x86_64-linux/       
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ cat tmp/work/i586-poky-linux/
bash/               expat/              glibc/              lz4/                perl/               sqlite3/
bc/                 flex/               glibc-locale/       m4/                 ptest-runner/       xz/
bzip2/              gawk/               libffi/             make/               python/             zlib/
cryptodev-linux/    gcc-runtime/        libgcc/             ncurses/            python3/            
db/                 gdbm/               libtool-cross/      openssl/            readline/           
diffutils/          gettext/            linux-libc-headers/ opkg-utils/         sed/                
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ cat tmp/work/i586-poky-linux/lz4/1_gitAUTOINC+7bb64ff2b6-r0/temp/log.task_order 
do_populate_lic_setscene (22856): log.do_populate_lic_setscene.22856
do_populate_sysroot_setscene (22857): log.do_populate_sysroot_setscene.22857
do_package_write_rpm_setscene (23062): log.do_package_write_rpm_setscene.23062
do_package_qa_setscene (23063): log.do_package_qa_setscene.23063
do_packagedata_setscene (23142): log.do_packagedata_setscene.23142
jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$ tail conf/local.conf
#ASSUME_PROVIDED += "libsdl-native"

# CONF_VERSION is increased each time build/conf/ changes incompatibly and is used to
# track the version of this file when it was generated. This can safely be ignored if
# this doesn't mean anything to you.
CONF_VERSION = "1"

SSTATE_MIRRORS="file://.* file:///home/jonathar/mirror/sstate-cache/PATH"
INHERIT += "buildhistory"

jonathar@lbrmn-lnxub108:~/work/yocto/buildhistory-issue/poky/build-10948$
Comment 26 Ed Bartosh 2017-06-09 11:51:34 UTC 
Patch has been sent for review: http://lists.openembedded.org/pipermail/openembedded-core/2017-June/137836.html
Comment 27 Ed Bartosh 2017-06-13 08:09:44 UTC 
The fix has been merged to poky master.