Created attachment 4618 [details] complete temp logs (2020-0129) Environment: Build Configuration: BB_VERSION = "1.44.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "ubuntu-18.04" TARGET_SYS = "aarch64-poky-linux" MACHINE = "bcm958742t" DISTRO = "poky" DISTRO_VERSION = "3.0+snapshot-20200129" TUNE_FEATURES = "aarch64 crypto cortexa72" From "log.do_unpack", I see do_create_source_date_epoch_stamp() is executed and generated correct "SOURCE_DATE_EPOCH: 1579843392". (but this correct SOURCE_DATE_EPOCH is never exported) However, after do_patch(), when do_deploy_source_date_epoch() is executed, from "run.do_deploy_source_date_epoch": export SOURCE_DATE_EPOCH="0" It does not set SOURCE_DATE_EPOCH to the correct EPOCH "1579843392" from do_create_source_date_epoch_stamp(). It's the same for do_configure() and do_compile(): export SOURCE_DATE_EPOCH="0" is first set before do_configure() and do_compile() start.
After some further debugging, it looks like after do_create_source_date_epoch_stamp(), the line 169 ~ line179 are not executed. http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/reproducible_build.bbclass python () { if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1': d.appendVarFlag("do_unpack", "postfuncs", " do_create_source_date_epoch_stamp") epochfile = d.getVar('SDE_FILE') source_date_epoch = "0" if os.path.isfile(epochfile): with open(epochfile, 'r') as f: source_date_epoch = f.read() bb.debug(1, "SOURCE_DATE_EPOCH: %s" % source_date_epoch) d.setVar('SOURCE_DATE_EPOCH', source_date_epoch) }
Does this issue reproduce every time? Were seeing issues which would be caused by something like this but we've yet to be able to trigger it reliably...
Created attachment 4619 [details] rpm-build-date.list
Yes, I could consistently reproduce the issue for my recipe. For Yocto's own recipes, I see most of packages are good, but some of them have the same issue for RPM Build Date. eg. alsa-state (has issue) and alsa-utils (look good) alsa-state-0.2.0-r5.aarch64.rpm Release : r5 Build Date: Wednesday, December 31, 1969 PM04:00:00 PST alsa-states-0.2.0-r5.aarch64.rpm Release : r5 Build Date: Wednesday, December 31, 1969 PM04:00:00 PST alsa-utils-1.2.1-r0.aarch64.rpm Release : r0 Build Date: Thursday, November 14, 2019 AM09:02:29 PST alsa-utils-aconnect-1.2.1-r0.aarch64.rpm Release : r0 Build Date: Thursday, November 14, 2019 AM09:02:29 PST alsa-utils-alsactl-1.2.1-r0.aarch64.rpm Release : r0 Build Date: Thursday, November 14, 2019 AM09:02:29 PST
Created attachment 4620 [details] reproducible_build.bbclass debug verbose patch
Created attachment 4621 [details] brcm-u-boot logs brcm-u-boot.env: bitbake mc:ns3:brcm-u-boot -f -e output brcm-u-boot.log: bitbake mc:ns3:brcm-u-boot -f output (fresh build with empty yocto-cache) temp: bitbake logs for brcm-u-boot
From brcm-u-boot.env: WORKDIR="/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/bcm958742t-poky-linux/brcm-u-boot/gitAUTOINC+749bcf8ae5-r0" From brcm-u-boot.log: epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/bcm958742t-poky-linux/brcm-u-boot/1.0-r0/source-date-epoch/__source_date_epoch.txt epochfile not exist: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/bcm958742t-poky-linux/brcm-u-boot/1.0-r0/source-date-epoch/__source_date_epoch.txt d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) From temp/log.do_unpack: DEBUG: Executing python function do_create_source_date_epoch_stamp sourcedir: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/bcm958742t-poky-linux/brcm-u-boot/gitAUTOINC+749bcf8ae5-r0/git DEBUG: git repository: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/bcm958742t-poky-linux/brcm-u-boot/gitAUTOINC+749bcf8ae5-r0/git/.git SOURCE_DATE_EPOCH: 1579843392 WARNING: cerate SDE_DIR! WARNING: write source_date_epoch=1579843392 WARNING: d.setVar: SOURCE_DATE_EPOCH=1579843392 (from do_create_source_date_epoch_stamp) But from log.do_configure, log.do_compile: SOURCE_DATE_EPOCH=0 SOURCE_DATE_EPOCH=0 from log.do_package_write_rpm: DEBUG: SOURCE_DATE_EPOCH=0
Created attachment 4622 [details] alsa-utils log files alsa-utils has correct SOURCE_DATE_EPOCH=1573750949 alsa-utils.env: (bitbake mc:ns3:alsa-utils -f -e output) WORKDIR="/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-utils/1.2.1-r0" alsa-utils.log: (bitbake mc:ns3:alsa-utils -f output) sourcedir: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-utils/1.2.1-r0/alsa-utils-1.2.1 SOURCE_DATE_EPOCH: 1573750949 WARNING: mc:ns3:alsa-utils-1.2.1-r0 do_unpack: cerate SDE_DIR! WARNING: mc:ns3:alsa-utils-1.2.1-r0 do_unpack: write source_date_epoch=1573750949 WARNING: mc:ns3:alsa-utils-1.2.1-r0 do_unpack: d.setVar: SOURCE_DATE_EPOCH=1573750949 (from do_create_source_date_epoch_stamp) epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-utils/1.2.1-r0/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=1573750949 WARNING: d.setVar: SOURCE_DATE_EPOCH=1573750949 (from anonymous) temp: (bitbake logs for alsa-utils) run.do_configure, run.do_compile: export SOURCE_DATE_EPOCH="1573750949"
Created attachment 4625 [details] alsa-state log files alsa-state has incorrect SOURCE_DATE_EPOCH=0 alsa-state.env: (bitbake mc:ns3:alsa-state -f -e output) WORKDIR="/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5" alsa-state.log: (bitbake mc:ns3:alsa-state -f output) epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt epochfile not exist: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) WARNING: mc:ns3:alsa-state-0.2.0-r5 do_deploy_source_date_epoch_setscene: read from SDE_FILE: SOURCE_DATE_EPOCH=0 WARNING: mc:ns3:alsa-state-0.2.0-r5 do_deploy_source_date_epoch_setscene: d.setVar: SOURCE_DATE_EPOCH=0 (from do_deploy_source_date_epoch_setscene) NOTE: Setscene tasks completed epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 WARNING: d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 WARNING: d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 WARNING: d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) WARNING: mc:ns3:alsa-state-0.2.0-r5 do_unpack: Reusing SOURCE_DATE_EPOCH from: /home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 WARNING: d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) epochfile=/home/rl891036/scrach/SOC-11124-timestamp/ns3-bcm958742t/poky/build/tmp/work/aarch64-poky-linux/alsa-state/0.2.0-r5/source-date-epoch/__source_date_epoch.txt WARNING: read from epochfile: SOURCE_DATE_EPOCH=0 WARNING: d.setVar: SOURCE_DATE_EPOCH=0 (from anonymous) temp: (bitbake logs for alsa-state) run.do_configure, run.do_compile: export SOURCE_DATE_EPOCH="0"
Thank you so much for providing these logs, it has been extremely helpful! I think I might have an idea as to what the problem is. In this specific recipe, there is a bad interaction between the anonymous SOURCE_DATE_EPOCH code and the AUTOINC code. The problem is that the PV variable is set to the default value of "1.0-r1" when the anon python code looks for the source_date_epoch.txt file (this is quite obvious in the logs with your recording patch). The problem is that this recipe uses AUTOREV, which changes PV in an anon python function *after* the SDE check. This means that the do_deploy_source_date_epoch task writes the timestamp out the correct path, as can be seen in the logs. I'm not sure how to fix it exactly yet, but I'm confident we can figure out something now that we know the problem.
Right now, in my specific recipe, I use a work-around similar to kernel class (LINE294) [http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/kernel.bbclass#n294] to re-export "${SOURCE_DATE_EPOCH}", but I re-export based on "${SDE_FILE}" content: do_configure_prepend() { export SOURCE_DATE_EPOCH="$(cat ${SDE_FILE})" } do_compile_prepend() { export SOURCE_DATE_EPOCH="$(cat ${SDE_FILE})" } do_package_write_rpm_prepend() { epochfile = d.getVar('SDE_FILE') if os.path.isfile(epochfile): with open(epochfile, 'r') as my_epoch: source_date_epoch = my_epoch.read() bb.debug(1, f"SOURCE_DATE_EPOCH={source_date_epoch} (read from epochfile)") my_epoch.close() d.setVar('SOURCE_DATE_EPOCH', source_date_epoch) }
I'm kinda new to Yocto, so I have 2 questions. 1. I know in Bitbake-style python function, we could export/update variable back to database "d" through d.setVar() so the variable can be available in subsequent tasks. Are we able to do the same in shell functions? When I export a variable in shell function, its scope seems within that function only. 2. Since I have many recipes using AUCOINC, I tried to put my workaround hack into reproducible_build.bbclass. But I run into issue that some recipes are not using shell function do_configure / do_compile (they are using python), so I got syntax error. Is there a way to prepend do_configure / do_compile differently based on their function type? eg. do_configure_prepend { # for shell } do_configure_prepend { # for python }
(In reply to comment #12) > I'm kinda new to Yocto, so I have 2 questions. > > 1. > I know in Bitbake-style python function, > we could export/update variable back to database "d" through d.setVar() > so the variable can be available in subsequent tasks. Not quite. If you change a variable in a given task, it doesn't propagate to other subsequent tasks (at least AFAIK). > > Are we able to do the same in shell functions? > When I export a variable in shell function, > its scope seems within that function only. > > > 2. > Since I have many recipes using AUCOINC, > I tried to put my workaround hack into reproducible_build.bbclass. > But I run into issue that some recipes are not using shell function > do_configure / do_compile (they are using python), > so I got syntax error. > > Is there a way to prepend do_configure / do_compile differently based on > their function type? > > eg. > do_configure_prepend { # for shell } > do_configure_prepend { # for python } No, but depending on what you need to do, you can add a prefunc that will execute before the given task. Because this executes independently, it can be either shell or python independent of the main task: python do_my_pre_func() { d.setVar('FOO', 'BAR') } do_configure[prefuncs] += "do_my_pre_func" Note that because these execute in the same context as the do_configure task, changes to bitbake variables in the prefunc *will* be visible in do_configure
Patch submitted to ML: http://lists.openembedded.org/pipermail/openembedded-core/2020-January/292273.html If you could give this a try and report back it would be a huge help!
Thank you a lot for the prompt fix. I'll try out your patch today.
>> 1. >> I know in Bitbake-style python function, >> we could export/update variable back to database "d" through d.setVar() >> so the variable can be available in subsequent tasks. > >Not quite. If you change a variable in a given task, it doesn't propagate to >other subsequent tasks (at least AFAIK). Yes, I just verified this. Even using d.setVar in one task to update/create variable, the change won't propagate to next tasks (even through d.getVar).
> http://lists.openembedded.org/pipermail/openembedded-core/2020-January/292273.html I confirm the patch is working for my recipe.
Excellent. Thanks again, this was extremely helpful in tracking down the bug, and this bug has been giving us a lot of failures on the autobuilder so we're really happy to be able to resolve it!
Landed in OE-core as b3313a10a
*** Bug 13754 has been marked as a duplicate of this bug. ***