15288 2023-11-20 21:55:20 +0000 systemtap is not reproducible 2024-11-21 22:04:05 +0000 1 1 7 Build System, Metadata & Runtime OE-Core kernel unspecified x86 Multiple RESOLVED FIXED AB-INT Medium+ normal 5.2 M1 1 alexandre.belloni mathieu.dubois-briand alexandre.belloni randy.macleod richard.purdie ross.burton tom.zanussi victor.kamensky7 --- --- No (bug/feature does not impact docs) oldest_to_newest 97284 0 alexandre.belloni 2023-11-20 21:55:20 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3955/steps/13/logs/stdio https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231120-n7gjv4_t/packages/diff-html/ 97286 1 alexandre.belloni 2023-11-20 22:16:26 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3950/steps/13/logs/stdio 97291 2 alexandre.belloni 2023-11-20 22:41:16 +0000 https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3940/steps/12/logs/stdio 97313 3 richard.purdie 2023-11-23 15:42:16 +0000 release-4.8-74-g418f0a45-dirty vs release-4.9a-dirty 97314 4 richard.purdie 2023-11-23 15:42:44 +0000 from http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231120-n7gjv4_t/packages/diff-html/ 97315 5 randy.macleod 2023-11-23 15:43:40 +0000 Likely useing git describe in the build... 97316 6 ross.burton 2023-11-23 15:47:18 +0000 Yeah that's a `git describe` call thats not reproducible because they pushed a tag. We need a patch to rip out the git describe and just replace it with PV. Ideally, in an upstreamable way, because this is a relatively common pattern and we've a few places where we force the version directly. If we're lucky there's already an option to override the value. 97623 7 bruce.ashfield 2023-12-19 18:48:25 +0000 There's a relatively extensive script that gathers information about the git repository. It isn't easy to replace it with PV, since the script is called from the Makefiles and there's no single variable to override all the callsites. We could always modify the generated version file AFTER configure runs, or we could remove .git before starting the build (I still support the recipe coming directly from git, but this would be a workaround). There's one thing that I'm not understanding / seeing though. Between the two runs that aren't reproducible, the script should be returning the same tag, as the repository is the same, and it isn't using a git hash from a local commit. What is the scenario between the two runs that is triggering the different version ? (all my runs are exactly the same). 97624 8 ross.burton 2023-12-19 20:36:54 +0000 So what's happening is that they're using "git describe" and that isn't stable over time. At one point in time the SHA 418f0a45 was 78 commits ahead of the "release-4.8" tag. Then in a later build another git fetch had been done and that shame SHA is now the "release-4.9a" tag. Thus: shame SHA, different embedded version. 97625 9 ross.burton 2023-12-19 20:40:00 +0000 That's an annoying implementation of git-version, some take a version override from the environment. Might be worth hacking that in and sending it upstream. Alternatively building from a tarball would also work, if they make them. 97627 10 bruce.ashfield 2023-12-19 21:07:58 +0000 (In reply to Ross Burton from comment #8) > So what's happening is that they're using "git describe" and that isn't > stable over time. > > At one point in time the SHA 418f0a45 was 78 commits ahead of the > "release-4.8" tag. Then in a later build another git fetch had been done > and that shame SHA is now the "release-4.9a" tag. > > Thus: shame SHA, different embedded version. That's what I mean. I wouldn't want that to be the same version, as it isn't. I only see this as a problem if the script is pulling in a different answer from the same SRCREV on two different machines, or a different answer on the same machine on two different builds. Just embed the +git in the PV and let the SRCPV background stuff take care of the versioning. 97628 11 bruce.ashfield 2023-12-19 21:09:10 +0000 (In reply to Bruce Ashfield from comment #10) > (In reply to Ross Burton from comment #8) > > So what's happening is that they're using "git describe" and that isn't > > stable over time. > > > > At one point in time the SHA 418f0a45 was 78 commits ahead of the > > "release-4.8" tag. Then in a later build another git fetch had been done > > and that shame SHA is now the "release-4.9a" tag. > > > > Thus: shame SHA, different embedded version. > > That's what I mean. > > I wouldn't want that to be the same version, as it isn't. > > I only see this as a problem if the script is pulling in a different answer > from the same SRCREV on two different machines, or a different answer on the > same machine on two different builds. > > Just embed the +git in the PV and let the SRCPV background stuff take care > of the versioning. I should also say, that the bouncing/moving tag is not something that commonly happens. So I don't think spending a lot of effort to worry about it, is a good use of time. Spending 100163 12 randy.macleod 2024-11-14 15:56:04 +0000 Fix underway/merged. A bit more work to do. 100181 13 richard.purdie 2024-11-15 11:22:59 +0000 (In reply to Bruce Ashfield from comment #10) > (In reply to Ross Burton from comment #8) > > So what's happening is that they're using "git describe" and that isn't > > stable over time. > > > > At one point in time the SHA 418f0a45 was 78 commits ahead of the > > "release-4.8" tag. Then in a later build another git fetch had been done > > and that shame SHA is now the "release-4.9a" tag. > > > > Thus: shame SHA, different embedded version. > > That's what I mean. > > I wouldn't want that to be the same version, as it isn't. > > I only see this as a problem if the script is pulling in a different answer > from the same SRCREV on two different machines, or a different answer on the > same machine on two different builds. > > Just embed the +git in the PV and let the SRCPV background stuff take care > of the versioning. I think there is a bit of confusion here. We're talking about a recipe with the same SRCREV, that doesn't change. What does change is the "git describe" output as upstream adds more commits. If I run it today I can see one output from git describe. If I run it later even on the same checked out revision, the string can change depending on what was pushed to the upstream repo in the meantime. This does break reproducibility as we want stable output. I do therefore think we need to do something to fix this... 100182 14 richard.purdie 2024-11-15 11:26:52 +0000 The version information coming from the script: Makefile.am:GIT_VERSION_CMD = $(SHELL) $(top_srcdir)/git_version.sh Makefile.am: $(srcdir)/git_version.sh -k --srcdir $(srcdir) -o git_version.h Makefile.in:GIT_VERSION_CMD = $(SHELL) $(top_srcdir)/git_version.sh Makefile.in: $(srcdir)/git_version.sh -k --srcdir $(srcdir) -o git_version.h i.e. git_version.sh 100183 15 richard.purdie 2024-11-15 11:31:27 +0000 My proposal is we just patch ${S}/version.h not to use the git codepath. A really simple patch, yes, we'd have to carry it but it should be simple. 100192 16 victor.kamensky7 2024-11-18 04:11:25 +0000 STAP_EXTENDED_VERSION is defined in version.h as #ifdef GIT_REPO #define STAP_EXTENDED_VERSION \ (STAP_EXTRA_VERSION[0] ? GIT_MESSAGE ", " STAP_EXTRA_VERSION : GIT_PRETTY_REV) #else #define STAP_EXTENDED_VERSION \ (STAP_EXTRA_VERSION[0] ? STAP_EXTRA_VERSION : GIT_MESSAGE) #endif STAP_EXTRA_VERSION could be defined with configure --with-extra-version option (i.e --with-extra-version="oe") in this case instead of GIT_PRETTY_REV (that I understand causes problem and comes from 'git describe' will not be used. In this case --version output would be something like this: ./tmp/work/x86_64-linux/systemtap-native/5.2/build/stap --version Systemtap translator/driver (version 5.2/0.191, git branch master, url sourceware.org/git/systemtap.git, commit 3a92ffe673c1621309a0b60892114495b651c9de-dirty, oe) Copyright (C) 2005-2024 Red Hat, Inc. and others This is free software; see the source for copying conditions. tested kernel versions: 3.10 ... 6.12-rc enabled features: BOOST_STRING_REF BPF PYTHON3 LIBSQLITE3 PROLOGUES READLINE MONITOR_LIBS JSON_C LIBDEBUGINFOD As far as I understand it would be reproducible since githash is fixed from SRCREV, and 'dirty' comes from our patches. Is it acceptable solution? If so, I can include adding --with-extra-version="oe" to EXTRA_OECONF along with SystemTap 5.2 update that I am currently testing. 100225 17 victor.kamensky7 2024-11-20 22:45:49 +0000 Fix with --with-extra-version "oe" was merged to master: https://git.openembedded.org/openembedded-core/commit/?id=37fb7c4e469baf53f85319c3cbb75af41a6ddbe7 Please retest. 100247 18 richard.purdie 2024-11-21 22:04:05 +0000 Thanks for the fix Victor, it gets rid of another annoying intermittent issue! :)