Bug 15288

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3950/steps/13/logs/stdio

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3940/steps/12/logs/stdio

release-4.8-74-g418f0a45-dirty vs release-4.9a-dirty

from http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231120-n7gjv4_t/packages/diff-html/

Likely useing git describe in the build...

Yeah that's a `git describe` call thats not reproducible because they pushed a tag. We need a patch to rip out the git describe and just replace it with PV.

Ideally, in an upstreamable way, because this is a relatively common pattern and we've a few places where we force the version directly.  If we're lucky there's already an option to override the value.

There's a relatively extensive script that gathers information about the git repository.

It isn't easy to replace it with PV, since the script is called from the Makefiles and there's no single variable to override all the callsites.

We could always modify the generated version file AFTER configure runs, or we could remove .git before starting the build (I still support the recipe coming directly from git, but this would be a workaround).

There's one thing that I'm not understanding / seeing though.

Between the two runs that aren't reproducible, the script should be returning the same tag, as the repository is the same, and it isn't using a git hash from a local commit.

What is the scenario between the two runs that is triggering the different version ? (all my runs are exactly the same).

So what's happening is that they're using "git describe" and that isn't stable over time.

At one point in time the SHA 418f0a45 was 78 commits ahead of the "release-4.8" tag.  Then in a later build another git fetch had been done and that shame SHA is now the "release-4.9a" tag.

Thus: shame SHA, different embedded version.

That's an annoying implementation of git-version, some take a version override from the environment.  Might be worth hacking that in and sending it upstream.

Alternatively building from a tarball would also work, if they make them.

(In reply to Ross Burton from comment #8)
> So what's happening is that they're using "git describe" and that isn't
> stable over time.
> 
> At one point in time the SHA 418f0a45 was 78 commits ahead of the
> "release-4.8" tag.  Then in a later build another git fetch had been done
> and that shame SHA is now the "release-4.9a" tag.
> 
> Thus: shame SHA, different embedded version.

That's what I mean.

I wouldn't want that to be the same version, as it isn't.

I only see this as a problem if the script is pulling in a different answer from the same SRCREV on two different machines, or a different answer on the same machine on two different builds.

Just embed the +git in the PV and let the SRCPV background stuff take care of the versioning.

(In reply to Bruce Ashfield from comment #10)
> (In reply to Ross Burton from comment #8)
> > So what's happening is that they're using "git describe" and that isn't
> > stable over time.
> > 
> > At one point in time the SHA 418f0a45 was 78 commits ahead of the
> > "release-4.8" tag.  Then in a later build another git fetch had been done
> > and that shame SHA is now the "release-4.9a" tag.
> > 
> > Thus: shame SHA, different embedded version.
> 
> That's what I mean.
> 
> I wouldn't want that to be the same version, as it isn't.
> 
> I only see this as a problem if the script is pulling in a different answer
> from the same SRCREV on two different machines, or a different answer on the
> same machine on two different builds.
> 
> Just embed the +git in the PV and let the SRCPV background stuff take care
> of the versioning.

I should also say, that the bouncing/moving tag is not something that commonly happens.

So I don't think spending a lot of effort to worry about it, is a good use of time.

Spending

Fix underway/merged. A bit more work to do.

(In reply to Bruce Ashfield from comment #10)
> (In reply to Ross Burton from comment #8)
> > So what's happening is that they're using "git describe" and that isn't
> > stable over time.
> > 
> > At one point in time the SHA 418f0a45 was 78 commits ahead of the
> > "release-4.8" tag.  Then in a later build another git fetch had been done
> > and that shame SHA is now the "release-4.9a" tag.
> > 
> > Thus: shame SHA, different embedded version.
> 
> That's what I mean.
> 
> I wouldn't want that to be the same version, as it isn't.
> 
> I only see this as a problem if the script is pulling in a different answer
> from the same SRCREV on two different machines, or a different answer on the
> same machine on two different builds.
> 
> Just embed the +git in the PV and let the SRCPV background stuff take care
> of the versioning.

I think there is a bit of confusion here. We're talking about a recipe with the same SRCREV, that doesn't change.

What does change is the "git describe" output as upstream adds more commits.

If I run it today I can see one output from git describe. If I run it later even on the same checked out revision, the string can change depending on what was pushed to the upstream repo in the meantime.

This does break reproducibility as we want stable output.

I do therefore think we need to do something to fix this...

The version information coming from the script:

Makefile.am:GIT_VERSION_CMD = $(SHELL) $(top_srcdir)/git_version.sh
Makefile.am:	$(srcdir)/git_version.sh -k --srcdir $(srcdir) -o git_version.h
Makefile.in:GIT_VERSION_CMD = $(SHELL) $(top_srcdir)/git_version.sh
Makefile.in:	$(srcdir)/git_version.sh -k --srcdir $(srcdir) -o git_version.h

i.e. git_version.sh

My proposal is we just patch ${S}/version.h not to use the git codepath. A really simple patch, yes, we'd have to carry it but it should be simple.

STAP_EXTENDED_VERSION is defined in version.h as

#ifdef GIT_REPO
#define STAP_EXTENDED_VERSION \
  (STAP_EXTRA_VERSION[0] ? GIT_MESSAGE ", " STAP_EXTRA_VERSION : GIT_PRETTY_REV)
#else
#define STAP_EXTENDED_VERSION \
  (STAP_EXTRA_VERSION[0] ? STAP_EXTRA_VERSION : GIT_MESSAGE)
#endif

STAP_EXTRA_VERSION could be defined with configure --with-extra-version option (i.e --with-extra-version="oe") in this case instead of GIT_PRETTY_REV (that
I understand causes problem and comes from 'git describe' will not be used.

In this case --version output would be something like this:

./tmp/work/x86_64-linux/systemtap-native/5.2/build/stap --version
Systemtap translator/driver (version 5.2/0.191, git branch master, url sourceware.org/git/systemtap.git, commit 3a92ffe673c1621309a0b60892114495b651c9de-dirty, oe)
Copyright (C) 2005-2024 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 3.10 ... 6.12-rc
enabled features: BOOST_STRING_REF BPF PYTHON3 LIBSQLITE3 PROLOGUES READLINE MONITOR_LIBS JSON_C LIBDEBUGINFOD

As far as I understand it would be reproducible since githash is fixed from
SRCREV, and 'dirty' comes from our patches.

Is it acceptable solution? If so, I can include adding --with-extra-version="oe"
to EXTRA_OECONF along with SystemTap 5.2 update that I am currently testing.

Fix with --with-extra-version "oe" was merged to master:
https://git.openembedded.org/openembedded-core/commit/?id=37fb7c4e469baf53f85319c3cbb75af41a6ddbe7
Please retest.

Thanks for the fix Victor, it gets rid of another annoying intermittent issue! :)