Bug 12291 - libc6: build not deterministic
Summary: libc6: build not deterministic
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.4.1
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.5
Assignee: Juro Bystricky
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-11-01 20:30 UTC by Juro Bystricky
Modified: 2018-02-05 14:18 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
diffoscope output demonstrating the bug. (407.63 KB, text/html)
2017-11-01 20:30 UTC, Juro Bystricky
no flags Details
diffoscope output demonstrating the bug. libc6-staticdev (375.45 KB, text/html)
2017-11-01 20:32 UTC, Juro Bystricky
no flags Details
diffoscope output demonstrating the bug. libc6-dbg (385.54 KB, text/html)
2017-11-01 20:33 UTC, Juro Bystricky
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Juro Bystricky 2017-11-01 20:30:38 UTC
Created attachment 4112 [details]
diffoscope output demonstrating the bug.

First please see the attached diffoscope generated files comparing various libc6 packages that *should* be identical. 
After some digging in, it is obvious that they were built with two different versions of plural.c. Visual inspection of the two plural.c files confirmed they were different. But how could this be, considering all packages are built from identical sources and plural.c is one of the existing files?

There actually is a reason for this: 
plural.c *may* get regenerated from plural.y, see Makefile:

<snip>
plural.c: plural.y
	$(BISON) $(BISONFLAGS) $@ $^
endif
<snip>

So sometimes plural.c gets generated, sometimes the original file is used,
based on the timestamps of the files plural.c and plural.y.

Digging even deeper, we see this is kind of by design, please see intl.m4 file:

"
  dnl intl/plural.c is generated from intl/plural.y. It requires bison,
  dnl because plural.y uses bison specific features. It requires at least
  dnl bison-2.7 for %define api.pure.
  dnl bison is only needed for the maintainer (who touches plural.y). But in
  dnl order to avoid separate Makefiles or --enable-maintainer-mode, we put
  dnl the rule in general Makefile. Now, some people carelessly touch the
  dnl files or have a broken "make" program, hence the plural.c rule will
  dnl sometimes fire. To avoid an error, defines BISON to ":" if it is not
  dnl present or too old.
...
"
 

AFAIK we do not carelessly touch any files, yet the Makefile rule sometimes fires anyway. I suspect it is because in git the original file mtime stamps are not preserved. 
Anyway, there are several ways to fix this:

1. Somewhere define BISON as ":", or make sure bison is not present on the build host

2. Purposely touch the file:
   A. touch intl/plural.c: file never regenerated 
      or:
   B: touch intl/plural.y: file regenerated iff bison found & bison version is OK.

Based on the above comment, it seems the original plural.c should be used (the file should be regenerated only in "maintainer mode") so the best fix would be 2A.
Comment 1 Juro Bystricky 2017-11-01 20:32:11 UTC
Created attachment 4113 [details]
diffoscope output demonstrating the bug. libc6-staticdev
Comment 2 Juro Bystricky 2017-11-01 20:33:22 UTC
Created attachment 4114 [details]
diffoscope output demonstrating the bug. libc6-dbg
Comment 4 Juro Bystricky 2017-11-30 22:24:17 UTC
Also sent a patch upstream to glibc, accepted:
https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commit;h=1faaf7035cabda101e1d6653bff7a539f201db91
Comment 5 Ross Burton 2018-02-05 14:18:34 UTC
Merged in oe-core b79eeb68c6ad7c92a3e54c6f97a8bf49f0098206.