As part of our effort to provide reproducible build output we should adopt the SOURCE_DATE_EPOCH spec. Export an appropriately set SOURCE_DATE_EPOCH[1][2] environment variable to the environment builds are performed in. We'll need to ensure we're using appropriate versions of upstream that support SOURCE_DATE_EPOCH and integrating relevant patches from Debian[3] where those aren't already upstream. Separate issues may need to be filed for this piece. 1. https://reproducible-builds.org/specs/source-date-epoch/ 2. https://reproducible-builds.org/docs/timestamps/ 3. https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Reading_the_variable
There is some preliminary work (bottom): https://bugzilla.yoctoproject.org/show_bug.cgi?id=5866
Some of the related patches for reproducibility and SOURCE_DATE_EPOCH enabling are being collected in the distrpatches github organisation: https://github.com/distropatches specifically the reproducible-builds repository: https://github.com/distropatches/reproducible-builds
nice link, seems like we could re-use some of those patches in Yocto recipes.
Basically, we need to be able to set SOURCE_DATE_EPOCH per each recipe. The date should be set to the latest date in the set of files used by the recipe. (This - in theory - should be Changelog, but that is not guaranteed, so different routes need to be taken based on context, i.e. date of the top commit in a git repo etc.)
I attached some initial patches to set SOURCE_DATE_EPOCH. The value for this variable is obtained after source code for a recipe has been unpacked, but before it is patched. If the code comes from a GIT repo, we get the timestamp from the top commit. (This usually corresponds to the mktime of "changelog".) Otherwise we go through all files and get the timestamp from the youngest one. We create a timestamp for each recipe. The timestamp is stored in the file 'src_date_epoch.txt'. Later on, each task reads this file and sets SOURCE_DATE_EPOCH based on the value found in the file. This works surprisingly well for .DEB packages. Building core-image-minimal in two different build folders and performing comparison results in: Same: 3617 Different: 249 Total: 3866 I attached the patches and the comparison shell script "repro-stats.sh" The script is located/called from the "poky" folder, i.e.: $ ./repro-stats.sh build-one build-two deb The other packages (.RPM, IPK) are not as successful, you can see the disappointing results: $ ./repro-stats.sh build-one build-two ipk $ ./repro-stats.sh build-one build-two rpm
Created attachment 3744 [details] patch to define a variable to select/de-select binary reproducible builds
Created attachment 3745 [details] patch to set environment variables, SOURCE_DATE_EPOCH
Created attachment 3746 [details] simple script to get some statistics (same/different/totals)
Probably worth mentioning: in local.conf: PACKAGE_CLASSES = "package_rpm package_deb package_ipk" BUILD_REPRODUCIBLE_BINARIES="1"
Sent patch to the ML: https://patchwork.openembedded.org/patch/143060/ Moving to 2.5 M1, a slightly better version is being tested.
Two patches address this issue: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a5644b6811e7d9db10c72c02fcfdd0930169f23c http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=cf6f3c023cfa731ce061949135e255458d474964