Bug 14525

Summary: rust-llvm isn't reproducible
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Richard Purdie <richard.purdie>
Component: devtools / tool chainAssignee: pgowda.cve
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: alex.kanavin, meta.mr.watcher, meta.watcher, randy.macleod
Version: 3.3   
Target Milestone: 4.0 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
oe-seltest result log
none
oe selftest log without patch none

Description Richard Purdie 2021-08-26 13:32:16 UTC
rust-llvm-liblto and rust-llvm-staticdev don't always reproduce:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210825-kaihham6/

https://autobuilder.yoctoproject.org/typhoon/#/builders/116/builds/595
Comment 1 Richard Purdie 2021-08-26 13:40:07 UTC
Workaround to avoid failures added for now: http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=7d196f4e9a1957db0119952e59c7d0497c128521
Comment 2 Richard Purdie 2021-08-26 13:40:36 UTC
"oeqa/selftest/reproducibile: Exclude rust packages"
Comment 3 Randy MacLeod 2021-08-26 14:51:59 UTC
usually make is file globing things and that's not deterministic.
Comment 4 pgowda.cve 2021-10-05 15:29:42 UTC
Ran the following command on many poky sources ranging from Aug 25(when rust
tests were marked exclude) to current sources.

"oe-selftest -r reproducible"

However, could not see the error message displayed in build-595.
I am running the above test in Debian environment shared by Richard.

Please let me know if anything is amiss from my side to run the test.
Can we include the rust tests in reproducible suite?

Please find attached the oe-selftest log that was run on current sources.
It mentions that rust-llvm-liblto and rust-llvm-staticdev have same rpm, ipk
and deb packages in reproducibleA and reproducibleB.

Thanks,
pgowda
Comment 5 pgowda.cve 2021-10-05 15:31:45 UTC
Created attachment 4837 [details]
oe-seltest result log
Comment 6 Randy MacLeod 2021-10-06 20:05:32 UTC
Naveen, did you revert the commit mentioned in:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14525#c1
Comment 7 pgowda.cve 2021-10-07 04:39:10 UTC
Hi Randy,

The patch which adds rust-llvm-liblto and rust-llvm-staticdev in 
exclude_packages was reverted before running the test.

Please find attached the "oe-selftest-results-without-patch.log" 
that was run without reverting the patch for same source.
Comment 8 pgowda.cve 2021-10-07 04:39:55 UTC
Created attachment 4838 [details]
oe selftest log without patch
Comment 9 Randy MacLeod 2021-10-07 15:15:46 UTC
We need to look at the last few master builds to see if rust builds reproducibly. 
Apparently the exclude does not stop the test but only stops the reproducible build test from failing.
Comment 11 Randy MacLeod 2021-10-08 01:30:33 UTC
Naveen, 

You may have to look at the data under:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20211006-ed3f97d6/


I got here from:
https://autobuilder.yoctoproject.org/typhoon/#/builders/115
and digging through each record looking for "repro-fail".

I looked around a bit and I don't see a rust reproducible build failure.
If you can confirm that by looking at more reproducible build logs, then plase submit a patch to take rust out of the exclude list and RP will schedule a YP AB run to confirm.
Comment 12 pgowda.cve 2021-10-08 08:45:34 UTC
Checked the builds for around 20 days in reproducible tests and the results
are fluctuating. 
https://autobuilder.yoctoproject.org/typhoon/#/builders/115

Some tests FAIL in some builds and are not constants. Even though the rust
tests are excluded, reproducible tests FAIL for many other packages but not regularly. I have also included RUST tests in reproducible testsuite for
poky sources around 2 months and did not notice any failure.
These tests were run under Debain poky-master environment.

Hence, the rust packages are included in reproducible tests and posted as
https://lists.openembedded.org/g/openembedded-core/message/156772