Bug 15701

Summary:	automate testing a single recipe's reproducibility
Product:	[QA/Testing] Functional (self) Testing	Reporter:	Yoann Congal <yoann.congal>
Component:	oe-selftest	Assignee:	Yoann Congal <yoann.congal>
Status:	IN PROGRESS IMPLEMENTATION ---	QA Contact:
Severity:	enhancement
Priority:	Medium	CC:	antonin.godard, randy.macleod
Version:	unspecified
Target Milestone:	6.2
Hardware:	x86
OS:	Multiple
See Also:	https://bugzilla.yoctoproject.org/show_bug.cgi?id=15898
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Yes (doc changes required)

Description Yoann Congal 2024-12-20 09:45:25 UTC

By default OEQA_REPRODUCIBLE_TEST_TARGET rebuild every dependencies without sstate. One can add these dependencies to OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS but finding the list might be cumbersome (listing DEPENDS is not enough).

Maybe all of this can be fully automated? Something like a "OEQA_REPRODUCIBLE_ONLY_TEST_TARGET" that would :
* compute every dependencies of the recipe (dynamic deps included)
* build these using sstate
* build the recipe with and without sstate and compare output (normal reproducible test)

See discussion in https://lists.yoctoproject.org/g/docs/topic/patch/110186946

Comment 1 Yoann Congal 2025-02-04 00:08:05 UTC

[PATCH 2/2] selftest/reproducible: Add a method to test a single recipe
https://lists.openembedded.org/g/openembedded-core/message/210744

Comment 2 Yoann Congal 2025-02-05 13:23:42 UTC

Merged in https://git.yoctoproject.org/poky/commit/?h=master&id=50a609576e399cd048480cfeab53d5a26129cd03

Still need to document this

Comment 3 Yoann Congal 2025-02-06 16:43:04 UTC

Next step: explore RP's idea to handle dynamic dependencies (coming from PATCHTOOL, PACKAGE_DEPENDS, ...):

In case of a "leaf" target, change the test:
- reproducibleA build using the "main" sstate.
- reproducibleB prebuild *with the target included* with a new "test" sstate directory (using "main" sstate as (PRE?)MIRROR)
- cleansstate for the target recipe (we can do that since we are in "test" sstate directory) 
- reproducibleB build with the "test" sstate (this the recipe sstate has been cleaned-up, it will be rebuilt from scratch)

Comment 4 Antonin Godard 2025-03-11 08:30:22 UTC

(In reply to Yoann Congal from comment #2)
> Merged in
> https://git.yoctoproject.org/poky/commit/
> ?h=master&id=50a609576e399cd048480cfeab53d5a26129cd03
> 
> Still need to document this

A documentation patch was merged with https://git.yoctoproject.org/yocto-docs/commit/?id=417b423ea3d2ba9903d21d930c1982ff2ac6f596.

Thanks Yoann!

Comment 5 Yoann Congal 2025-04-29 14:41:57 UTC

Bulk move to 5.3

Comment 6 Yoann Congal 2025-11-06 10:57:04 UTC

Bulk move to 6.0 M1