Bug 12597

Summary:	ptest: UPDATE_ALTERNATIVES ignored
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Juro Bystricky <juro.bystricky>
Component:	core	Assignee:	Mariano Lopez <just.another.mariano>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	just.another.mariano, meta.mr.watcher, meta.watcher, randy.macleod, ross.burton, stephano
Version:	2.5
Target Milestone:	4.99
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Juro Bystricky 2018-03-12 14:41:31 UTC

The purpose of ptest is to test a cross-compiled package on a target platform.
Due to UPDATE_ALTERNATIVES, sometimes we end up testing utilities belonging to another package (which may have a different ptest implemented). For example, a core-image-minimal has three different implementations of "kill". Testing "busybox" will test "kill" as a part of the test suite. In my experience this will fail, as we may test "kill" provided by util-linux instead of the one provided by busybox. Alternatively, the test may pass (false positive) where the "real" busybox kill would have failed.

So, I believe all recipes with both ptest and UPDATE_ALTERNATIVES should be revisited and the corresponding run-ptest routines should be something like:

1. save current UPDATE_ALTERNATIVES as used by this package
2. set UPDATE_ALTERNATIVES to the ones provided by this package
3. do testing as before
4. restore the original UPDATE_ALTERNATIVES

Comment 1 Ross Burton 2018-03-15 08:06:27 UTC

Alternatively, change the test suite to call kill.busybox instead of kill?  Not sure what is less invasive.

Fiddling alternatives will mean the tests can't be parallelised but I suspect that on usual hardware that isn't a massive problem anyway.

Comment 2 Mariano Lopez 2019-04-16 03:43:43 UTC

Juro, Ross,

I have implemented this feature using symlinks to call the "correct" binaries that were modified by update-alternatives when the rootfs was generated.

The complete process is as follows:

1. Set PTEST_BINDIR to "1" in order to generate a bin directory within the ptest directory that will be created in the rootfs. This bin directory has symlinks to the binaries renamed by update-alternatives.
2. Modify the run-ptest script to include the generated bin directory as the first directory of PATH.
3. Execute the ptest using the new PATH.

You can check the patches here:

1. Change to the ptest class:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=6995ec65e8019dc6fdf96fc9d508ba600b4551d9

2. Change to the util-linux recipe/ptest script:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=db9a0dd7849edcfa1ae4b56d0c991997b850e913

I also changed busybox as Ross requested in the Mailing List:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=677bacd446247d9a83401a516352b214d384dfd3

Is there something else to be implemented as part of this bug?

Comment 3 Mariano Lopez 2019-04-24 03:26:35 UTC

I'll mark this as resolved and if there is something else to be added, we can reopen this bug.