Based on Richards request to setup the Autobuilder for lsb testing via testimage, I attempted a build with DISTRO=poky-lsb and core-image-full-cmdline. When I ran the -c testimage, I got the following failures. This clearly shows that we need some modifications to the test suite for correctly testing LSB images: | ====================================================================== | FAIL: test_pam (oeqa.runtime.pam.PamBasicTest) | ---------------------------------------------------------------------- | Traceback (most recent call last): | File "/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py", line 102, in wrapped_f | return func(*args) | File "/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f | return f(*args) | File "/srv/hdd/poky/meta/lib/oeqa/runtime/pam.py", line 25, in test_pam | self.assertEqual(status, 0, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output)) | AssertionError: 127 != 0 : useradd command does not work as expected. Status and output:127 and sh: useradd: command not found | | ====================================================================== | FAIL: test_skeleton_availability (oeqa.runtime.skeletoninit.SkeletonBasicTest) | ---------------------------------------------------------------------- | Traceback (most recent call last): | File "/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f | return f(*args) | File "/srv/hdd/poky/meta/lib/oeqa/runtime/skeletoninit.py", line 19, in test_skeleton_availability | self.assertEqual(status, 0, msg = "skeleton init script not found. Output:\n%s " % output) | AssertionError: 2 != 0 : skeleton init script not found. Output: | ls: cannot access /etc/init.d/skeleton: No such file or directory | | ====================================================================== | FAIL: test_skeleton_script (oeqa.runtime.skeletoninit.SkeletonBasicTest) | ---------------------------------------------------------------------- | Traceback (most recent call last): | File "/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py", line 102, in wrapped_f | return func(*args) | File "/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f | return f(*args) | File "/srv/hdd/poky/meta/lib/oeqa/runtime/skeletoninit.py", line 29, in test_skeleton_script | self.assertEqual(status, 0, msg = "Skeleton script could not be started:\n%s\n%s" % (output1, output2)) | AssertionError: 1 != 0 : Skeleton script could not be started: | sh: /etc/init.d/skeleton: No such file or directory | | | ---------------------------------------------------------------------- | Ran 21 tests in 36.737s | This will block the proper implementation of this on the Autobuilder as all LSB images will be red, therefore I am marking this as a Critical High for M3.
A minor note - core-image-full-cmdline is not an LSB image. It is used as the basis for LSB images though.
It seems that the oeRuntimeTest.hasPackage("service") matches a package group: packagegroup-core-full-cmdline-sys-services, not the meta-skeleton service package, so part of the problem might be that hasPackage() needs to exact matching or use some regular expression format.
It does look like it's doing a re.search() on the entire package list file contents - which seems wrong to me especially given how it is being used in the various tests. It ought to be doing a re.match() on each item individually if we even need regex support.
i made a test by replacing re.search() with re.match(). This time testimage signals skipped packages and tests go on as expected. Still i spotted an issue with useadd but i guess this is not related to this bug: ====================================================================== | FAIL: test_pam (oeqa.runtime.pam.PamBasicTest) | ---------------------------------------------------------------------- | Traceback (most recent call last): | File "/media/yp_builds/5poky/meta/lib/oeqa/utils/decorators.py", line 102, in wrapped_f | return func(*args) | File "/media/yp_builds/5poky/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f | return f(*args) | File "/media/yp_builds/5poky/meta/lib/oeqa/runtime/pam.py", line 25, in test_pam | self.assertEqual(status, 0, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output)) | AssertionError: 127 != 0 : useradd command does not work as expected. Status and output:127 and sh: useradd: command not found | | ----------------------------------------------------------------------
related to useradd the resulting image contains luseradd so this is why the related assert to the useradd --help fails
It needs to be more than just a swap of re.search() with re.match() - we also need to do the matching on individual items rather than the whole list.
Hi Paul, I followed your recommendation and did a split by each line. However i got a situation similar to the initial one for syslog. I built the same image as on comment#1 and in the .manifest file i can find busybox-syslog. So even if the service exists, testimage is not treating it: "skipped 'syslog: No syslog package in image'" Should i add an exception for busybox-* ? Thanks
We don't want a hardcoded exception here - if the test genuinely works for both syslog and busybox-syslog then you will have to check *in the test* for both packages.
patch submitted upstream
patch merged: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=393bd7496d71eb101f21234c1233a2d18fd2c73e
Verified on master: 71b0568fa43285f0946fae93fb43cea5f3bbecec