Bug 12342 - lib32-core-image-sato -ctestimage failed due to wrong package names
Summary: lib32-core-image-sato -ctestimage failed due to wrong package names
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.0 M2
Assignee: Saul Wold
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-11-14 01:07 UTC by Robert Yang
Modified: 2024-01-10 22:16 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Yang 2017-11-14 01:07:36 UTC
MACHINE = "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "core2-32"

INHERIT += "testimage"

$ bitbake lib32-core-image-sato -ctestimage

NOTE: ======================================================================
NOTE: FAIL: test_dnf_info (dnf.DnfBasicTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
  File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/workspace2/lyang1/poky/meta/lib/oeqa/runtime/cases/dnf.py", line 41, in test_dnf_info
    self.dnf('info dnf')
  File "/workspace2/lyang1/poky/meta/lib/oeqa/runtime/cases/dnf.py", line 18, in dnf
    self.assertEqual(status, expected, message)
AssertionError: 1 != 0 : dnf info dnf
Error: No matching Packages to list

NOTE: ======================================================================
NOTE: FAIL: test_parselogs (parselogs.ParseLogsTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
  File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/workspace2/lyang1/poky/meta/lib/oeqa/runtime/cases/parselogs.py", line 363, in test_parselogs
    self.assertEqual(errcount, 0, msg=self.msg)
AssertionError: 3 != 0 : Log: /workspace2/lyang1/test_y_ml/tmp/work/qemux86_64-pokymllib32-linux/lib32-core-image-sato/1.0-r0/dnf.log
-----------------------
Central error: 2017-11-13T18:09:03Z CRITICAL Error: No matching Packages to list
***********************
    return cli_run(cli, base)
  File "/usr/lib/python3.5/site-packages/dnf/cli/main.py", line 115, in cli_run
    cli.run()
  File "/usr/lib/python3.5/site-packages/dnf/cli/cli.py", line 1009, in run
    return self.command.run()
  File "/usr/lib/python3.5/site-packages/dnf/cli/commands/__init__.py", line 208, in run
    self.opts.packages)
  File "/usr/lib/python3.5/site-packages/dnf/cli/cli.py", line 475, in output_packages
    raise dnf.exceptions.Error(_('No matching Packages to list'))
dnf.exceptions.Error: No matching Packages to list
Comment 1 Randy MacLeod 2019-06-27 15:29:55 UTC
Chen, I've given this one to Kai. Take it back if you like.
Comment 2 Randy MacLeod 2021-06-23 15:45:12 UTC
Kai is not available so moved to unassigned and M3.
Comment 3 Randy MacLeod 2023-07-26 21:21:59 UTC
Bulk move of 64 bugs to 4.3 M3 after a quick review.
If a bug is actually fixed, please add a commit link and resolve it.
-- Randy for YP bug team.
Comment 4 Trevor Gamblin 2023-09-14 15:44:22 UTC
Moved to M4.
Comment 5 Randy MacLeod 2023-10-30 15:29:10 UTC
Bulk move to 5.0 M1. -- Randy
Comment 6 Randy MacLeod 2023-12-21 23:27:07 UTC
I haven't worked with the test code but I thought I'd try to reproduce this.

I did the following on master:
cat >> conf/local.conf
MACHINE = "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 = "core2-32"
IMAGE_CLASSES += "testimage"

^D

$ bitbake lib32-core-image-sato
$ bitbake lib32-core-image-sato -ctestimage  
Build Configuration:                                                                                                                                                                          
BB_VERSION           = "2.6.1"                                                                 
BUILD_SYS            = "x86_64-linux"                                                          
NATIVELSBSTRING      = "ubuntu-22.04"                                                          
TARGET_SYS           = "x86_64-poky-linux"                                                     
MACHINE              = "qemux86-64"                                                                                                                                                           
DISTRO               = "poky"                                                                  
DISTRO_VERSION       = "4.3+snapshot-e24ff21afb6fbf187ac2af94c33770eb31615eae"                                                                                                                
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "master:e24ff21afb6fbf187ac2af94c33770eb31615eae"
WARNING: Disabling uninative as unable to install uninative tarball: Command 'mkdir -p /ala-lpggp31/rmacleod/src/distro/yocto/b/yp-12342/tmp/sysroots-uninative; cd /ala-lpggp31/rmacleod/src.
WARNING: To build your own uninative loader, please bitbake uninative-tarball and set UNINATIVE_TARBALL appropriately.
Sstate summary: Wanted 0 Local 0 Mirrors 0 Missed 0 Current 435 (0% match, 100% complete)######################################################################                | ETA:  0:00:00
Initialising tasks: 100% |#####################################################################################################################################################| Time: 0:00:03
NOTE: Executing Tasks
QMP Available for connection at /ala-lpggp31/rmacleod/src/distro/yocto/b/yp-12342/tmp/.xpom8kok
QMP connected to QEMU at 12/21/23 14:07:25 and took 0.56 seconds
QMP released QEMU at 12/21/23 14:07:25 and took 0.08 seconds from connect
Test requires apt to be installed
Test requires dropbear, or openssh-sshd to be installed
Test requires autoconf to be installed
Test requires gtk+3 to be installed
Test requires autoconf to be installed
Test requires connman to be installed
Test case connman.ConnmanTest.test_connmand_running depends on connman.ConnmanTest.test_connmand_help but it didn't pass/run.
Test requires coreutils, or busybox to be installed
Test requires coreutils, or busybox to be installed
Test case dnf.DnfBasicTest.test_dnf_help depends on ssh.SSHTest.test_ssh but it didn't pass/run.
Test case dnf.DnfBasicTest.test_dnf_history depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run.
Test case dnf.DnfBasicTest.test_dnf_info depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run.
Test case dnf.DnfBasicTest.test_dnf_search depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run.
Test case dnf.DnfBasicTest.test_dnf_version depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_makecache depends on dnf.DnfBasicTest.test_dnf_help but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_exclude depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_install depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_install_dependency depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_install_from_disk depends on dnf.DnfRepoTest.test_dnf_install_dependency but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_install_from_http depends on dnf.DnfRepoTest.test_dnf_install_from_disk but it didn't pass/run.
Test requires busybox to be installed
Test requires busybox to be installed
Test case dnf.DnfRepoTest.test_dnf_reinstall depends on dnf.DnfRepoTest.test_dnf_install but it didn't pass/run.
Test case dnf.DnfRepoTest.test_dnf_repoinfo depends on dnf.DnfRepoTest.test_dnf_makecache but it didn't pass/run.
Test requires gcc to be installed
Test requires g++ to be installed
Test requires g++ to be installed
Test requires make to be installed
Test requires python3-pygobject to be installed
Test requires go-helloworld to be installed
Test requires gcc to be installed
Test case ldd.LddTest.test_ldd depends on ssh.SSHTest.test_ssh but it didn't pass/run.
Test requires logrotate to be installed
Test case logrotate.LogrotateTest.test_logrotate_newlog depends on logrotate.LogrotateTest.test_logrotate_wtmp but it didn't pass/run.
Test requires busybox-syslog, or rsyslog, or sysklogd, or syslog-ng to be installed
Test case oe_syslog.SyslogTestConfig.test_syslog_logger depends on oe_syslog.SyslogTest.test_syslog_running but it didn't pass/run.
Test case oe_syslog.SyslogTestConfig.test_syslog_restart depends on oe_syslog.SyslogTest.test_syslog_running but it didn't pass/run.
Test requires busybox-syslog to be installed
Test requires opkg to be installed
Test requires shadow-base to be installed
Test case parselogs.ParseLogsTest.test_parselogs depends on ssh.SSHTest.test_ssh but it didn't pass/run.
Test requires perl to be installed
Test requires ptest-runner to be installed
Test requires ptest-runner to be installed
Test requires python3-core to be installed
Test case rpm.RpmBasicTest.test_rpm_help depends on ssh.SSHTest.test_ssh but it didn't pass/run.
Test case rpm.RpmBasicTest.test_rpm_query depends on rpm.RpmBasicTest.test_rpm_help but it didn't pass/run.
Test case rpm.RpmBasicTest.test_rpm_query_nonroot depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run.
Test case rpm.RpmInstallRemoveTest.test_rpm_install depends on rpm.RpmBasicTest.test_rpm_query but it didn't pass/run.
Test case rpm.RpmInstallRemoveTest.test_rpm_remove depends on rpm.RpmInstallRemoveTest.test_rpm_install but it didn't pass/run.
Test case rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size depends on rpm.RpmInstallRemoveTest.test_rpm_remove but it didn't pass/run.
Test requires rust-c-lib-example-bin to be installed
Test requires openssh-scp to be installed
Test requires openssh-scp to be installed
Test requires openssh-scp to be installed
Test requires kernel-devsrc to be installed
Test case systemd.SystemdBasicTests.test_systemd_basic depends on ssh.SSHTest.test_ssh but it didn't pass/run.
Test case systemd.SystemdBasicTests.test_systemd_failed depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run.
Test case systemd.SystemdBasicTests.test_systemd_list depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run.
Test case systemd.SystemdJournalTests.test_systemd_boot_time depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run.
Test case systemd.SystemdJournalTests.test_systemd_journal depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run.
Test requires busybox to be installed
Test case systemd.SystemdServiceTests.test_systemd_status depends on systemd.SystemdBasicTests.test_systemd_basic but it didn't pass/run.
Test case systemd.SystemdServiceTests.test_systemd_disable_enable depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run.
Test is only meant to run with read-only-rootfs in IMAGE_FEATURES
Test case systemd.SystemdServiceTests.test_systemd_stop_start depends on systemd.SystemdServiceTests.test_systemd_status but it didn't pass/run.
Test requires wayland-utils to be installed
Test requires weston to be installed
Test requires weston to be installed
Test requires weston to be installed
Test requires xserver-nodm-init to be installed
Output from runqemu:
runqemu - INFO - Received signal: 15
runqemu - INFO - Cleaning up
runqemu - INFO - Host uptime: 31694068.84

tput: No value for $TERM and no -T specified

RESULTS:
RESULTS - parselogs.ParseLogsTest.test_get_context: PASSED (0.00s)
RESULTS - ping.PingTest.test_ping: PASSED (0.03s)
RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.00s)
RESULTS - buildcpio.BuildCpioTest.test_cpio: SKIPPED (0.00s)
RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s)
RESULTS - buildlzip.BuildLzipTest.test_lzip: SKIPPED (0.00s)
RESULTS - connman.ConnmanTest.test_connmand_help: SKIPPED (0.00s)
RESULTS - connman.ConnmanTest.test_connmand_running: SKIPPED (0.00s)
RESULTS - date.DateTest.test_date: SKIPPED (0.00s)
RESULTS - df.DfTest.test_df: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gcc_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gpp2_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gpp_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_make: SKIPPED (0.00s)
RESULTS - gi.GObjectIntrospectionTest.test_python: SKIPPED (0.00s)
RESULTS - go.GoHelloworldTest.test_gohelloworld: SKIPPED (0.00s)
RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: SKIPPED (0.00s)
RESULTS - ldd.LddTest.test_ldd: SKIPPED (0.00s)
RESULTS - logrotate.LogrotateTest.test_logrotate_newlog: SKIPPED (0.00s)
RESULTS - logrotate.LogrotateTest.test_logrotate_wtmp: SKIPPED (0.00s)
RESULTS - oe_syslog.SyslogTest.test_syslog_running: SKIPPED (0.00s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: SKIPPED (0.00s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: SKIPPED (0.00s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s)
RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: SKIPPED (0.00s)
RESULTS - pam.PamBasicTest.test_pam: SKIPPED (0.00s)
RESULTS - parselogs.ParseLogsTest.test_parselogs: SKIPPED (0.00s)
RESULTS - perl.PerlTest.test_perl_works: SKIPPED (0.00s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: SKIPPED (0.00s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s)
RESULTS - python.PythonTest.test_python3: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s)
RESULTS - rust.RustCLibExampleTest.test_rust_c_lib_example: SKIPPED (0.00s)
RESULTS - rust.RustCompileTest.test_cargo_compile: SKIPPED (0.00s)
RESULTS - rust.RustCompileTest.test_rust_compile: SKIPPED (0.00s)
RESULTS - scp.ScpTest.test_scp_file: SKIPPED (0.00s)
RESULTS - ssh.SSHTest.test_ssh: SKIPPED (0.00s)
RESULTS - stap.StapTest.test_stap: SKIPPED (0.00s)
RESULTS - systemd.SystemdBasicTests.test_systemd_basic: SKIPPED (0.00s)
RESULTS - systemd.SystemdBasicTests.test_systemd_failed: SKIPPED (0.00s)
RESULTS - systemd.SystemdBasicTests.test_systemd_list: SKIPPED (0.00s)
RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: SKIPPED (0.00s)
RESULTS - systemd.SystemdJournalTests.test_systemd_journal: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_coredump_minidebuginfo: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_status: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_wayland_info: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_can_initialize_new_wayland_compositor: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_running: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_supports_xwayland: SKIPPED (0.00s)
RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
SUMMARY:
lib32-core-image-sato () - Ran 73 tests in 1.963s
lib32-core-image-sato - OK - All required tests passed (successes=2, skipped=71, failures=0, errors=0)
NOTE: Tasks Summary: Attempted 1475 tasks of which 1474 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages.
$

so at least that works, albeit while SKIPPING most tests and specifically the dnf ones that this bug is about. 

Adding in conf/local.conf:
PACKAGE_CLASSES ?= "package_rpm"
IMAGE_FEATURES += "package-management ssh-server-openssh"
IMAGE_INSTALL:append = " lib32-dnf"

$ bitbake lib32-core-image-sato
builds and has ssh, sshd, dnf:
qemux86-64 login: root
root@qemux86-64:~# which ssh
/usr/bin/ssh
root@qemux86-64:~# which sshd
/usr/sbin/sshd
root@qemux86-64:~# which dnf
/usr/bin/dnf
root@qemux86-64:~# dnf --version
4.18.1

but the tests are not being run due to:

Test case dnf.DnfBasicTest.test_dnf_help depends on ssh.SSHTest.test_ssh but it didn't pass/run.  
...                                                                                            
RESULTS - ssh.SSHTest.test_ssh: SKIPPED (0.00s)

root@qemux86-64:~#  /usr/sbin/sshd -v
unknown option -- v
OpenSSH_9.5p1, OpenSSL 3.2.0 23 Nov 2023

Tips welcome! 
All for today.
Comment 7 Saul Wold 2023-12-23 15:57:33 UTC
As the title of this bug implies the wrong package names are being used, an initial investigation shows the failure to be in the @OEHasPackage() Decorator which uses the manifest to test (using isdisjoint) against a set of needed_pkgs or unneeded_pkgs and error if they are missing.

In this case the image is a multilib:lib32 image and all the packages in the manifest are prefixed with lib32- do when dropbear, openssh-sshd or dnf are passed in to the set they do not match the lib32-<PN> variation.

One possible fix would be to add the multi-lib prefix (lib32-) to the set of needed_pkgs when MULTILIB is set.

Another solution would be to remove the multilib prefix from the manifest when multilib is set.

I have to determine if it's possible to get to the bitbake 'd' dictionary to figure out if MULTILIB is set at the different times.
Comment 8 Saul Wold 2023-12-23 20:38:30 UTC
(In reply to Saul Wold from comment #7)
> As the title of this bug implies the wrong package names are being used, an
> initial investigation shows the failure to be in the @OEHasPackage()
> Decorator which uses the manifest to test (using isdisjoint) against a set
> of needed_pkgs or unneeded_pkgs and error if they are missing.
> 
> In this case the image is a multilib:lib32 image and all the packages in the
> manifest are prefixed with lib32- do when dropbear, openssh-sshd or dnf are
> passed in to the set they do not match the lib32-<PN> variation.
> 
> One possible fix would be to add the multi-lib prefix (lib32-) to the set of
> needed_pkgs when MULTILIB is set.
> 
> Another solution would be to remove the multilib prefix from the manifest
> when multilib is set.
> 
> I have to determine if it's possible to get to the bitbake 'd' dictionary to
> figure out if MULTILIB is set at the different times.

Ah, I found that 'd' is available in self.case.td and I can get MULTILIBS from there.

Proposed fix:
diff --git a/meta/lib/oeqa/runtime/decorator/package.py b/meta/lib/oeqa/runtime/decorator/package.py
index 8aba3f325b..b6c7f84795 100644
--- a/meta/lib/oeqa/runtime/decorator/package.py
+++ b/meta/lib/oeqa/runtime/decorator/package.py
@@ -38,11 +38,18 @@ class OEHasPackage(OETestDecorator):
         if isinstance(self.need_pkgs, str):
             self.need_pkgs = [self.need_pkgs,]
 
+        ml = self.case.td.get("MULTILIBS").split(':')[1]
         for pkg in self.need_pkgs:
             if pkg.startswith('!'):
-                unneed_pkgs.add(pkg[1:])
+                if ml:
+                    unneed_pkgs.add(ml + "-" + pkg[1:])
+                else:
+                    unneed_pkgs.add(pkg[1:])
             else:
-                need_pkgs.add(pkg)
+                if ml:
+                    need_pkgs.add(ml + "-" + pkg)
+                else:
+                    need_pkgs.add(pkg)
 
         if unneed_pkgs:
             msg = 'Checking if %s is not installed' % ', '.join(unneed_pkgs)