Bug 14923

Summary: devtool.DevtoolUpgradeTests.test_devtool_virtual_kernel_modify fails during 'devtool build linux-yocto'
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Luca Ceresoli <luca.ceresoli>
Component: kernelAssignee: Bruce Ashfield <bruce.ashfield>
Status: RESOLVED WORKSFORME QA Contact:
Severity: normal    
Priority: Medium CC: alexandre.belloni, randy.macleod, richard.purdie, tom.zanussi
Version: unspecified   
Target Milestone: 5.1   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT AB-HAVE-PLAN
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Luca Ceresoli 2022-10-07 21:41:16 UTC
Got this failure once:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4159/steps/14/logs/stdio



Log:

2022-10-07 07:18:00,969 - oe-selftest - INFO - devtool.DevtoolUpgradeTests.test_devtool_virtual_kernel_modify (subunit.RemotedTestCase)
2022-10-07 07:18:00,970 - oe-selftest - INFO -  ... FAIL
Stderr:
2022-10-07 06:51:41,103 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1192384/conf/local.conf
2022-10-07 06:51:41,104 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2022-10-07 07:18:00,970 - oe-selftest - INFO - 5: 13/30 368/503 (1060.51s) (devtool.DevtoolUpgradeTests.test_devtool_virtual_kernel_modify)
2022-10-07 07:18:00,970 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/devtool.py", line 2036, in test_devtool_virtual_kernel_modify
    runCmd('devtool build %s' % kernel_provider)
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'devtool build linux-yocto' returned non-zero exit status 1:

...

ERROR: linux-yocto-5.19.9+git999-r0 do_package: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:split_kernel_module_packages(d)
     0003:
File: '/tmp/devtoolqa7pq7w2jz/core-copy/meta/classes-recipe/kernel-module-split.bbclass', lineno: 184, function: split_kernel_module_packages
     0180:    module_pattern_prefix = d.getVar('KERNEL_MODULE_PACKAGE_PREFIX')
     0181:    module_pattern_suffix = d.getVar('KERNEL_MODULE_PACKAGE_SUFFIX')
     0182:    module_pattern = module_pattern_prefix + kernel_package_name + '-module-%s' + module_pattern_suffix
     0183:
 *** 0184:    modules = do_split_packages(d, root='${nonarch_base_libdir}/modules', file_regex=module_regex, output_pattern=module_pattern, description='%s kernel module', postinst=postinst, postrm=postrm, recursive=True, hook=frob_metadata, extra_depends='%s-%s' % (kernel_package_name, kernel_version))
     0185:    if modules:
     0186:        d.appendVar('RDEPENDS:' + metapkg, ' '+' '.join(modules))
     0187:
     0188:    # If modules-load.d and modprobe.d are empty at this point, remove them to
File: '/tmp/devtoolqa7pq7w2jz/core-copy/meta/classes-global/package.bbclass', lineno: 240, function: do_split_packages
     0236:            d.setVar('pkg_postinst:' + pkg, postinst)
     0237:        if postrm:
     0238:            d.setVar('pkg_postrm:' + pkg, postrm)
     0239:        if callable(hook):
 *** 0240:            hook(f, pkg, file_regex, output_pattern, m.group(1))
     0241:
     0242:    d.setVar('PACKAGES', ' '.join(packages))
     0243:    return list(split_packages)
     0244:
File: '/tmp/devtoolqa7pq7w2jz/core-copy/meta/classes-recipe/kernel-module-split.bbclass', lineno: 92, function: frob_metadata
     0088:            vals[m.group(1)] = m.group(2)
     0089:        return vals
     0090:
     0091:    def frob_metadata(file, pkg, pattern, format, basename):
 *** 0092:        vals = extract_modinfo(file)
     0093:
     0094:        dvar = d.getVar('PKGD')
     0095:
     0096:        # If autoloading is requested, output /etc/modules-load.d/<name>.conf and append
File: '/tmp/devtoolqa7pq7w2jz/core-copy/meta/classes-recipe/kernel-module-split.bbclass', lineno: 74, function: extract_modinfo
     0070:                raise msg
     0071:            cmd = "%sobjcopy -j .modinfo -O binary %s %s" % (d.getVar("HOST_PREFIX") or "", tmpkofile, tmpfile)
     0072:        else:
     0073:            cmd = "%sobjcopy -j .modinfo -O binary %s %s" % (d.getVar("HOST_PREFIX") or "", file, tmpfile)
 *** 0074:        subprocess.check_call(cmd, shell=True)
     0075:        # errors='replace': Some old kernel versions contain invalid utf-8 characters in mod descriptions (like 0xf6, 'รถ')
     0076:        f = open(tmpfile, errors='replace')
     0077:        l = f.read().split("\000")
     0078:        f.close()
File: '/usr/lib/python3.9/subprocess.py', lineno: 373, function: check_call
     0369:    if retcode:
     0370:        cmd = kwargs.get("args")
     0371:        if cmd is None:
     0372:            cmd = popenargs[0]
 *** 0373:        raise CalledProcessError(retcode, cmd)
     0374:    return 0
     0375:
     0376:
     0377:def check_output(*popenargs, timeout=None, **kwargs):
Exception: subprocess.CalledProcessError: Command 'x86_64-poky-linux-objcopy -j .modinfo -O binary /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1192384/tmp/work/qemux86_64-poky-linux/linux-yocto/5.19.9+git999-r0/package/lib/modules/5.19.9-yocto-standard/kernel/sound/pci/oxygen/snd-oxygen-lib.ko /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1192384/tmp/work/qemux86_64-poky-linux/linux-yocto/5.19.9+git999-r0/tmptdmd5vqh' returned non-zero exit status 1.
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-1192384/tmp/work/qemux86_64-poky-linux/linux-yocto/5.19.9+git999-r0/temp/log.do_package.2737236
NOTE: recipe linux-yocto-5.19.9+git999-r0: task do_package: Failed
ERROR: Task (/tmp/devtoolqa7pq7w2jz/core-copy/meta/recipes-kernel/linux/linux-yocto_5.19.bb:do_package) failed with exit code '1'



When I noticed this bug, the log was already gone from the worker.
Comment 1 Richard Purdie 2023-03-02 16:08:46 UTC
We need to ensure devtool is using the subprocess error wrapping to get a better error next time this happens
Comment 2 Randy MacLeod 2024-06-06 14:59:02 UTC
We believe the mtime caching issues in bitbake have resolved this problem.
Re-open if seen again.