Bug 14895

Summary: AB-INT: python module lock issue in importlib
Product: [QA/Testing] Functional (self) Testing Reporter: Alexandre Belloni <alexandre.belloni>
Component: oe-selftestAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED WORKSFORME QA Contact:
Severity: normal    
Priority: Medium CC: alexandre.belloni, randy.macleod
Version: unspecified   
Target Milestone: 5.2   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
bitbake-cookerdaemon.log none

Description Alexandre Belloni 2022-08-17 15:33:18 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1948/steps/14/logs/stdio

oe-selftest debian11-ty-2

Parsing of 934 .bb files complete (0 cached, 934 parsed). 1719 targets, 50 skipped, 0 masked, 0 errors.
2022-08-16 23:34:57,959 - oe-selftest - INFO - 5: 16/25 301/497 (101.64s) (prservice.BitbakePrTests.test_pr_service_ipk_arch_dep)
2022-08-16 23:34:57,959 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/selftest/cases/prservice.py", line 109, in test_pr_service_ipk_arch_dep
    self.run_test_pr_service('m4', 'ipk', 'do_package')
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/selftest/cases/prservice.py", line 62, in run_test_pr_service
    stamp_1 = self.get_task_stamp(package_name, track_task)
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/selftest/cases/prservice.py", line 32, in get_task_stamp
    stampdata = get_bb_var('STAMP', target=package_name).split('/')
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 281, in get_bb_var
    return get_bb_vars([var], target, postconfig)[var]
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 243, in get_bb_vars
    bbenv = get_bb_env(target, postconfig=postconfig)
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 237, in get_bb_env
    return bitbake("-e %s" % target, postconfig=postconfig).output
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 229, in bitbake
    return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 207, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake  -e m4' returned non-zero exit status 1:
NOTE: Reconnecting to bitbake server...
ERROR: Unable to parse Var <OE_IMPORTED[:=]>
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/classes-global/base.bbclass", line 35, in oe_import(d=<bb.data_smart.DataSmart object at 0x7f34038623d0>):
                 # Make a python object accessible from the metadata
    >            bb.utils._context[toimport.split(".", 1)[0]] = __import__(toimport)
             except AttributeError as e:
  File "/home/pokybuild/yocto-worker/oe-selftest/build/meta/lib/oe/package.py", line 8, in <module>:
     import stat
    >import mmap
     import subprocess
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load(name='mmap', import_=<built-in function __import__>)
  File "<frozen importlib._bootstrap>", line 158, in _ModuleLockManager.__enter__()
  File "<frozen importlib._bootstrap>", line 110, in _ModuleLock.acquire()
bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception KeyError: 139861830178624
The variable dependency chain for the failure is: OE_IMPORTED[:=]
Comment 1 Alexandre Belloni 2022-08-17 15:38:58 UTC 
Created attachment 4898 [details]
bitbake-cookerdaemon.log
Comment 2 Alexandre Belloni 2022-08-17 15:45:52 UTC 
build directory saved at autobuilder.yocto.io/pub/failed-builds-data/bug14895
Comment 3 Randy MacLeod 2022-08-18 14:56:34 UTC 
Likely related to:
  https://github.com/python/cpython/issues/91351
There is a PR that we might want to apply as a patch locally.
Comment 4 Alexandre Belloni 2022-09-01 19:07:43 UTC 
https://autobuilder.yoctoproject.org/typhoon/#builders/79/builds/4072/steps/14/logs/stdio

oe-selftest-centos alma9-ty-2
Comment 5 Alexandre Belloni 2022-11-02 20:44:53 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4248/steps/14/logs/stdio

oe-selftest-debian debian11-ty-3
Comment 7 Alexandre Belloni 2023-01-11 22:14:03 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4544/steps/14/logs/stdio
oe-selftest-debian debian11-ty-3
Comment 8 Richard Purdie 2023-01-21 07:23:29 UTC 
Fix merged into python upstream:
https://github.com/python/cpython/pull/94504
Sadly we'll continue to see issues until we switch to python 3.12. We could switch to a newer buildtools tarball on workers where we use one to get the fix earlier for some machines.
Comment 9 Randy MacLeod 2023-03-02 16:01:33 UTC 
This is a bug in the host python. The fix is complicated and won't be backported so we'll leave this defect for a few years until no distro have a vulnerable version: python 3.12.
Comment 10 Alexandre Belloni 2023-06-01 22:04:36 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5205/steps/14/logs/stdio

oe-selftest-fedora fedora37-ty-3
Comment 11 Randy MacLeod 2024-10-24 14:50:50 UTC 
Not seen in months. The fixes may have been back-ported to distro python. 
Re-open if seen again.
Comment 12 Randy MacLeod 2024-10-24 14:51:06 UTC 
See previous comment.