Bug 15689 - AB-INT: bb.process.CmdError: Cannot allocate memory
Summary: AB-INT: bb.process.CmdError: Cannot allocate memory
Status: RESOLVED WORKSFORME
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 5.3
Assignee: Unassigned
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2024-12-12 09:35 UTC by Mathieu Dubois-Briand
Modified: 2025-05-29 14:54 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mathieu Dubois-Briand 2024-12-12 09:35:16 UTC
>   2117 ERROR: git-native-2.47.0-r0 do_create_package_spdx_setscene: Error executing a python function in exec_func_python() autogenerated:
>   2118 
>   2119 The stack trace of python calls that resulted in this exception/failure was:
>   2120 File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>   2121      0001:
>   2122  *** 0002:do_create_package_spdx_setscene(d)
>   2123      0003:
>   2124 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/meta/classes/create-spdx-3.0.bbclass', lineno: 174, function: do_create_package_spdx_setscene
>   2125      0170:do_create_package_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}"
>   2126      0171:do_create_package_spdx[file-checksums] += "${SPDX3_LIB_DEP_FILES}"
>   2127      0172:
>   2128      0173:python do_create_package_spdx_setscene () {
>   2129  *** 0174:    sstate_setscene(d)
>   2130      0175:}
>   2131      0176:addtask do_create_package_spdx_setscene
>   2132      0177:
>   2133      0178:do_create_package_spdx[dirs] = "${SPDXRUNTIMEDEPLOY}"
>   2134 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/meta/classes-global/sstate.bbclass', lineno: 758, function: sstate_setscene
>   2135      0754:            pass
>   2136      0755:
>   2137      0756:def sstate_setscene(d):
>   2138      0757:    shared_state = sstate_state_fromvars(d)
>   2139  *** 0758:    accelerate = sstate_installpkg(shared_state, d)
>   2140      0759:    if not accelerate:
>   2141      0760:        msg = "No sstate archive obtainable, will run full task instead."
>   2142      0761:        bb.warn(msg)
>   2143      0762:        raise bb.BBHandledException(msg)
>   2144 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/meta/classes-global/sstate.bbclass', lineno: 398, function: sstate_installpkg
>   2145      0394:    d.setVar('SSTATE_FIXMEDIR', ss['fixmedir'])
>   2146      0395:
>   2147      0396:    for f in (d.getVar('SSTATEPREINSTFUNCS') or '').split() + ['sstate_unpack_package']:
>   2148      0397:        # All hooks should run in the SSTATE_INSTDIR
>   2149  *** 0398:        bb.build.exec_func(f, d, (sstateinst,))
>   2150      0399:
>   2151      0400:    return sstate_installpkgdir(ss, d)
>   2152      0401:
>   2153      0402:def sstate_installpkgdir(ss, d):
>   2154 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/build.py', lineno: 262, function: exec_func
>   2155      0258:    with bb.utils.fileslocked(lockfiles):
>   2156      0259:        if ispython:
>   2157      0260:            exec_func_python(func, d, runfile, cwd=adir)
>   2158      0261:        else:
>   2159  *** 0262:            exec_func_shell(func, d, runfile, cwd=adir)
>   2160      0263:
>   2161      0264:    try:
>   2162      0265:        curcwd = os.getcwd()
>   2163      0266:    except:
>   2164 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/build.py', lineno: 529, function: exec_func_shell
>   2165      0525:    with open(fifopath, 'r+b', buffering=0) as fifo:
>   2166      0526:        try:
>   2167      0527:            bb.debug(2, "Executing shell function %s" % func)
>   2168      0528:            with open(os.devnull, 'r+') as stdin, logfile:
>   2169  *** 0529:                bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)])
>   2170      0530:        except bb.process.ExecutionError as exe:
>   2171      0531:            # Find the backtrace that the shell trap generated
>   2172      0532:            backtrace_marker_regex = re.compile(r"WARNING: Backtrace \(BB generated script\)")
>   2173      0533:            stdout_lines = (exe.stdout or "").split("\n")
>   2174 File: '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/bitbake/lib/bb/process.py', lineno: 174, function: run
>   2175      0170:    except OSError as exc:
>   2176      0171:        if exc.errno == 2:
>   2177      0172:            raise NotFoundError(cmd)
>   2178      0173:        else:
>   2179  *** 0174:            raise CmdError(cmd, exc)
>   2180      0175:
>   2181      0176:    if log:
>   2182      0177:        stdout, stderr = _logged_communicate(pipe, log, input, extrafiles)
>   2183      0178:    else:
>   2184 Exception: bb.process.CmdError: Execution of '/srv/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-2771838/tmp/work/x86_64-linux/git-native/2.47.0/temp/run.sstat
>   2184 e_unpack_package.171609' failed: [Errno 12] Cannot allocate memory

https://valkyrie.yoctoproject.org/#/builders/76/builds/521/steps/14/logs/stdio
Comment 1 Mathieu Dubois-Briand 2024-12-12 09:53:16 UTC
oe-selftest-centos rocky9-vk-2 mathieu/master-next completed at 2024-12-01T17:18:25Z
https://valkyrie.yoctoproject.org/#/builders/76/builds/521/steps/14/logs/stdio
Comment 2 Randy MacLeod 2024-12-12 15:50:00 UTC
Let's see if this happens again.
It would be nice to a utility to snapshot what was happening on the build host.
Comment 3 Richard Purdie 2024-12-12 15:52:25 UTC
Looking at the logs for rocky9-vk-2 there didn't appear to be anything else other than indexing running at the time which makes this all the more strange.
Comment 4 Mathieu Dubois-Briand 2025-01-03 12:09:15 UTC
Not sure this is the same one: same issue with failed memory allocation, but a
completely different place in the code.

musl-qemux86-64 stream9-vk-1 master completed at 2024-12-29T01:13:19Z
https://valkyrie.yoctoproject.org/#/builders/3/builds/738/steps/11/logs/stdio
Comment 5 Mathieu Dubois-Briand 2025-02-10 13:02:03 UTC
oe-selftest-centos alma9-vk-1 master completed at 2025-02-08T02:31:18Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/76/builds/932/steps/14/logs/stdio
Comment 6 Mathieu Dubois-Briand 2025-04-24 14:13:58 UTC
qemuppc alma9-vk-2 stable/kirkstone-nut completed at 2025-03-20T15:47:43Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/31/builds/1215/steps/18/logs/stdio
Comment 7 Randy MacLeod 2025-05-01 14:07:09 UTC
Bulk move of all unassigned 5.2 medium importance bugs to 5.3.
Comment 8 Mathieu Dubois-Briand 2025-05-16 12:32:02 UTC
oe-selftest-fedora rocky9-vk-3 master-next completed at 2025-05-13T17:35:58Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1495/steps/14/logs/stdio
Comment 9 Randy MacLeod 2025-05-29 14:54:57 UTC
We think this was diffoscope abusing RAM usage which is now limited.
Re-open if seen again.