Bug 14251 - core-image-sato-1.0-r0:do_testimage FAILED when starting qemu
Summary: core-image-sato-1.0-r0:do_testimage FAILED when starting qemu
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 3.3
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 3.4 M2
Assignee: Unassigned
QA Contact:
URL:
Whiteboard: AB-INT AB-INT-TMPFS
Depends on:
Blocks:
 
Reported: 2021-02-23 17:14 UTC by Leonardo Sandoval
Modified: 2025-02-14 16:48 UTC (History)
11 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Leonardo Sandoval 2021-02-23 17:14:52 UTC
AB Collection [1], build [2]

ERROR: core-image-sato-1.0-r0 do_testimage: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_testimage(d)
     0003:
File: '/home/pokybuild/yocto-worker/qemumips-alt/build/meta/classes/testimage.bbclass', lineno: 131, function: do_testimage
     0127:    netstat -an
     0128:}
     0129:
     0130:python do_testimage() {
 *** 0131:    testimage_main(d)
     0132:}
     0133:
     0134:addtask testimage
     0135:do_testimage[nostamp] = "1"
File: '/home/pokybuild/yocto-worker/qemumips-alt/build/meta/classes/testimage.bbclass', lineno: 375, function: testimage_main
     0371:    orig_sigterm_handler = signal.signal(signal.SIGTERM, sigterm_exception)
     0372:    try:
     0373:        # We need to check if runqemu ends unexpectedly
     0374:        # or if the worker send us a SIGTERM
 *** 0375:        tc.target.start(params=d.getVar("TEST_QEMUPARAMS"), runqemuparams=d.getVar("TEST_RUNQEMUPARAMS"))
     0376:        import threading
     0377:        try:
     0378:            threading.Timer(int(d.getVar("TEST_OVERALL_TIMEOUT")), handle_test_timeout, (int(d.getVar("TEST_OVERALL_TIMEOUT")),)).start()
     0379:        except ValueError:
File: '/home/pokybuild/yocto-worker/qemumips-alt/build/meta/lib/oeqa/core/target/qemu.py', lineno: 71, function: start
     0067:            if self.runner.server_ip:
     0068:                self.server_ip = self.runner.server_ip
     0069:        else:
     0070:            self.stop()
 *** 0071:            raise RuntimeError("FAILED to start qemu - check the task log and the boot log")
     0072:
     0073:    def stop(self):
     0074:        self.runner.stop()
Exception: RuntimeError: FAILED to start qemu - check the task log and the boot log


[1] https://swatbot.yoctoproject.org/collection/543/
[2] https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/1410/steps/13/logs/stdio
Comment 1 Aníbal Limón 2021-02-25 16:00:37 UTC
For previous bugs I remember that qemu speed (without accell) is affected by the I/O load of the host machine, may be related to qemu design.
Comment 2 Aníbal Limón 2021-02-25 16:00:46 UTC
For previous bugs I remember that qemu speed (without accell) is affected by the I/O load of the host machine, may be related to qemu design.
Comment 4 Alejandro 2021-04-07 19:40:24 UTC
This seems to be the same, but it happened during oe-selftest this time (runtime_test.SystemTap.test_crosstap_pstree):

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1973
Comment 7 Naveen Saini 2021-04-19 07:29:15 UTC
Happened again:
https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/3300
Comment 8 Naveen Saini 2021-04-19 07:57:13 UTC
(In reply to comment #7)
> Happened again:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/3300

It is on master branch.
Worker: centos7-ty-4
Builder: pkgman-non-rpm
Comment 9 Naveen Saini 2021-04-19 08:19:54 UTC
For debugging purpose, adding AB link for old failure:

Branch: master-next.
Worker: fedora33-ty-1
Builder: qemux86
https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/3290
Comment 10 Alexandre Belloni 2021-04-29 23:01:42 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/3362/steps/13/logs/stdio

master
qemuarm
debian10-ty-1
Comment 11 Alexandre Belloni 2021-05-01 22:26:23 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/3321/steps/13/logs/stdio

master-next on debian10-ty-3
Comment 12 Alexandre Belloni 2021-05-06 12:24:21 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2035/steps/13/logs/stdio

qemux86-64
master on centos7-ty-4
Comment 13 Alexandre Belloni 2021-05-06 16:14:37 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/3367/steps/13/logs/stdio

qemux86-64
master on debian10-ty-3
Comment 17 Jon Mason 2021-05-18 23:54:29 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/2108/steps/14/logs/stdio
oe-selftest-debian
debian10-ty-2
Comment 19 Alexandre Belloni 2021-05-27 22:57:36 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/3462

musl-qemux86
ubuntu1604-ty-1
Comment 20 Alexandre Belloni 2021-05-27 23:03:27 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/3484/steps/13/logs/stdio

qemuarm
debian10-ty-2
Comment 21 Alexandre Belloni 2021-06-16 01:36:44 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1471/steps/15/logs/stdio

oe-selftest
centos7-ty-4
Comment 22 Alexandre Belloni 2021-06-16 08:20:31 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/3459/steps/13/logs/stdio

qemux86-64
debian10-ty-2
Comment 24 Alexandre Belloni 2021-06-16 09:58:53 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/2384/steps/13/logs/stdio

qemux86-alt
debian10-ty-1
Comment 25 Alexandre Belloni 2021-06-16 10:05:52 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/2386/steps/12/logs/stdio

qemux86-alt
debian10-ty-2
Comment 26 Alexandre Belloni 2021-06-24 13:00:53 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3559/steps/13/logs/stdio

qemumips64 debian10-ty-2
Comment 27 Alexandre Belloni 2021-06-24 13:07:32 UTC
I forgot to add this one has: runqemu - INFO - Copy done in 302.5292055606842 seconds which would indicate an IO load issue
Comment 28 Alexandre Belloni 2021-06-24 13:53:52 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/3581/steps/12/logs/stdio

musl-qemux86 ubuntu1804-ty-1
Comment 29 Richard Purdie 2021-06-28 09:58:10 UTC
Most likely cause of these is the rcu stall issue, fixed with http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1f802cdf857475eefb9f3ae5032a6f1ce3e01d97

We can open new bugs if issues recur. If this does recur, please try and capture the qemu boot log from WORKDIR/testimage/qemu_*.log
Comment 30 Randy MacLeod 2025-02-14 16:46:30 UTC
bulk change to add Saul's non-WR address.
Comment 31 Randy MacLeod 2025-02-14 16:48:22 UTC
Bulk change: Remove Saul's old WR address.