Bug 13309

Summary: [master] qemuppc fails to shutdown intermittently
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Armin Kuster <akuster>
Component: coreAssignee: Kevin Hao <kexin.hao>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: amanda.r.brindle, kexin.hao, meta.mr.watcher, meta.watcher, Qi.Chen, randy.macleod, richard.purdie
Version: 2.8   
Target Milestone: 2.8 M1   
Hardware: x86   
OS: Multiple   
URL: https://errors.yoctoproject.org/Errors/Details/238410/
Whiteboard: with backport
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Armin Kuster 2019-04-26 02:04:18 UTC
runqemu.QemuTest.test_qemu_can_shutdown (subunit.RemotedTestCase)
2019-04-24 09:32:33,366 - oe-selftest - INFO -  ... FAIL
2019-04-24 09:32:33,366 - oe-selftest - INFO - 1: 1/1 16/16 (200.74s) (runqemu.QemuTest.test_qemu_can_shutdown)
2019-04-24 09:32:33,366 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemuppc/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 200, in test_qemu_can_shutdown
    self.assertTrue(qemu_shutdown_succeeded, 'Failed: %s does not shutdown within timeout(%s)' % (self.machine, shutdown_timeout))
  File "/usr/lib/python3.6/unittest/case.py", line 682, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true : Failed: qemuppc does not shutdown within timeout(120)
Comment 1 Kevin Hao 2019-04-29 05:58:44 UTC
This is an issue triggered by the commit:
commit ad522ea6a64e
Author: Robert Yang <liezhi.yang@windriver.com>
Date:   Fri Feb 15 16:14:51 2019 +0800

    runqemu: Let qemuparams override default settings
    
    Fixed:
    In meta/conf/machine/include/qemuboot-x86.inc:
    QB_CPU_x86-64 = "-cpu core2duo"
    
    $ runqemu qemux86-64 qemuparams="-cpu coreduo"
    Check /proc/cpuinfo, it should use coreduo rather than core2duo since user
    specifies it, but it doesn't, append qemuparams to the last can fix the
    problem.
    
    [YOCTO #11773]
    
    (From OE-Core rev: a847dd7202a2c493788c45d11eb86866264af7a4)
    
    Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

After this commit, the sequence of the two "-serial" parameters when running qemu have been switched. The effect of this is that the logging thread will use ttyS1 (of course can't capture the kernel boot message anymore), and the test command will run on the ttyS0. So the output of the test command may be mangled by the kernel message (such as call trace), and let the test command produce a fake timeout error message. This issue can be fixed by the codes like the follow, but it will break the qemu machines (such as qemuarm64) which use a virtio serial console.
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 1abb2c1b6cbf..a93ca2907383 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -153,11 +153,11 @@ class QemuRunner:
         # and analyze descendents in order to determine it.
         if os.path.exists(self.qemu_pidfile):
             os.remove(self.qemu_pidfile)
-        self.qemuparams = 'bootparams="{0}" qemuparams="-serial tcp:127.0.0.1:{1} -pidfile {2}"'.format(bootparams, threadport, self.qemu_pidfile)
+        self.qemuparams = 'bootparams="{0}" qemuparams="-serial tcp:127.0.0.1:{1} -pidfile {2}"'.format(bootparams, self.serverport, self.qemu_pidfile)
         if qemuparams:
             self.qemuparams = self.qemuparams[:-1] + " " + qemuparams + " " + '\"'
 
-        launch_cmd += ' tcpserial=%s %s' % (self.serverport, self.qemuparams)
+        launch_cmd += ' tcpserial=%s %s' % (threadport, self.qemuparams)
 
         self.origchldhandler = signal.getsignal(signal.SIGCHLD)
         signal.signal(signal.SIGCHLD, self.handleSIGCHLD)
Comment 2 Amanda Brindle 2019-05-13 17:47:39 UTC
This failed again:

https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/585
Comment 4 Armin Kuster 2019-05-18 15:16:33 UTC
(In reply to comment #3)
> I think 
> http://git.yoctoproject.org/cgit.cgi/poky/commit/
> ?id=d0d0d3e20f63ea29bcd6234ddefe2f390ddbdad1
> might fix this?

Warrior next has that commit and still seeing issue.


https://errors.yoctoproject.org/Errors/Details/241175/
Comment 5 Richard Purdie 2019-05-30 13:30:38 UTC
*** Bug 13370 has been marked as a duplicate of this bug. ***
Comment 6 Randy MacLeod 2019-05-30 15:30:12 UTC
The project sees this problem frequently. Kevin, will you be able to get to this defect soon?
Comment 7 Richard Purdie 2019-06-07 22:00:30 UTC
Thanks Kevin, this helps us a lot!
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=720474ee8d83a403123d7b317f095d49a3ea3e0a
Comment 8 Armin Kuster 2019-06-10 15:44:17 UTC
warrior backport out for review.