| Summary: | qemu image failed ping test | ||
|---|---|---|---|
| Product: | [QA/Testing] Runtime Testing | Reporter: | Richard Purdie <richard.purdie> |
| Component: | testimage | Assignee: | Unassigned <unassigned> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | alejandro, alexandre.belloni, anuj.mittal, bluelightning, chee.yang.lee, flowergom, Jagadheesan_Duraisamy, oobitots, randy.macleod, steve |
| Version: | unspecified | ||
| Target Milestone: | 4.1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | AB-INT AB-INT-NET | ||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Richard Purdie
2020-08-27 23:16:46 UTC
We think that with the new 'nice' scheme implemented on the autobuilder around Sept 6-9, the responsiveness of qemu has improved and we believe that this bug may not occur again. Lowering to medium priority. I got this issue today while building next set of changes for gatesgarth. This was on Ubuntu1804-ty2 while executing do_testimage for core-image-sato. https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/2745/steps/9/logs/step2c Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qa-extras2/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qa-extras2/build/meta/lib/oeqa/runtime/cases/ping.py", line 19, in test_ping output += proc.communicate()[0].decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 850, in communicate stdout = self.stdout.read() File "/home/pokybuild/yocto-worker/qa-extras2/build/meta/lib/oeqa/core/decorator/oetimeout.py", line 19, in _timeoutHandler "seconds of execution" % timeout) oeqa.core.exception.OEQATimeoutError: Timed out after 30 seconds of execution happen again ( master, ubuntu1804-ty-2, core-image-sato ) https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/2955/steps/13/logs/stdio Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/2979/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/1304/steps/13/logs/stdio Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/3048/steps/12/logs/stdio Happened again: https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/1797/steps/12/logs/stdio *** Bug 14240 has been marked as a duplicate of this bug. *** New occurence ( opensuse151-ty-1 , qemumips64 ): https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3114/steps/13/logs/stdio Happened again: ubuntu1804-ty-3 qemumips64 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3217/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/1366 https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/3209/steps/13/logs/stdio Started HTTPService on 0.0.0.0:38301 Test requires apt to be installed Stopped HTTPService on 0.0.0.0:38301 Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemuppc/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-worker/qemuppc/build/meta/lib/oeqa/runtime/cases/ping.py", line 19, in test_ping output += proc.communicate()[0].decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 850, in communicate stdout = self.stdout.read() File "/home/pokybuild/yocto-worker/qemuppc/build/meta/lib/oeqa/core/decorator/oetimeout.py", line 19, in _timeoutHandler "seconds of execution" % timeout) oeqa.core.exception.OEQATimeoutError: Timed out after 30 seconds of execution qemux86-64 core-image-sato lib32 https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/3267 Seems reasonable to assume this is the same issue: (qemuarm, ping succeeded but SSH subsequently failed): https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/3263 qemuppc ping test failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/3236/steps/13/logs/stdio multilib mip64 n32 ping test failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/3278/steps/26/logs/stdio Happened again: qemumips64: https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3289 multilib: https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/3324/steps/18/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/3380/steps/21/logs/stdio master on ubuntu1804-ty-2 https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/3368/steps/26/logs/stdio qemumips64 master-next on ubuntu2004-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3359/steps/13/logs/stdio qemumips64 master on centos8-ty-2 https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/3511/steps/13/logs/stdio qemumips64 debian10-ty-1 Wait a month or two to see if it happens now that the RCU hang bug fix is in. It may still happen on due to heavy load and RCU stalls. https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/4646/steps/13/logs/stdio stream8-ty-1 has an SSD https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/4669/steps/12/logs/stdio centos8-ty-2 non-SSD Both during the same a-full build https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/2823/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/3475/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3082/steps/14/logs/stdio All on stream8-ty-1 that has an SSD Interestingly, the last one has: ping: socket: Operation not permitted https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/2834 https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/4631 Those two are again on stream8 which has an issue https://autobuilder.yoctoproject.org/typhoon/#/builders/57/builds/4612/steps/12/logs/stdio centos8-ty-1, is this the same issue as on stream8? https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3104/steps/14/logs/stdio centos8-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/3496/steps/13/logs/stdio centos8-ty-1 https://autobuilder.yoctoproject.org/typhoon/#/builders/69/builds/4636/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/69/builds/4636/steps/15/logs/stdio Still centos8-ty-1 (note that stream8 is currently paused) https://autobuilder.yoctoproject.org/typhoon/#/builders/69/builds/4642/steps/12/logs/stdio Still centos8 https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/1748/steps/14/logs/stdio This is on ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/59/builds/4651/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/4673/steps/12/logs/stdio Both on centos8 https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/4643/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/69/builds/4644/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/47/builds/4673/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/2845/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/57/builds/4617/steps/12/logs/stdio All on centos8 https://autobuilder.yoctoproject.org/typhoon/#builders/107/builds/2661/steps/13/logs/stdio qemuppc-alt ubuntu2004-ty-1 Dunfell occurrence: https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/4786 qemumips64 ubuntu2004-ty-1 Dunfell occurrence: https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/2760 qemuppc-alt ubuntu1804-ty-3 https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/2752/steps/14/logs/stdio qemumips-alt debian9-ty-2 https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/4877/steps/29/logs/stdio qa-extras2 centos8-ty-2 https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/2888/steps/12/logs/stdio qemumips-alt ubuntu1804-ty-3 AssertionError: Ping timeout error for address 192.168.7.6, count 0, output: PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms PING 192.168.7.6 (192.168.7.6) 56(84) bytes of data. From 192.168.7.5 icmp_seq=1 Destination Host Unreachable --- 192.168.7.6 ping statistics --- 1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms We hope the systemd issues are resolved with: https://git.yoctoproject.org/poky/commit?id=89efab2850766378d89579c094670357775b69b6 If there are similar problems, please open a new bug. |