Bug 14566

Summary: SSH connection failures
Product: [QA/Testing] Runtime Testing Reporter: Oleksiy Obitotskyy <oobitots>
Component: oeqa/sdkAssignee: Unassigned <unassigned>
Status: RESOLVED OBSOLETE QA Contact:
Severity: normal    
Priority: Medium CC: alexandre.belloni, randy.macleod, richard.purdie
Version: unspecified   
Target Milestone: 4.1   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT AB-INT-NET
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Oleksiy Obitotskyy 2021-09-24 14:25:40 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/4037
https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/4012

Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemuarm/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemuarm/build/meta/lib/oeqa/runtime/cases/ssh.py", line 15, in test_ssh
    self.assertEqual(status, 0, msg='SSH Test failed: %s' % output)
AssertionError: 255 != 0 : SSH Test failed: ssh: connect to host 192.168.7.4 port 22: No route to host
Comment 1 Alexandre Belloni 2022-01-13 00:54:59 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/3458/steps/13/logs/stdio

AssertionError: 255 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl --version 
ssh: connect to host 192.168.7.6 port 22: No route to host

AssertionError: 255 != 0 : Xorg does not appear to be running ssh: connect to host 192.168.7.6 port 22: No route to host

qemuarm-alt fedora35-ty-1


fedora35-ty-1 has an SSD
Comment 2 Alexandre Belloni 2022-01-13 00:58:50 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/3460/steps/14/logs/stdio
qemuarm-alt debian9-ty-2
Comment 3 Alexandre Belloni 2022-01-13 00:59:37 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/2481/steps/14/logs/stdio
qemumips-alt debian9-ty-2

AssertionError: -15 != 0 : SSH Test failed: 
Process killed - no output for 300 seconds. Total running time: 305 seconds.
Comment 4 Alexandre Belloni 2022-01-13 01:02:42 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/4417/steps/13/logs/stdio
qemuppc ubuntu1804-ty-3
Comment 5 Alexandre Belloni 2022-01-13 08:40:58 UTC
The two previous occurrences on qemuarm-alt are actually bug 14673
Comment 6 Alexandre Belloni 2022-02-16 21:39:51 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/96/builds/2757/steps/12/logs/stdio
ubuntu2004-arm-1 non-SSD
Comment 7 Oleksiy Obitotskyy 2022-02-18 13:00:57 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/2905/steps/12/logs/stdio

qemux86-64-ltp fedora35-ty-2
Comment 8 Alexandre Belloni 2022-03-31 12:28:49 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/3850/steps/12/logs/stdio

qemuarm-alt debian11-ty-1 no SSD
Comment 9 Alexandre Belloni 2022-04-11 13:01:42 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/131/builds/416/steps/13/logs/stdio

qemuarm64-alt centos8-ty-1

We believe this one is du to a misconfiguration of connmand when using systemd and this is fixed in master-next
Comment 10 Randy MacLeod 2022-04-11 14:53:48 UTC
Alex. Did the commit make it to master? I assume we'll wait a month or two to see if it happens again so I've moved it to 3.6/4.1.
Comment 11 Alexandre Belloni 2022-04-14 13:22:35 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/131/builds/416/steps/13/logs/stdio

qemuarm64-alt centos8-ty-1 no SSD
Comment 12 Alexandre Belloni 2022-04-14 13:50:59 UTC
https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/3136/steps/12/logs/stdio

qemux86-64-ltp stream8-ty-1

https://autobuilder.yocto.io/pub/failed-builds-data/stream8-ltp-20220410/


WARNING: core-image-sato-1.0-r0 do_testimage: Copy returned non-zero exit status 1:
ssh: connect to host 192.168.7.6 port 22: No route to host
WARNING: core-image-sato-1.0-r0 do_testimage: File could not be copied. Output: ssh: connect to host 192.168.7.6 port 22: No route to host
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemux86-64-ltp/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemux86-64-ltp/build/meta/lib/oeqa/runtime/cases/ltp.py", line 115, in test_ltp_groups
    self.runltp(ltp_group)
  File "/home/pokybuild/yocto-worker/qemux86-64-ltp/build/meta/lib/oeqa/runtime/cases/ltp.py", line 87, in runltp
    results, sections  = parser.parse(dst)
  File "/home/pokybuild/yocto-worker/qemux86-64-ltp/build/meta/lib/oeqa/utils/logparser.py", line 116, in parse
    with open(logfile, errors='replace') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/qemux86-64-ltp/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testimage/ltp_log.20220409205903/controllers'
Stdout:
Tried to dump info from target but serial console failed
Failed CMD: find /var/log/ -type f 2>/dev/null -exec echo "====================" \; -exec echo {} \; -exec echo "====================" \; -exec cat {} \; -exec echo "" \;
Comment 13 Richard Purdie 2022-04-16 10:14:18 UTC
This is another confused bug. I think the -alt testimage failres are from the connmand issue on systemd images. The ltp failures are different, those are where the kernel crashes part way through testing. I opened a new bug for the ltp issue and am closing this one. We need to try and keep similar issues together.