Bug 8561

Summary: dependency issues when testing multilib images
Product: [QA/Testing] Runtime Testing Reporter: Alexandru Georgescu <alexandru.c.georgescu>
Component: generalAssignee: Lucian Musat <georgex.l.musat>
Status: RESOLVED NOTABUG QA Contact: Cristina Agurida <cristina-danielax.agurida>
Severity: normal    
Priority: Medium CC: jku, richard.purdie
Version: 2.0   
Target Milestone: 2.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on: 8550    
Bug Blocks:    

Description Alexandru Georgescu 2015-10-23 05:56:32 UTC
http://errors.yoctoproject.org/Errors/Details/20802/

Task: do_testimage
Recipe: core-image-sato
Machine: qemux86-64
Distro: poky
Build system: x86_64-linux
Target system: x86_64-poky-linux
Host distro: Ubuntu-12.04
Branch: heads/jethro
Commit:505a82673ac2487df5ea343a6422c2fc47018831
Build:#9448

When running the sanity tests on AB, we have the following error: 

ERROR
ERROR
test_check_multilib_libc (oeqa.runtime.multilib.MultilibTest) ... skipped 'Testcase dependency not met: test_ssh'
test_file_connman (oeqa.runtime.multilib.MultilibTest) ... skipped 'Testcase dependency not met: test_check_multilib_libc'

======================================================================
ERROR: setUpClass (oeqa.runtime.rpm.RpmInstallRemoveTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-multilib/build/meta/lib/oeqa/runtime/rpm.py", line 37, in setUpClass
    oeRuntimeTest.tc.target.copy_to(os.path.join(rpmdir,testrpmfile), "/tmp/rpm-doc.rpm")
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-multilib/build/meta/lib/oeqa/targetcontrol.py", line 107, in copy_to
    return self.connection.copy_to(localpath, remotepath)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-multilib/build/meta/lib/oeqa/utils/sshcontrol.py", line 150, in copy_to
    return self._internal_run(command, ignore_status=False)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-multilib/build/meta/lib/oeqa/utils/sshcontrol.py", line 127, in _internal_run
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, status, output))
AssertionError: Command '['scp', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-multilib/build/build/tmp/deploy/rpm/core2_64/rpm-doc-5.4.14-r0.core2_64.rpm', 'root@192.168.7.4:/tmp/rpm-doc.rpm']' returned non-zero exit status 1:
ssh: connect to host 192.168.7.4 port 22: No route to host
lost connection
Comment 1 Richard Purdie 2015-10-23 06:37:11 UTC
The key failure isn't mentioned here, specifically: "FAIL: test_ping (oeqa.runtime.ping.PingTest)", clearly networking either never came up or died for this test run.
Comment 2 Lucian Musat 2015-10-23 11:49:17 UTC
This is not a dependency issue, it's a networking issue. The ping/ssh tests failed so all the tests that have @skipunlesspassed decorator depending on network will skip after that.
I cannot reproduce this issue on my local machine though. Tried with multiple qemus open at the same time, running runtime tests on two instances in parallel, repeated the process multiple times and still cannot reproduce.
I believe that this could be a race condition somewhere when running multiple qemus, as these kinds of errors don't seem to appear on our local Autobuilder as we don't test them in parallel.
Comment 3 Jussi Kukkonen 2015-10-27 14:11:13 UTC
(In reply to comment #2)
> This is not a dependency issue, it's a networking issue. The ping/ssh tests
> failed so all the tests that have @skipunlesspassed decorator depending on
> network will skip after that.

The networking setup issue (on qemu multilib) is bug 8550: patch is on mailing list. That problem should be 100% reproducable with something like this in local.conf:

  require conf/multilib.conf
  MULTILIBS = "multilib:lib32"
  DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
  RPM_PREFER_ELF_ARCH = "1"
  IMAGE_INSTALL_append = " lib32-connman-gnome"
  MACHINE = "qemux86-64"


But looking at the log, the test fails in SetUpClass because it tries to run scp: I don't think it should do that when ssh_test has not passed.
Comment 4 Lucian Musat 2015-10-27 15:20:37 UTC
setUpClass is not decorated with @skipUnlessPassed and I believe is run before all the tests in the class.
Comment 5 Lucian Musat 2015-10-30 16:02:52 UTC
This error is generated by the setup class in rpm test suite because it tries to scp a file when no network is available (No route to host), so it is actually not a bug here. 
Closing this as notabug.
Comment 6 Jussi Kukkonen 2015-11-02 08:35:11 UTC
(In reply to comment #5)
> This error is generated by the setup class in rpm test suite because it
> tries to scp a file when no network is available (No route to host), so it
> is actually not a bug here. 
> Closing this as notabug.

Can you explain that? Is it not a bug that it tries to scp even though the ssh test failed?
Comment 7 Lucian Musat 2015-11-02 09:06:30 UTC
The fail is not due to the code per se, it's because of a network problem so it is expected to fail in that case.
The setUpClass is not really a test so it cannot be decorated with skipunlesspassed decorator. Now I agree that it's not a good thing that it tries the scp there without any check but I believe we could open a new enhancement for that, and close this bug regarding multilib as it's treated in 8550.
Comment 8 Jussi Kukkonen 2015-11-02 09:24:37 UTC
(In reply to comment #7)
> The fail is not due to the code per se, it's because of a network problem so
> it is expected to fail in that case.


The testing code doesn't just fail, it asserts.


Anyway, just wanted to make sure we don't miss something by accident: I'll leave this to you now.