Created attachment 3678 [details] Copy of the logs from AB's stdout monitor On the new Autobuilder cluster we saw a failure (log snippet below and copy of log attached) when running testimage for nightly-x86-lsb. The buildgalculator.GalculatorTest test is trying to fetch the galculator-2.1.4.tar.bz2 tarball to /tmp despite it being present in DL_DIR. [joshuagl@fedora24 ~]$ ls /srv/autobuilder/autobuilder.yoctoproject.org/current_sources/galculator-2.1.4.tar.bz2 -la -rw-r--r--. 1 pokybuild users 472989 Sep 5 2015 /srv/autobuilder/autobuilder.yoctoproject.org/current_sources/galculator-2.1.4.tar.bz2 We can see from the stdout of the builder (attached) that: DL_DIR=/srv/autobuilder/autobuilder.yoctoproject.org/current_sources therefore we'd expect oeqa.utils.buildproject._download_archive() to copy the file from DL_DIR, but clearly this isn't happening. Note: This is related to but not a duplicate of #11115 Log on AB (transient link): https://autobuilder.yocto.io/builders/nightly-x86-lsb/builds/222/steps/Running%20Sanity%20Tests/logs/stdio 8<------------ Log snippet -------------------------------------------------- NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: ====================================================================== NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: ERROR: setUpClass (buildgalculator.GalculatorTest) NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: ---------------------------------------------------------------------- NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/runtime/cases/buildgalculator.py", line 16, in setUpClass cls.project.download_archive() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/runtime/utils/targetbuildproject.py", line 17, in download_archive status, output = self.target.copyTo(self.localarchive, self.targetdir) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/target/ssh.py", line 103, in copyTo return self._run(scpCmd, ignore_status=False) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/target/ssh.py", line 61, in _run "status %d:\n%s" % (command, status, output)) AssertionError: Command '['scp', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '/tmp/galculator-2.1.4.tar.bz2', 'root@192.168.7.10:~/']' returned non-zero exit status 1: /tmp/galculator-2.1.4.tar.bz2: No such file or directory NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: ---------------------------------------------------------------------- NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Ran 47 tests in 97.868s NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: FAILED (errors=1, skipped=1) NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Stopping logging thread NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Stop event received NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Tearing down logging thread NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Sending SIGTERM to runqemu core-image-lsb-sdk-1.0-r0 do_testimage: SUMMARY: core-image-lsb-sdk-1.0-r0 do_testimage: core-image-lsb-sdk () - Ran 47 tests in 97.870s core-image-lsb-sdk-1.0-r0 do_testimage: core-image-lsb-sdk - FAIL - Required tests failed (skipped=1) core-image-lsb-sdk-1.0-r0 do_testimage: RESULTS: core-image-lsb-sdk-1.0-r0 do_testimage: RESULTS - buildcpio.BuildCpioTest.test_cpio - Testcase 205: PASSED core-image-lsb-sdk-1.0-r0 do_testimage: RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase 1526: ERROR core-image-lsb-sdk-1.0-r0 do_testimage: Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/runtime/cases/buildgalculator.py", line 16, in setUpClass cls.project.download_archive() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/runtime/utils/targetbuildproject.py", line 17, in download_archive status, output = self.target.copyTo(self.localarchive, self.targetdir) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/target/ssh.py", line 103, in copyTo return self._run(scpCmd, ignore_status=False) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/meta/lib/oeqa/core/target/ssh.py", line 61, in _run "status %d:\n%s" % (command, status, output)) AssertionError: Command '['scp', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '/tmp/galculator-2.1.4.tar.bz2', 'root@192.168.7.10:~/']' returned non-zero exit status 1: /tmp/galculator-2.1.4.tar.bz2: No such file or directory
I just tried this on the autobuilder with: diff --git a/meta/lib/oeqa/utils/buildproject.py b/meta/lib/oeqa/utils/buildproject.py index fc8879c..b1b78ae 100644 --- a/meta/lib/oeqa/utils/buildproject.py +++ b/meta/lib/oeqa/utils/buildproject.py @@ -25,9 +25,11 @@ class BuildProject(metaclass=ABCMeta): # Download self.archive to self.localarchive def _download_archive(self): if self.dl_dir and os.path.exists(os.path.join(self.dl_dir, self.archive)): + print("here 1") shutil.copyfile(os.path.join(self.dl_dir, self.archive), self.localarchive) return + print("here 2") cmd = "wget -O %s %s" % (self.localarchive, self.uri) subprocess.check_output(cmd, shell=True) and it clearly prints "here 1" so the code path is executing. The problem is the /tmp/ prefix which it copies the file to, we need to use a path within workdir or a proper temp directory.
GalculatorTest creates a project object which is an instance of SDKBuildProject with the first, testpath, parameter set to self.tc.sdk_dir + "/galculator/". When the super oeqa.utils.BuildProject constructor is called it's passed testpath to its named tmpdir parameter, and it's here that the _download_archive() method tries to copy or fetch the tarball to.
(In reply to comment #2) > GalculatorTest creates a project object which is an instance of > SDKBuildProject with the first, testpath, parameter set to self.tc.sdk_dir + > "/galculator/". > > When the super oeqa.utils.BuildProject constructor is called it's passed > testpath to its named tmpdir parameter, and it's here that the > _download_archive() method tries to copy or fetch the tarball to. I am stupid and was looking at meta/lib/oeqa/sdk/buildgalculator.py not meta/lib/oeqa/runtime/cases/buildgalculator.py. I've submitted some patches to remove the hard-coding of /tmp: http://lists.openembedded.org/pipermail/openembedded-core/2017-April/135194.html https://patchwork.openembedded.org/series/6163/#
http://git.yoctoproject.org/clean/cgit.cgi/poky/commit/?id=04cf8ad80e11f88257fb8278755c2710dde2c7e7 http://git.yoctoproject.org/clean/cgit.cgi/poky/commit/?id=fa2e536c441218cca09f51c514de9799f76ef9cb http://git.yoctoproject.org/clean/cgit.cgi/poky/commit/?id=633ad6c9f436f5d2b6ee1a005b697661a054a394