Bug 12699 - [2.5 M3 rc1] [OE-core] [autobuilder2] - buildoptions.ArchiverTest.test_arch_work_dir_and_export_source - Testcase 926: FAILED - oescripts.BuildhistoryDiffTests.test_buildhistory_diff - Testcase 295: FAILED - runtime_test.TestImage.test_testimage_dnf - Tes
Summary: [2.5 M3 rc1] [OE-core] [autobuilder2] - buildoptions.ArchiverTest.test_arch_w...
Status: RESOLVED NOTABUG
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.5 M4
Assignee: Aaron Chan
QA Contact: Aaron Chan
URL: http://centaurus02.png.intel.com:8010...
Whiteboard:
Depends on:
Blocks:
 
Reported: 2018-04-20 01:28 UTC by Aaron Chan
Modified: 2018-05-17 08:01 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Found by QA test)
Verified:
Documentation change: Don't know


Attachments
centos7-distro-autobuilder2-logfiles (26.04 KB, application/x-zip-compressed)
2018-04-20 01:46 UTC, Aaron Chan
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Chan 2018-04-20 01:28:48 UTC
==============================================================================
NOTE: This bug is filed to exercise the same QA cycle on between old codebase
      legacy autobuilder and newer codebase (autobuilder2)
==============================================================================

DISTRO : CentOS7
OE-CORE: oe-selftest


AUTOBUILDER [Legacy]
---------------------
REFERENCE : http://centaurus01.png.intel.com:8010/builders/nightly-oe-selftest-centos7/builds/16/steps/Running%20oe-selftest/logs/stdio
STATUS    : PASSED ALL

AUTOBUILDER [New]
---------------------
REFERENCE : http://centaurus02.png.intel.com:8010/#/builders/18/builds/17/steps/5/logs/stdio
STATUS    : FAILED (3 testcases)
TESTCASES :

oescripts.BuildhistoryDiffTests.test_buildhistory_diff - Testcase 295: FAILED
buildoptions.ArchiverTest.test_arch_work_dir_and_export_source - Testcase 926: FAILED
runtime_test.TestImage.test_testimage_dnf - Testcase 1883: FAILED

2018-04-16 22:12:53,996 - oe-selftest - INFO - ======================================================================
2018-04-16 22:12:53,997 - oe-selftest - INFO - FAIL: test_arch_work_dir_and_export_source (buildoptions.ArchiverTest)
2018-04-16 22:12:53,997 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-04-16 22:12:53,997 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/buildoptions.py", line 166, in test_arch_work_dir_and_export_source
    self.assertTrue((g.glob(src_file_glob) and g.glob(tar_file_glob)), "Couldn't find .src.rpm and .tar.gz files under %s/allarch*/xcursor*" % deploy_dir_src)
AssertionError: [] is not true : Couldn't find .src.rpm and .tar.gz files under /home/ab/yocto-worker/nightly-oe-selftest/build/build/tmp/deploy/sources/allarch*/xcursor*
2018-04-16 22:12:53,997 - oe-selftest - INFO - ======================================================================
2018-04-16 22:12:53,998 - oe-selftest - INFO - FAIL: test_buildhistory_diff (oescripts.BuildhistoryDiffTests)
2018-04-16 22:12:53,998 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-04-16 22:12:53,998 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/oescripts.py", line 13, in test_buildhistory_diff
    result = runCmd("buildhistory-diff -p %s" % get_bb_var('BUILDHISTORY_DIR'))
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'buildhistory-diff -p /home/ab/yocto-worker/nightly-oe-selftest/build/build/tmp_buildhistory_20180416200849' returned non-zero exit status 1:
Please install GitPython (python3-git) 0.3.4 or later in order to use this script
2018-04-16 22:12:53,998 - oe-selftest - INFO - ======================================================================
2018-04-16 22:12:53,999 - oe-selftest - INFO - FAIL: test_testimage_dnf (runtime_test.TestImage)
2018-04-16 22:12:53,999 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-04-16 22:12:53,999 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/runtime_test.py", line 163, in test_testimage_dnf
    bitbake('-c testimage core-image-full-cmdline')
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 213, in bitbake
    return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake  -c testimage core-image-full-cmdline' returned non-zero exit status 1:
Loading cache...done.
Loaded 1300 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

TESTCASES (MANUAL)
------------------
oescripts.BuildhistoryDiffTests.test_buildhistory_diff - Testcase 295: FAILED - REPRODUCIBLE
buildoptions.ArchiverTest.test_arch_work_dir_and_export_source - Testcase 926: FAILED - REPRODUCIBLE
runtime_test.TestImage.test_testimage_dnf - Testcase 1883: FAILED - REPRODUCIBLE

2018-04-21 00:18:49,361 - oe-selftest - INFO - ======================================================================
2018-04-21 00:18:49,362 - oe-selftest - INFO - FAIL: test_buildhistory_diff (oescripts.BuildhistoryDiffTests)
2018-04-21 00:18:49,362 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-04-21 00:18:49,362 - oe-selftest - INFO - Traceback (most recent call last):
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/oescripts.py", line 13, in test_buildhistory_diff
    result = runCmd("buildhistory-diff -p %s" % get_bb_var('BUILDHISTORY_DIR'))
  File "/home/ab/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'buildhistory-diff -p /home/ab/yocto-worker/nightly-oe-selftest/build/build/tmp_buildhistory_20180421001753' returned non-zero exit status 1:
Please install GitPython (python3-git) 0.3.4 or later in order to use this script

NOTE: More additional debug messages when run manually compare running on autobuilder.
Comment 1 Aaron Chan 2018-04-20 01:46:05 UTC
Created attachment 4276 [details]
centos7-distro-autobuilder2-logfiles
Comment 2 Paul Eggleton 2018-04-22 14:10:21 UTC
Well, at least for test_buildhistory_diff it says: "Please install GitPython (python3-git) 0.3.4 or later in order to use this script" - so it looks like GitPython needs to be installed on that machine and that should fix that one.

For the dnf tests it appears to be some kind of problem connecting to the host from the target, or alternatively the package repo isn't being shared properly. Can you please insert some calls to wget or similar into the test to try to see what's going wrong?

For test_arch_work_dir_and_export_source, I'm not sure what the problem might be, so I guess we'll have to run through the steps of the test.
Comment 3 Stephano Cetola 2018-04-26 07:37:09 UTC
See Comment 2.
Comment 4 Ross Burton 2018-04-26 07:37:58 UTC
The networking thing is almost definitely a firewall. Red Hat and derivates has an incredibly strong firewall by default, which you'll need to turn off.
Comment 5 Aaron Chan 2018-04-26 19:11:58 UTC
I am just a little confuse on the version is higher than 0.3.4 (python3-git) error messages from the bitbake build and yet am still getting this error. Anything that i missed out?

$ rpm -qa | grep GitPython

GitPython-1.0.1-7.el7.noarch

$ sudo yum install GitPython

Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.exabytes.com.my
 * epel: archive.linux.duke.edu
 * extras: centos.biz.net.id
 * updates: centos.biz.net.id
Package GitPython-1.0.1-7.el7.noarch already installed and latest version
Nothing to do

Hi Ross,

This error is on centos7 distro and not found in Fedora distro. Check the firewall rules on this distro and its running

$ sudo systemctl status firewalld

firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-04-13 23:22:35 +08; 1 weeks 6 days ago
     Docs: man:firewalld(1)
 Main PID: 1073 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─1073 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
Comment 6 Ross Burton 2018-04-27 02:51:10 UTC
firewalld is basically stopping all traffic, turn it off.
Comment 7 Aaron Chan 2018-04-30 02:56:37 UTC
Hi Ross,

The firewalld services has already been disabled in this Centos7 Distro but not sure how this can be related to the
test cases which are found to fail in this case.

Rerun the entire 2.5 M3 rc1 - Oe-selftest on CentOS7 Distro

http://centaurus02.png.intel.com:8010/#/builders/18/builds/26
Comment 8 Ross Burton 2018-05-11 05:52:14 UTC
"Please install GitPython (python3-git) 0.3.4 or later in order to use this script"

Centos 7 doesn't have a standard install of Python 3, so however you installed that you'll also need to install the corresponding python-git library.
Comment 9 Richard Purdie 2018-05-17 08:01:24 UTC
As per the comments, you need to install python3-git so this isn't a bug.