Bug 8937 - SystemdServiceTest failed on the autobuilder
Summary: SystemdServiceTest failed on the autobuilder
Status: RESOLVED DUPLICATE of bug 8142
Alias: None
Product: System Startup
Classification: Runtime
Component: system-startup (show other bugs)
Version: 2.0
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1 M4
Assignee: Aníbal Limón
QA Contact:
URL: http://errors.yoctoproject.org/Errors...
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-01-15 21:17 UTC by Saul Wold
Modified: 2016-04-08 17:31 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: Regression (Used to work)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
Log file from Autobuilder for systemdbasictests failure (25.33 KB, text/plain)
2016-01-15 21:17 UTC, Saul Wold
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Saul Wold 2016-01-15 21:17:42 UTC
Created attachment 2931 [details]
Log file from Autobuilder for systemdbasictests failure

Build Configuration:
BB_VERSION        = "1.28.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Fedora-21"
TARGET_SYS        = "x86_64-poky-linux"
MACHINE           = "qemux86-64"
DISTRO            = "poky"
DISTRO_VERSION    = "2.0"
TUNE_FEATURES     = "m64 core2"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "heads/sgw/2.0.1-test:e52406900c0d347a97dcd618aa7773938ea3d836"


| test_systemd_status (oeqa.runtime.systemd.SystemdServiceTests) ... FAIL
| test_systemd_status dump data stored in /tmp/oe-saved-tests/201601151118_test_systemd_status
| test_systemd_stop_start (oeqa.runtime.systemd.SystemdServiceTests) ... skipped 'Testcase dependency not met: test_systemd_status'
| 
| ======================================================================
| FAIL: test_systemd_failed (oeqa.runtime.systemd.SystemdBasicTests)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
|     return func(*args, **kwargs)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/decorators.py", line 94, in wrapped_f
|     return f(*args, **kwargs)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/runtime/systemd.py", line 65, in test_systemd_failed
|     self.assertTrue(match, msg="Some systemd units failed:\n%s" % output)
| AssertionError: None is not true : Some systemd units failed:
|   UNIT                   LOAD   ACTIVE SUB    DESCRIPTION
| ● avahi-daemon.service   loaded failed failed Avahi mDNS/DNS-SD Stack
| ● bluetooth.service      loaded failed failed Bluetooth service
| ● ofono.service          loaded failed failed Telephony service
| ● wpa_supplicant.service loaded failed failed WPA supplicant
| 
| LOAD   = Reflects whether the unit definition was properly loaded.
| ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
| SUB    = The low-level unit activation state, values depend on unit type.
| 
| 4 loaded units listed. Pass --all to see loaded but inactive units, too.
| To show all installed unit files use 'systemctl list-unit-files'.● qemux86-64
|     State: degraded
|      Jobs: 0 queued
|    Failed: 4 units
|     Since: Fri 2016-01-15 10:56:44 UTC; 21min ago
|    CGroup: /
|            ├─ 1 /sbin/init
|            ├─80 /sbin/v86d
|            └─system.slice
|              ��─dbus.service
|              │ └─280 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
|              ├─distcc.service
|              │ ├─495 /usr/bin/distccd --verbose --no-detach --daemon --allow 192.168.7.0/24
|              │ ├─503 /usr/bin/distccd --verbose --no-detach --daemon --allow 192.168.7.0/24
|              │ ├─507 /usr/bin/distccd --verbose --no-detach --daemon --allow 192.168.7.0/24
|              │ └─510 /usr/bin/distccd --verbose --no-detach --daemon --allow 192.168.7.0/24
|              ├─busybox-klogd.service
|              │ └─251 /sbin/klogd -n
|              ├─system-serial\x2dgetty.slice
|              │ ├─serial-getty@ttyS1.service
|              │ │ └─498 -sh
|              │ └─serial-getty@ttyS0.service
|              │   └─497 /sbin/agetty -8 -L --keep-baud ttyS0 115200 xterm
|              ├─systemd-journald.service
|              │ └─78 /lib/systemd/systemd-journald
|              ├─connman.service
|              │ └─430 /usr/sbin/connmand -n
|              ├─systemd-timesyncd.service
|              │ └─662 /lib/systemd/systemd-timesyncd
|              ├─systemd-logind.service
|              │ └─499 /lib/systemd/systemd-logind
|              ├─system-getty.slice
|              │ └─getty@tty1.service
|              │   └─275 /sbin/agetty --noclear tty1 linux
|              ├─systemd-udevd.service
|              │ └─101 /lib/systemd/systemd-udevd
|              ├─busybox-syslog.service
|              │ └─582 /sbin/syslogd -n -C
|              ├─system-dropbear.slice
|              │ ├─dropbear@13-192.168.7.6:22-192.168.7.5:37914.service
|              │ │ └─617 x11vnc -display :0 -bg -o x11vnc.log
|              │ └─dropbear@135-192.168.7.6:22-192.168.7.5:38055.service
|              │   ├─1773 /usr/sbin/dropbear -i -r /etc/dropbear/dropbear_rsa_host_key -B
|              │   ├─1774 sh -c  . /etc/profile; systemctl status --full --failed
|              │   └─1775 systemctl status --full --failed
|              └─xserver-nodm.service
|                ├─264 xinit /etc/X11/Xsession -- /usr/bin/Xorg :0 -br -pn
|                ├─292 /usr/bin/Xorg :0 -br -pn
|                ├─353 matchbox-window-manager -theme Sato -use_desktop_mode decorated -use_cursor no
|                ├─358 dbus-launch --sh-syntax --exit-with-session
|                ├─359 /usr/bin/dbus-daemon --fork --print-pid 4 --print-address 6 --session
|                ├─376 connman-applet
|                ├─386 /usr/lib/gconf/gconfd-2
|                ├─401 matchbox-desktop
|                ├─402 matchbox-panel --titlebar --start-applets showdesktop,windowselector --end-applets clock,battery,,systray,startup-notify,notify
|                └─403 /usr/bin/settings-daemon
| 
| ======================================================================
| FAIL: test_systemd_status (oeqa.runtime.systemd.SystemdServiceTests)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/decorators.py", line 94, in wrapped_f
|     return f(*args, **kwargs)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/runtime/systemd.py", line 77, in test_systemd_status
|     self.systemctl('status --full', 'avahi-daemon.service')
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/runtime/systemd.py", line 21, in systemctl
|     self.assertEqual(status, expected, message)
| AssertionError: 3 != 0 : systemctl status --full avahi-daemon.service
| ● avahi-daemon.service - Avahi mDNS/DNS-SD Stack
|    Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
|    Active: failed (Result: timeout) since Fri 2016-01-15 10:58:55 UTC; 19min ago
|  Main PID: 262 (code=exited, status=0/SUCCESS)
|    Status: "avahi-daemon 0.6.31 starting up."
| 
| Jan 15 10:58:23 qemux86-64 avahi-daemon[262]: Server startup complete. Host name is qemux86-64.local. Local service cookie is 3756040364.
| Jan 15 10:58:24 qemux86-64 avahi-daemon[262]: Service "qemux86-64" (/services/ssh.service) successfully established.
| Jan 15 10:58:24 qemux86-64 avahi-daemon[262]: Service "qemux86-64" (/services/sftp-ssh.service) successfully established.
| Jan 15 10:58:55 qemux86-64 systemd[1]: avahi-daemon.service: Start operation timed out. Terminating.
| Jan 15 10:58:55 qemux86-64 avahi-daemon[262]: Got SIGTERM, quitting.
| Jan 15 10:58:55 qemux86-64 avahi-daemon[262]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.7.6.
| Jan 15 10:58:55 qemux86-64 avahi-daemon[262]: avahi-daemon 0.6.31 exiting.
| Jan 15 10:58:55 qemux86-64 systemd[1]: Failed to start Avahi mDNS/DNS-SD Stack.
| Jan 15 10:58:55 qemux86-64 systemd[1]: avahi-daemon.service: Unit entered failed state.
| Jan 15 10:58:55 qemux86-64 systemd[1]: avahi-daemon.service: Failed with result 'timeout'.
| 
| ----------------------------------------------------------------------
Comment 1 Randy MacLeod 2016-01-21 16:26:07 UTC
This bug seems to be specific to the YP autobuilder.
Is the system just overloaded or dropping packets?

Yi, Have you seen this problem at all? Can you run an overnight test loop to help us understand how rare it is and if it can be reproduced on non-YP test instrastructure. Thanks.
Comment 2 Yi Zhao 2016-01-22 07:04:26 UTC
(In reply to comment #1)
> This bug seems to be specific to the YP autobuilder.
> Is the system just overloaded or dropping packets?
> 
> Yi, Have you seen this problem at all? Can you run an overnight test loop to
> help us understand how rare it is and if it can be reproduced on non-YP test
> instrastructure. Thanks.

I hadn't seen this before since our local autobuilder doesn't build systemd image. I will try do reproduce it in local.
Comment 3 Paul Eggleton 2016-04-07 22:22:12 UTC
This bug looks very similar to bug 8142 - assigning this one to Anibal as well.
Comment 4 Aníbal Limón 2016-04-08 17:31:03 UTC
For me this bug is the same as 8142 so closed as duplicated.

The main reason for this is because systemd is killing avahi, blutooth, wpa_supplicant and ofono due to timeout.

Example:

| Jan 15 10:58:55 qemux86-64 systemd[1]: avahi-daemon.service: Start operation timed out. Terminating.
| Jan 15 10:58:55 qemux86-64 avahi-daemon[262]: Got SIGTERM, quitting.
| Jan 15 10:58:55 qemux86-64 avahi-daemon[262]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.7.6.

*** This bug has been marked as a duplicate of bug 8142 ***