Bug 13504

Summary: [master-next] qemumips-lsb sytemd failure
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Armin Kuster <akuster>
Component: coreAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: High CC: meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie
Version: 3.0   
Target Milestone: 2.8 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
local.conf none

Description Armin Kuster 2019-09-07 21:54:34 UTC 
Created attachment 4559 [details]
local.conf

Test requires systemtap to be installed
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/qemumips-lsb/build/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-worker/qemumips-lsb/build/meta/lib/oeqa/runtime/cases/systemd.py", line 98, in test_systemd_failed
    self.assertTrue(match, msg='Some systemd units failed:\n%s' % output)
AssertionError: None is not true : Some systemd units failed:
  UNIT                        LOAD   ACTIVE SUB    DESCRIPTION              
● systemd-hwdb-update.service loaded failed failed Rebuild Hardware Database
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
1 loaded units listed.● qemumips
    State: degraded
     Jobs: 0 queued
   Failed: 1 units
    Since: Sat 2019-09-07 16:02:44 UTC; 6min ago
   CGroup: /
           ├─user.slice
           │ └─user-0.slice
           │   ├─session-c54.scope
           │   │ ├─522 sshd: root@notty
           │   │ ├─524 sh -c export PATH=/usr/sbin:/sbin:/usr/bin:/bin; SYSTEMD_BUS_TIMEOUT=240s systemctl status --full --failed 
           │   │ └─525 systemctl status --full --failed
           │   ├─session-c1.scope
           │   │ ├─162 /bin/login --
           │   │ └─173 -sh
           │   └─user@0.service
           │     └─init.scope
           │       ├─167 /lib/systemd/systemd --user
           │       └─168 (sd-pam)
           ├─init.scope
           │ └─1 /sbin/init
           └─system.slice
             ├─rngd.service
             │ └─99 /usr/sbin/rngd -f -r /dev/hwrng
             ├─systemd-timesyncd.service
             │ └─252 /lib/systemd/systemd-timesyncd
             ├─nfs-statd.service
             │ └─160 /usr/sbin/rpc.statd -F
             ├─syslogd.service
             │ └─342 /sbin/syslogd
             ├─dbus.service
             │ └─145 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
             ├─system-serial\x2dgetty.slice
             │ └─serial-getty@ttyS0.service
             │   └─164 /sbin/agetty -8 -L ttyS0 115200 xterm
             ├─system-getty.slice
             │ └─getty@tty1.service
             │   └─163 /sbin/agetty -o -p -- \u --noclear tty1 linux
             ├─rpcbind.service
             │ └─149 /usr/sbin/rpcbind
             ├─systemd-logind.service
             │ └─147 /lib/systemd/systemd-logind
             ├─systemd-resolved.service
             │ └─157 /lib/systemd/systemd-resolved
             ├─crond.service
             │ └─151 /usr/sbin/crond -n
             ├─klogd.service
             │ └─158 /sbin/klogd
             ├─systemd-udevd.service
             │ └─129 /lib/systemd/systemd-udevd
             ├─atd.service
             │ └─148 /usr/sbin/atd -f
             ├─systemd-journald.service
             │ └─109 /lib/systemd/systemd-journald
             └─systemd-networkd.service
               └─135 /lib/systemd/systemd-networkd
Startup finished in 24.698s (kernel) + 2min 17.063s (userspace) = 2min 41.761s.
Target boot time 161.761 exceeds systemd's TimeoutStartSec 90
Comment 2 Kai Kang 2019-09-11 08:37:25 UTC 
There is no error there and just as it reports: 'timeout'. 

-- Logs begin at Wed 2019-09-11 07:52:05 UTC, end at Wed 2019-09-11 07:56:29 UTC. --
Sep 11 07:52:13 qemumips systemd[1]: Starting Rebuild Hardware Database...
Sep 11 07:53:43 qemumips systemd[1]: systemd-hwdb-update.service: Start operation timed out. Terminating.
Sep 11 07:53:43 qemumips systemd[1]: systemd-hwdb-update.service: Main process exited, code=killed, status=15/TERM
Sep 11 07:53:43 qemumips systemd[1]: systemd-hwdb-update.service: Failed with result 'timeout'.
Sep 11 07:53:43 qemumips systemd[1]: Failed to start Rebuild Hardware Database.


There is only one time failure during about 20 times qemu boot. But when I use command 'stress' to
produce high cpu load, it fails to start systemd-hwdb-update everytime.

I suggest to add 'timeout' to exceptions in the test cases.
Comment 3 Richard Purdie 2019-09-17 15:59:12 UTC 
seems mips is just slow. https://github.com/systemd/systemd/issues/13581
Comment 4 Ross Burton 2019-10-07 19:40:05 UTC 
Patch on the list.  Turns out we already generate the hwdb at rootfs time *and* have postinsts to do this after the event, so we just need to delete the service file like several other distros already do.