Bug 12570

Summary: [stable/pyro-next] nightly-x86-64-lsb fails
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Armin Kuster <akuster>
Component: coreAssignee: Armin Kuster <akuster>
Status: RESOLVED FIXED QA Contact:
Severity: critical    
Priority: High CC: juro.bystricky, meta.mr.watcher, meta.watcher, stephano
Version: 2.3.4   
Target Milestone: 2.3.4   
Hardware: x86   
OS: Multiple   
URL: https://autobuilder.yocto.io/builders/nightly-x86-64-lsb/builds/806/steps/Running%20Sanity%20Tests/logs/stdio
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Armin Kuster 2018-02-25 16:25:39 UTC
selftest failed for unknown reason on debian8.yocto.io

NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Sending SIGTERM to runqemu
ERROR: core-image-lsb-1.0-r0 do_testimage: testimage received SIGTERM, shutting down...
ERROR: core-image-lsb-1.0-r0 do_testimage: core-image-lsb - FAILED - tests were interrupted during execution
ERROR: core-image-lsb-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/temp/log.do_testimage.55682
NOTE: recipe core-image-lsb-1.0-r0: task do_testimage: Failed
ERROR: Task (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/recipes-extended/images/core-image-lsb.bb:do_testimage) failed with exit code '1'
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: testimage received SIGTERM, shutting down...
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: core-image-lsb-sdk - FAILED - tests were interrupted during execution
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/temp/log.do_testimage.55681
NOTE: recipe core-image-lsb-sdk-1.0-r0: task do_testimage: Failed
ERROR: Task (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/recipes-extended/images/core-image-lsb-sdk.bb:do_testimage) failed with exit code '1'
Comment 1 Armin Kuster 2018-02-25 16:26:19 UTC
Can not reproduce on Ubuntu 17. I don't see any errors in console output.
Comment 2 Armin Kuster 2018-02-28 18:53:43 UTC
backported several runqemu logging patches
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=stable/pyro-next

I get more debug out but it did not fix this issue nor is it giving more info.
Comment 3 Armin Kuster 2018-02-28 18:54:13 UTC
on the build host.

[    3.096996] EXT4-fs (vda): couldn't mount as ext3 due to feature incompatibilities
[    3.098375] EXT4-fs (vda): couldn't mount as ext2 due to feature incompatibilities
[    3.101180] EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: (null)
[    3.103241] VFS: Mounted root (ext4 filesystem) on device 253:0.
[    3.104504] devtmpfs: mounted
[    3.105734] Freeing unused kernel memory: 1356K
[    3.106475] Write protecting the kernel read-only data: 14336k
[    3.107815] Freeing unused kernel memory: 588K
[    3.108786] Freeing unused kernel memory: 368K
INIT: version 2.88 booting
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
[  100.320053] random: nonblocking pool is initialized
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
INIT: PANIC: segmentation violation! sleeping for 30 seconds.
Comment 4 Armin Kuster 2018-02-28 19:07:35 UTC
Command run:
 Running tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin//qemu-system-x86_64 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -drive file=/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/build/tmp/deploy/images/qemux86-64/core-image-lsb-qemux86-64.ext4,if=virtio,format=raw -vga vmware -show-cursor -usb -usbdevice tablet -device virtio-rng-pci  -serial tcp:127.0.0.1:59521 -pidfile pidfile_29068  -cpu core2duo -enable-kvm -m 256 -serial tcp:127.0.0.1:59231 -snapshot -kernel tmp/deploy/images/qemux86-64/bzImage--4.1.49+git0+4e12cb8f8e_a315195324-r0.2-qemux86-64-20180224035021.bin -append 'root=/dev/vda rw highres=off  mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1 console=tty1 console=ttyS0,115200n8 printk.time=1'
Comment 5 Armin Kuster 2018-02-28 19:21:14 UTC
I just reproduced this at home:

I see it if QEMU_USE_KVM = "True"


cuting '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/oe-testimage-repo/noarch' ...
NOTE: core-image-lsb-1.0-r0 do_testimage: Executing '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/oe-testimage-repo/noarch' ...
NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Executing '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/oe-testimage-repo/core2_64' ...
NOTE: core-image-lsb-1.0-r0 do_testimage: Executing '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/oe-testimage-repo/core2_64' ...
NOTE: core-image-lsb-sdk-1.0-r0 do_testimage: Executing '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/oe-testimage-repo/qemux86_64' ...
NOTE: core-image-lsb-1.0-r0 do_testimage: Executing '/home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/recipe-sysroot-native/usr/bin/createrepo_c --update -q /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/oe-testimage-repo/qemux86_64' ...
WARNING: core-image-lsb-sdk-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201802281114_qemu
WARNING: core-image-lsb-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201802281114_qemu
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: testimage received SIGTERM, shutting down...
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: core-image-lsb-sdk - FAILED - tests were interrupted during execution
ERROR: core-image-lsb-sdk-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: Logfile of failure stored in: /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-sdk/1.0-r0/temp/log.do_testimage.17645
ERROR: Task (/home/akuster/oss/maint/poky/meta/recipes-extended/images/core-image-lsb-sdk.bb:do_testimage) failed with exit code '1'
ERROR: core-image-lsb-1.0-r0 do_testimage: testimage received SIGTERM, shutting down...
ERROR: core-image-lsb-1.0-r0 do_testimage: core-image-lsb - FAILED - tests were interrupted during execution
ERROR: core-image-lsb-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: Logfile of failure stored in: /home/akuster/oss/maint/poky/build/tmp/work/qemux86_64-poky-linux/core-image-lsb/1.0-r0/temp/log.do_testimage.17646
ERROR: Task (/home/akuster/oss/maint/poky/meta/recipes-extended/images/core-image-lsb.bb:do_testimage) failed with exit code '1'
Comment 6 Juro Bystricky 2018-02-28 19:58:34 UTC
I am pretty sure I tried (and failed) to reproduce this with 
QEMU_ENABLE_KVM="True" (Ubuntu 16.04)
I will re-try, just to make sure.
Comment 7 Armin Kuster 2018-02-28 22:36:56 UTC
This fails on my system with this simple case:
runqemu qemux86-64 kvm

without kvm, it boots fine.
Comment 8 Armin Kuster 2018-03-01 01:07:54 UTC
This appears to be released to the kernel. removing the 4.1.49 updated, things work. I now have a range of commits to look at.
Comment 9 Armin Kuster 2018-03-01 01:10:59 UTC
looks like it in the meltdown changes. commit to 4.1.49 sync works.
Comment 10 Armin Kuster 2018-03-02 16:22:25 UTC
found a fix for the 4.1. 

runny some build test sequences before I send a pull request for 4.1
Comment 11 Armin Kuster 2018-03-05 03:17:04 UTC
kernel pull request sent.
Comment 12 Armin Kuster 2018-03-08 08:15:27 UTC
patches sent.

New build under way