Bug 10436

Summary: core-image-rt for qemux86 fails to boot
Product: [Build System, Metadata & Runtime] BSPs Reporter: Maxin B. John <maxin.john>
Component: bsps-oe-coreAssignee: Paul Gortmaker <paulg>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: bruce.ashfield, paulg
Version: 2.2   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
bisect log none

Description Maxin B. John 2016-10-14 09:55:40 UTC 
While trying to boot core-image-rt for qemux86, the boot process doesn't reach upto the login prompt. It hangs after printing this (No kernel panic): 

"Performance Events: PMU not available due to virtualization, using software events only"

Build machine: Fedora release 22 
 
Build Configuration:
BB_VERSION        = "1.32.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    
meta-selftest     = "master:371ab635086e2878ada1f00a59c7ec826481ba0a"


Image is built with the following settings in local.conf:
MACHINE ?= "qemux86"
PREFERRED_PROVIDER_virtual/kernel="linux-yocto-rt"

Please note that "core-image-rt" for "qemux86-64" boots as expected. 

In the case of "qemux86" boot failure, switching from sysv to systemd has the same effect (both fails to boot)


$ runqemu qemux86  core-image-rt nographic
runqemu - INFO - Assuming MACHINE = qemux86
runqemu - INFO - Running MACHINE=qemux86 bitbake -e...
runqemu - INFO - MACHINE: qemux86
runqemu - INFO - DEPLOY_DIR_IMAGE: /media/poky/build/tmp/deploy/images/qemux86
runqemu - INFO - CONFFILE: /media/poky/build/tmp/deploy/images/qemux86/core-image-rt-qemux86.qemuboot.conf
runqemu - INFO - Continuing with the following parameters:

KERNEL: [/media/poky/build/tmp/deploy/images/qemux86/bzImage]
MACHINE: [qemux86]
FSTYPE: [ext4]
ROOTFS: [/media/poky/build/tmp/deploy/images/qemux86/core-image-rt-qemux86.ext4]
CONFFILE: [/media/poky/build/tmp/deploy/images/qemux86/core-image-rt-qemux86.qemuboot.conf]

runqemu - INFO - Running /usr/sbin/ip link...
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock...
runqemu - INFO - Using preconfigured tap device tap0
runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap0.skip to make runqemu skip tap0.
runqemu - INFO - Running ldd /media/poky/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-system-i386...
runqemu - INFO - Running /media/poky/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-system-i386 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no  -nographic  -cpu qemu32 -m 256 -drive file=/media/poky/build/tmp/deploy/images/qemux86/core-image-rt-qemux86.ext4,if=virtio,format=raw -vga vmware -show-cursor -usb -usbdevice tablet -device virtio-rng-pci -kernel /media/poky/build/tmp/deploy/images/qemux86/bzImage -append 'root=/dev/vda rw highres=off  console=ttyS0 mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1'
[    0.000000] Linux version 4.8.0-rt1-yocto-preempt-rt (maxin@mbabyjoh-desk.ger.corp.intel.com) (gcc version 6.2.0 (GCC) ) #1 SMP PREEMPT RT Thu Oct 13 18:47:44 EEST 2016
[    0.000000] x86/fpu: Legacy x87 FPU detected.
[    0.000000] x86/fpu: Using 'eager' FPU context switches.
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000000ffdbfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000ffdc000-0x000000000fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[    0.000000] e820: user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] user: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000000ffdbfff] usable
[    0.000000] user: [mem 0x000000000ffdc000-0x000000000fffffff] reserved
[    0.000000] user: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] SMBIOS 2.8 present.
[    0.000000] e820: last_pfn = 0xffdc max_arch_pfn = 0x100000
[    0.000000] MTRR: Disabled
[    0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.000000] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC  
[    0.000000] found SMP MP-table at [mem 0x000f6bc0-0x000f6bcf] mapped at [c00f6bc0]
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000000F69D0 000014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 0x000000000FFE18DC 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACP 0x000000000FFE17B8 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.000000] ACPI: DSDT 0x000000000FFE0040 001778 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACS 0x000000000FFE0000 000040
[    0.000000] ACPI: APIC 0x000000000FFE182C 000078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.000000] ACPI: HPET 0x000000000FFE18A4 000038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.000000] 0MB HIGHMEM available.
[    0.000000] 255MB LOWMEM available.
[    0.000000]   mapped low ram: 0 - 0ffdc000
[    0.000000]   low ram: 0 - 0ffdc000
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.000000]   Normal   [mem 0x0000000001000000-0x000000000ffdbfff]
[    0.000000]   HighMem  empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.000000]   node   0: [mem 0x0000000000100000-0x000000000ffdbfff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000000ffdbfff]
[    0.000000] Using APIC driver default
[    0.000000] ACPI: PM-Timer IO Port: 0x608
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] e820: [mem 0x10000000-0xfffbffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] percpu: Embedded 20 pages/cpu @cfd43000 s52256 r0 d29664 u81920
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 64762
[    0.000000] Kernel command line: root=/dev/vda rw highres=off  console=ttyS0 mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Initializing CPU#0
[    0.000000] Initializing HighMem for node 0 (00000000:00000000)
[    0.000000] Memory: 244000K/261608K available (9172K kernel code, 773K rwdata, 3208K rodata, 768K init, 516K bss, 17608K reserved, 0K cma-reserved, 0K highmem)
[    0.000000] virtual kernel memory layout:
[    0.000000]     fixmap  : 0xfff16000 - 0xfffff000   ( 932 kB)
[    0.000000]     pkmap   : 0xff800000 - 0xffc00000   (4096 kB)
[    0.000000]     vmalloc : 0xd07dc000 - 0xff7fe000   ( 752 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xcffdc000   ( 255 MB)
[    0.000000]       .init : 0xc1cdc000 - 0xc1d9c000   ( 768 kB)
[    0.000000]       .data : 0xc18f53c6 - 0xc1cda700   (3988 kB)
[    0.000000]       .text : 0xc1000000 - 0xc18f53c6   (9172 kB)
[    0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[    0.000000] 	RCU kthread priority: 1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS:2304 nr_irqs:256 16
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 3000.015 MHz processor
[    0.004574] Calibrating delay loop (skipped), value calculated using timer frequency.. 6000.03 BogoMIPS (lpj=3000015)
[    0.004679] pid_max: default: 32768 minimum: 301
[    0.005174] ACPI: Core revision 20160422
[    0.040102] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.040107] 
[    0.040884] Security Framework initialized
[    0.041353] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.041382] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.055534] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.055551] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.170537] Freeing SMP alternatives memory: 36K (c1d9c000 - c1da5000)
[    0.176615] ftrace: allocating 33605 entries in 66 pages
[    0.275404] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.280071] smpboot: Max logical packages: 1
[    0.280439] Enabling APIC mode:  Flat.  Using 1 I/O APICs
[    0.283000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.293000] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (family: 0x6, model: 0x6, stepping: 0x3)
[    0.293214] Performance Events: PMU not available due to virtualization, using software events only.
Comment 1 Maxin B. John 2016-10-17 10:44:44 UTC 
With MACHINE ?= "qemux86"
PREFERRED_PROVIDER_virtual/kernel="linux-yocto-rt"

core-image-minimal fails to boot as well.
Comment 2 California Sullivan 2016-10-18 21:48:27 UTC 
Created attachment 3478 [details]
bisect log

Unfortunately it appears that qemux86 has failed to boot the preempt-rt kernel since it was added in cb169d3a.

See the attached bisect log.
Comment 3 Bruce Ashfield 2016-10-18 21:57:01 UTC 
Paul gortmaker is refreshing the -rt series this week.  We'll boot test 32bit x86 as part of that update.
Comment 4 Bruce Ashfield 2016-10-21 17:41:04 UTC 
This is broken in the mainline -rt, PaulG has sorted it out and will provide a fix.
Comment 5 Paul Gortmaker 2016-10-21 18:17:26 UTC 
Fix details:

https://lkml.kernel.org/r/20161021181051.25437-1-paul.gortmaker@windriver.com

Note that the GET_THREAD_INFO macro won't be in a shared header until I've handed off the other updates to Bruce, so if you want to test locally, you'll have to copy the GET_THREAD_INFO macro from the x86-64 file.
Comment 6 Maxin B. John 2016-10-24 08:29:02 UTC 
After applying the patch and copying GET_THREAD_INFO macro, preempt-rt kernel for qemux86 builds and boots successfully.

root@qemux86:~# uname -a
Linux qemux86 4.8.0-rt1-yocto-preempt-rt #1 SMP PREEMPT RT Mon Oct 24 11:19:43 EEST 2016 i686 i686 i386 GNU/Linux

Thank you!
Comment 7 Paul Gortmaker 2016-10-27 03:18:29 UTC 
Fix is also now a part of upstream preempt-rt release.

https://lkml.kernel.org/r/20161024090322.h7jhlgm34tyt4uze@linutronix.de

As it was a late breaking change, it is at the bottom of the changelog.