| Summary: | Occasional qemuarm kernel panic (linux 4.1, gcc 4.9.3) | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Richard Purdie <richard.purdie> |
| Component: | kernel | Assignee: | Aníbal Limón <anibal.limon> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | anibal.limon, bruce.ashfield, elizabeth.flanagan, jku, juro.bystricky, tom.zanussi |
| Version: | 1.8 | ||
| Target Milestone: | 2.0 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Richard Purdie
2015-07-28 11:00:35 UTC
ERROR: Qemu log output from /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-sato-sdk/1.0-r0/testimage/qemu_boot_log.20150728064520:
Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0x0
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 4.1.2-yocto-standard (pokybuild@opensuse131) (gcc version 4.9.3 (GCC) ) #1 PREEMPT Wed Jul 22 08:52:25 GMT 2015
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00093177
CPU: VIVT data cache, VIVT instruction cache
Machine: ARM-Versatile PB
Memory policy: Data cache writeback
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32480
Kernel command line: root=/dev/sda rw console=ttyAMA0,115200 console=tty ip=192.168.7.6::192.168.7.5:255.255.255.0 mem=128M highres=off rootfstype=ext4 console=tty1 console=ttyS0,115200n8
PID hash table entries: 512 (order: -1, 2048 bytes)
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 119480K/131072K available (6880K kernel code, 514K rwdata, 1868K rodata, 360K init, 631K bss, 11592K reserved, 0K cma-reserved)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xc8800000 - 0xff000000 ( 872 MB)
lowmem : 0xc0000000 - 0xc8000000 ( 128 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc0893434 (8750 kB)
.init : 0xc0894000 - 0xc08ee000 ( 360 kB)
.data : 0xc08ee000 - 0xc096ebc4 ( 515 kB)
.bss : 0xc096ebc4 - 0xc0a0cb40 ( 632 kB)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
Additional per-CPU info printed with stalls.
NR_IRQS:224
VIC @f1140000: id 0x00041190, vendor 0x41
FPGA IRQ chip 0 "SIC" @ f1003000, 13 irqs, parent IRQ: 63
clocksource timer3: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
Console: colour dummy device 80x30
console [tty0] enabled
Calibrating delay loop... 471.85 BogoMIPS (lpj=2359296)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
Initializing cgroup subsys blkio
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys debug
CPU: Testing write buffer coherency: ok
ftrace: allocating 22469 entries in 66 pages
Setting up static identity map for 0x8400 - 0x8458
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 1 part 10 variant 9 rev 0
clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
xor: measuring software checksum speed
arm4regs : 596.000 MB/sec
8regs : 317.600 MB/sec
32regs : 762.800 MB/sec
xor: using function: 32regs (762.800 MB/sec)
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
Serial: AMBA PL011 UART driver
dev:f1: ttyAMA0 at MMIO 0x101f1000 (irq = 44, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
dev:f2: ttyAMA1 at MMIO 0x101f2000 (irq = 45, base_baud = 0) is a PL011 rev1
dev:f3: ttyAMA2 at MMIO 0x101f3000 (irq = 46, base_baud = 0) is a PL011 rev1
fpga:09: ttyAMA3 at MMIO 0x10009000 (irq = 70, base_baud = 0) is a PL011 rev1
PCI core found (slot 11)
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x50000000-0x5fffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff pref]
pci_bus 0000:00: root bus resource [io 0x1000-0xffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
PCI: bus0: Fast back to back transfers disabled
pci 0000:00:0d.0: BAR 2: assigned [mem 0x50000000-0x50001fff]
pci 0000:00:0d.0: BAR 1: assigned [mem 0x50002000-0x500023ff]
pci 0000:00:0c.0: BAR 0: assigned [mem 0x50002400-0x500024ff]
pci 0000:00:0d.0: BAR 0: assigned [io 0x1000-0x10ff]
raid6: int32x1 gen() 249 MB/s
raid6: int32x1 xor() 153 MB/s
raid6: int32x2 gen() 256 MB/s
raid6: int32x2 xor() 157 MB/s
raid6: int32x4 gen() 110 MB/s
raid6: int32x4 xor() 91 MB/s
raid6: int32x8 gen() 123 MB/s
raid6: int32x8 xor() 78 MB/s
raid6: using algorithm int32x2 gen() 256 MB/s
raid6: .... xor() 157 MB/s, rmw enabled
raid6: using intx1 recovery algorithm
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Switched to clocksource timer3
NET: Registered protocol family 2
TCP established hash table entries: 1024 (order: 0, 4096 bytes)
TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
TCP: Hash tables configured (established 1024 bind 1024)
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:00:0c.0: enabling device (0100 -> 0102)
futex hash table entries: 256 (order: -1, 3072 bytes)
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
aufs 4.1-20150629
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
clcd-pl11x dev:20: PL110 rev0 at 0x10120000
clcd-pl11x dev:20: Versatile hardware, VGA display
Console: switching to colour frame buffer device 80x30
brd: module loaded
sym53c8xx 0000:00:0d.0: enabling device (0100 -> 0103)
sym0: <895a> rev 0x0 at pci 0000:00:0d.0 irq 94
sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
sym0: SCSI BUS has been reset.
scsi host0: sym-2.2.3
sym0: unknown interrupt(s) ignored, ISTAT=0x5 DSTAT=0x80 SIST=0x0
scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 2.3. PQ: 0 ANSI: 5
scsi target0:0:0: tagged command queuing enabled, command queue depth 16.
scsi target0:0:0: Beginning Domain Validation
scsi target0:0:0: Domain Validation skipping write tests
scsi target0:0:0: Ending Domain Validation
scsi 0:0:2:0: CD-ROM QEMU QEMU CD-ROM 2.3. PQ: 0 ANSI: 5
scsi target0:0:2: tagged command queuing enabled, command queue depth 16.
scsi target0:0:2: Beginning Domain Validation
scsi target0:0:2: Domain Validation skipping write tests
scsi target0:0:2: Ending Domain Validation
smc91x.c: v1.1, sep 22 2004 by Nicolas Pitre <nico@fluxnic.net>
sd 0:0:0:0: [sda] 4788610 512-byte logical blocks: (2.45 GB/2.28 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
smc91x smc91x.0 eth0: SMC91C11xFD (rev 1) at c8a64000 IRQ 57
[nowait]
smc91x smc91x.0 eth0: Ethernet addr: 52:54:00:12:34:56
sd 0:0:0:0: [sda] Attached SCSI disk
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-pci 0000:00:0c.0: OHCI PCI host controller
ohci-pci 0000:00:0c.0: new USB bus registered, assigned bus number 1
ohci-pci 0000:00:0c.0: irq 93, io mem 0x50002400
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 3 ports detected
uhci_hcd: USB Universal Host Controller Interface driver
mousedev: PS/2 mouse device common for all mice
rtc-pl031 dev:e8: rtc core: registered pl031 as rtc0
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
md: raid10 personality registered for level 10
md: multipath personality registered for level -4
md: faulty personality registered for level -5
device-mapper: ioctl: 4.31.0-ioctl (2015-3-12) initialised: dm-devel@redhat.com
mmci-pl18x fpga:05: No vmmc regulator found
mmci-pl18x fpga:05: No vqmmc regulator found
mmci-pl18x: probe of fpga:05 failed with error -22
mmci-pl18x fpga:0b: No vmmc regulator found
mmci-pl18x fpga:0b: No vqmmc regulator found
mmci-pl18x: probe of fpga:0b failed with error -22
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
oprofile: no performance counters
oprofile: using timer interrupt.
u32 classifier
Actions configured
NET: Registered protocol family 10
input: AT Raw Set 2 keyboard as /devices/fpga:06/serio0/input/input0
sit: IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
Key type dns_resolver registered
Btrfs loaded
console [netcon0] enabled
netconsole: network logging started
rtc-pl031 dev:e8: setting system clock to 2015-07-28 06:46:35 UTC (1438065995)
smc91x smc91x.0 eth0: link up
IP-Config: Complete:
device=eth0, hwaddr=52:54:00:12:34:56, ipaddr=192.168.7.6, mask=255.255.255.0, gw=192.168.7.5
host=192.168.7.6, domain=, nis-domain=(none)
bootserver=255.255.255.255, rootserver=255.255.255.255, rootpath=
usb 1-1: new full-speed USB device number 2 using ohci-pci
input: Wacom Penpartner Pen as /devices/pci0000:00/0000:00:0c.0/usb1/1-1/1-1:1.0/0003:056A:0000.0001/input/input3
wacom 0003:056A:0000.0001: : USB HID v10.01 Mouse [QEMU Wacom PenPartner] on usb-0000:00:0c.0-1/input0
input: ImExPS/2 Generic Explorer Mouse as /devices/fpga:07/serio1/input/input2
md: Waiting for all devices to be available before autodetect
md: If you don't use raid, use raid=noautodetect
md: Autodetecting RAID arrays.
md: Scanned 0 and added 0 devices.
md: autorun ...
md: ... autorun DONE.
EXT4-fs (sda): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 8:0.
devtmpfs: mounted
Freeing unused kernel memory: 360K (c0894000 - c08ee000)
udevd[70]: starting version 182
random: nonblocking pool is initialized
sd 0:0:0:0: [sda] ABORT operation started
sd 0:0:0:0: ABORT operation timed-out.
Kernel panic - not syncing: assertion "i && sym_get_cam_status(cp->cmd) == DID_SOFT_ERROR" failed: file "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work-shared/qemuarm/kernel-source/drivers/scsi/sym53c8xx_2/sym_hipd.c", line 3399
CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.2-yocto-standard #1
Hardware name: ARM-Versatile PB
[<c0016ba4>] (unwind_backtrace) from [<c0013044>] (show_stack+0x20/0x24)
[<c0013044>] (show_stack) from [<c0634b20>] (dump_stack+0x20/0x28)
[<c0634b20>] (dump_stack) from [<c0630b08>] (panic+0x94/0x1ec)
[<c0630b08>] (panic) from [<c042022c>] (sym_interrupt+0x59c/0x17a0)
[<c042022c>] (sym_interrupt) from [<c041bb64>] (sym53c8xx_intr+0x54/0xbc)
[<c041bb64>] (sym53c8xx_intr) from [<c0059aa8>] (handle_irq_event_percpu+0xb8/0x280)
[<c0059aa8>] (handle_irq_event_percpu) from [<c0059ce0>] (handle_irq_event+0x70/0x90)
[<c0059ce0>] (handle_irq_event) from [<c005cb3c>] (handle_level_irq+0xe8/0x14c)
[<c005cb3c>] (handle_level_irq) from [<c00591ec>] (generic_handle_irq+0x30/0x40)
[<c00591ec>] (generic_handle_irq) from [<c03a5588>] (fpga_irq_handle+0x9c/0xa8)
[<c03a5588>] (fpga_irq_handle) from [<c00591ec>] (generic_handle_irq+0x30/0x40)
[<c00591ec>] (generic_handle_irq) from [<c0059430>] (__handle_domain_irq+0xa0/0xdc)
[<c0059430>] (__handle_domain_irq) from [<c0009450>] (vic_handle_irq+0x74/0x98)
[<c0009450>] (vic_handle_irq) from [<c063a140>] (__irq_svc+0x40/0x6c)
Exception stack(0xc08efe28 to 0xc08efe70)
fe20: 00000000 c078b4d3 00000101 20000113 c090ae90 ffff9883
fe40: 00000202 c096f300 00000001 c08eff30 0000000a c08efebc c08efe60 c08efe70
fe60: c0395f50 c0024ea0 20000113 ffffffff
[<c063a140>] (__irq_svc) from [<c0024ea0>] (__do_softirq+0x84/0x344)
[<c0024ea0>] (__do_softirq) from [<c0025410>] (irq_exit+0x94/0xbc)
[<c0025410>] (irq_exit) from [<c0059438>] (__handle_domain_irq+0xa8/0xdc)
[<c0059438>] (__handle_domain_irq) from [<c0009450>] (vic_handle_irq+0x74/0x98)
[<c0009450>] (vic_handle_irq) from [<c063a140>] (__irq_svc+0x40/0x6c)
Exception stack(0xc08eff30 to 0xc08eff78)
ff20: 00000000 00093177 00092177 60000013
ff40: c08ee000 00000000 c08f0080 c08f0080 c08cfab8 c7ffc840 008cce44 c08eff84
ff60: 600000d3 c08eff78 c0010268 c0010274 60000013 ffffffff
[<c063a140>] (__irq_svc) from [<c0010274>] (arch_cpu_idle+0x3c/0x44)
[<c0010274>] (arch_cpu_idle) from [<c004da4c>] (cpu_startup_entry+0x22c/0x280)
[<c004da4c>] (cpu_startup_entry) from [<c063022c>] (rest_init+0x80/0x98)
[<c063022c>] (rest_init) from [<c0894ca4>] (start_kernel+0x36c/0x3dc)
---[ end Kernel panic - not syncing: assertion "i && sym_get_cam_status(cp->cmd) == DID_SOFT_ERROR" failed: file "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work-shared/qemuarm/kernel-source/drivers/scsi/sym53c8xx_2/sym_hipd.c", line 3399
ERROR: Function failed: core-image-sato-sdk - FAILED to start qemu - check the task log and the boot log
Other builds at the same time with the same kernel worked so its an intermittent issue. (In reply to comment #2) > Other builds at the same time with the same kernel worked so its an > intermittent issue. Looks like fun. I'll start from the backtrace and see if anything shows up. There is another -stable update to 4.1, so I'll apply it as well. One additional question though: has this ever been seen on subsequent runs with the same qemu ? (it's probably not something that gets done in our QA tests, but I was just wondering, since that will help isolate this to a race, versus compiler, versus something else issue). seems like others ran into a similar problem: https://www.mail-archive.com/nix-commits@lists.science.uu.nl/msg20945.html This is not really a segfault but a kernel panic due to an assertion: ---[ end Kernel panic - not syncing: assertion "i && sym_get_cam_status(cp->cmd) == DID_SOFT_ERROR" failed: file "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work-shared/qemuarm/kernel-source/drivers/scsi/sym53c8xx_2/sym_hipd.c", line 3399 It appears Juro has answered the NEEDINFO. If more information is needed please move the bug back to NEEDINFO. This seems to continue to be an issue: https://autobuilder.yoctoproject.org/main/builders/nightly-arm-lsb/builds/458/steps/Running%20Sanity%20Tests/logs/stdio *** Bug 8269 has been marked as a duplicate of this bug. *** The common denominator here seems to be that it is failing on opensuse131 workers. Searching i found many discussions that talks about problems on SCSI sym driver [1][2]. We use virtio drivers into qemuarm64 and due to SCSI issues, i'm starting to enable virtio blk device into runqemu also we need to enable kernel config. [1] https://lists.gnu.org/archive/html/qemu-devel/2007-03/msg00093.html [2] https://lists.nongnu.org/archive/html/qemu-devel/2010-07/msg01473.html |