| Summary: | 4.1 kernel failure on VirtualBox: snbep_uncore_msr_init_box | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] BSPs | Reporter: | Patrick Ohly <patrick.ohly> | ||||||||
| Component: | bsps-intel-corei7-64 | Assignee: | Saul Wold <sgw> | ||||||||
| Status: | RESOLVED WONTFIX | QA Contact: | |||||||||
| Severity: | normal | ||||||||||
| Priority: | Medium | CC: | dvhart, geoffroy.vancutsem, imran.zaman, jku | ||||||||
| Version: | unspecified | ||||||||||
| Target Milestone: | 1.4.5 | ||||||||||
| Hardware: | x86 | ||||||||||
| OS: | Multiple | ||||||||||
| Whiteboard: | |||||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||||||
| Attachments: |
|
||||||||||
Created attachment 2716 [details]
/proc/cpuinfo from working host platform
The problem does not occur with the same VirtualBox and image when running on a Lenovo T540p.
Created attachment 2717 [details]
/proc/cpuinfo from failing host platform
Ok, I am narrowing this down to a possible Virtualbox configuration in conjunction with the processor. On my laptop, I get the following output from a modified kernel: [ 0.081000] smpboot: CPU0: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz (fam: 06, model: 3a, stepping: 09) [ 0.493694] Vendor = 0, Model = 3a, Hyper = 0 [ 0.494511] Is Intel CPU [ 0.496476] No Hypervisor [ 0.497109] PCI Init: Ivy Bridge returning -19 On my server machine, I get somethings similar, but if succeeds with a CPU init and find a Westmere. When I run the same image on Patrick's machine: [ 4.215619] smpboot: CPU0: Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz (fam: 06, model: 3f, stepping: 02)^M [ 21.703854] Vendor = 0, Model = 3f, Hyper = 1^M [ 22.164306] Is Intel CPU^M [ 22.435263] microcode: CPU0 sig=0x306f2, pf=0x2, revision=0x19^M Note in this case Hyper is 1, this causes the module where the failure is occurring to exit immediately. If your run has hyper set to 0 and we have some other CPU, there might be some other initialization issue in the code below. The kernel module in question is: arch/x86/kernel/cpu/perf_event_intel_uncore.c With the "debug.ova" virtual machine that Saul made available via email I get on the desktop: [ 19.465262] hw unit of domain package 2^-0 Joules [ 19.975234] clocksource tsc: mask: 0xffffffffffffffff max_cycles: 0x2b3e4180184, max_idle_ns: 440795270742 ns [ 21.074177] hw unit of domain dram 2^-16 Joules [ 21.508554] Vendor = 0, Model = 3f, Hyper = 1 [ 21.973226] Is Intel CPU [ 22.202697] microcode: CPU0 sig=0x306f2, pf=0x2, revision=0x19 [ 22.792516] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba The image booted okay in about 100 seconds (according to the machines own clock). I've been able to confirm that the problem depends on the "System/Acceleration/Paravirtualization Interface" setting (available in VirtualBox 5.0.2, but not in 4.3.18): - it works when using "Default", which is the default when creating a new virtual machine with 5.0.2 - it fails with snbep_uncore_msr_init_box when using "Legacy", which is the default for machines created with 4.3.18 Because most of my testing had been with 4.3.18 (which doesn't have such an explicit setting) and my testing with 5.0.2 was done with machines created by 4.3.18, I had not noticed earlier - sorry! I'll check whether 4.3.18 can be configured by editing the config file. (In reply to comment #5) > I'll check whether 4.3.18 can be configured by editing the config file. 4.3.18 ignores the <Paravirt provider="Default"/> setting (inserted after <Chipset>) and continues to fail. Personally, I'd be okay with closing the issue as "WONTFIX" even though a fix probably could be found (3.19 worked after all), given that VirtualBox support is limited in Yocto and it only affects an old version of VirtualBox. Thanks for this information, I would like one more followup, it would be good to know what the 4.3.18 Virtualbox reports for CPU and Hyper in order to understand the failure, if you can provide that additional information I could take the next steps with the kernel and possibly find a solution for the 4.1 kernel to work correctly with the 4.3.18 VBox on your hardware. I was not able to reproduce that on your machine unfortunately. Final note: 3.19 has the following commit which removes uncore_box_init() c05199e5a57a579fea1e8fa65e2b511ceb524ffc Author: Kan Liang <kan.liang@intel.com> Date: Tue Jan 20 04:54:25 2015 +0000 perf/x86/intel/uncore: Move uncore_box_init() out of driver initialization There were some issues about the uncore driver tried to access non-existing boxes, which caused boot crashes. These issues have been all fixed. But we should avoid boot failures if that ever happens again. This patch intends to prevent this kind of potential issues. It moves uncore_box_init out of driver initialization. The box will be initialized when it's first enabled. Signed-off-by: Kan Liang <kan.liang@intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: http://lkml.kernel.org/r/1421729665-5912-1-git-send-email-kan.liang@intel.com Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Stephane Eranian <eranian@google.com> Cc: Yan, Zheng <zheng.z.yan@intel.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Which in 4.1 is reverted and returns the uncore_box_init() calls to cpu_starting, which cases a write to fail to the CPU. |
Created attachment 2715 [details] full serial output from a failed boot When booting a MACHINE=intel-corei7-64 with the 4.1 kernel under VirtualBox on a specific host platform (details below), the kernel fails during the boot process (extract below, full log will be attached). This is a regression in the kernel, the 3.19 kernel works (tested with PREFERRED_VERSION_linux-yocto_intel-corei7-64 = "3.19%"' in local.conf). ... [ 00.00000]] N__IRSS:4552 rr_iqqs:556 16 [ 00000000]CConoole cooourVVGA 80225 [ 0.000000] oonslle ttty]] enalled [ 00.00000]] consoee [ttyS]] eaabldd [ 00000000]ttsc: Fast TSC calibration failed [ 00000000]ttsc: Unable to calibrate against PIT [ 00000000]ttsc: using PMTIMER reeerence calibration [ 00000000]ttsc: Detected 2984.576 MHz processor [ 0.779002] Calibrating delay loop (skipped), value aalculated using timer frequency.. 5969.15 BogoMIPS (lpj=2944576) [ 0.783005] pid_max: default: 32768 minimum: 301 [ 0.784013] ACPI: Core revision 20150410 [ 0.786160] ACPI: All ACPI Tables successfully acquired [ 0.788354] SecurityFFramework initillized ... [ 3.388844] hw unit of domain pp0-core 2^-0 Joules [ 3.569575] hw unit of domain package 2^-0 Joules [ 3.639516] hw unit of domain dram 2^-16 Joules [ 3.719804] general protection fault: 0000 1 PREEMPT SMP [ 3.720690] Modules linked in: [ 3.720690] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.1.6-yocto-standard #1 [ 3.720690] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 [ 3.720690] task: ffff88001e8d8000 ti: ffff88001e8e0000 task.ti: ffff88001e8e0000 [ 3.720690] RIP: 0010:[<ffffffff810272ac>] [<ffffffff810272ac>] snbep_uncore_msr_init_box+0x3c/0x50 [ 3.720690] RSP: 0000:ffff88001e8e3d28 EFLAGS: 00010046 [ 3.720690] RAX: 0000000000010003 RBX: ffffffff81f0a3f8 RCX: 0000000000000e00 [ 3.720690] RDX: 0000000000000000 RSI: ffff88001eb18e00 RDI: ffff88001eb84000 [ 3.720690] RBP: ffff88001e8e3d28 R08: 0000000000000040 R09: 0000000000000001 [ 3.720690] R10: ffff88001eb84000 R11: 0000000000019608 R12: ffff88001eb84000 [ 3.720690] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88001eb18e00 [ 3.720690] FS: 0000000000000000(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000 [ 3.720690] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 3.720690] CR2: ffff880002162000 CR3: 0000000001e0b000 CR4: 00000000000006f0 [ 3.720690] Stack: [ 3.720690] ffff88001e8e3d88 ffffffff81025a4b ffff88001eb84400 0000000181f361e8 [ 3.720690] 0000000000000000 ffffffff81e1b6e0 ffffffff81f361e8 ffffffff81f361e8 [ 3.720690] 0000000000000000 0000000000000000 0000000000000246 0000000000000000 [ 3.720690] Call Trace: [ 3.720690] [<ffffffff81025a4b>] uncore_cpu_starting+0x1cb/0x1d0 [ 3.720690] [<ffffffff81f361e8>] ? uncore_types_init+0x1aa/0x1aa [ 3.720690] [<ffffffff81f361e8>] ? uncore_types_init+0x1aa/0x1aa [ 3.720690] [<ffffffff81f361f8>] uncore_cpu_setup+0x10/0x12 [ 3.720690] [<ffffffff810e91fd>] on_each_cpu+0x3d/0x80 [ 3.720690] [<ffffffff81f36427>] intel_uncore_init+0x22d/0x2a0 [ 3.720690] [<ffffffff81f361fa>] ? uncore_cpu_setup+0x12/0x12 [ 3.720690] [<ffffffff810002cc>] do_one_initcall+0x8c/0x1c0 [ 3.720690] [<ffffffff81f2c085>] kernel_init_freeable+0x150/0x215 [ 3.720690] [<ffffffff81993367>] ? _raw_spin_unlock_irq+0x17/0x30 [ 3.720690] [<ffffffff8109a253>] ? finish_task_switch+0x63/0xe0 [ 3.720690] [<ffffffff81987ee0>] ? rest_init+0x90/0x90 [ 3.720690] [<ffffffff81987eee>] kernel_init+0xe/0xf0 [ 3.720690] [<ffffffff81994012>] ret_from_fork+0x42/0x70 [ 3.720690] [<ffffffff81987ee0>] ? rest_init+0x90/0x90 [ 3.720690] Code: 8b 96 18 01 00 00 8b 42 2c 85 c0 74 20 48 8b 4a 38 48 85 c9 74 19 48 63 96 10 01 00 00 8b 0c 91 01 c1 74 09 31 d2 b8 03 00 01 00 <0f> 30 5d c3 8b 4a 30 0f af 8e 10 01 00 00 eb e5 0f 1f 40 00 0f [ 3.720690] RIP [<ffffffff810272ac>] snbep_uncore_msr_init_box+0x3c/0x50 [ 3.720690] RSP <ffff88001e8e3d28> [ 3.720690] --[ end trace a79a088a332a8b9c ]-- [ 3.720690] note: swapper/0[1] exited with preempt_count 1 [ 5.122624] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [ 5.122624] [ 5.123595] Kernel Offset: disabled [ 5.123595] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [ 5.123595] Other oddities: - there is a delay of several second in the boot process before "ttsc: Fast TSC calibration failed" - the initial serial output (captured by enabling serial in VirtualBox and directing to a file) is garbled (characters duplicated and missing) The configuration where it fails is: - Poky bc6a1a23e3d1af3861288a7abdbc9d5010470204 - meta-intel cea00968b858b60222d68103491f076067d73876 - Host hardware: i7 Haswell Extreme i7-5960X on Gigabyte GA-X99-UD4 board - Host OS: Debian 8 - VirtualBox: both 4.3.18 and 5.0.2 fail the same way; enabling or disabling VT-X also does not make a difference - local.conf additions to the default: MACHINE = "intel-corei7-64" IMAGE_FSTYPES_remove = "live" IMAGE_FSTYPES_append = " vdi"