| Summary: | oprofile sampling doesn't work on qemux86 in the default event-based interrupt mode | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Dexuan Cui <dexuan.cui> |
| Component: | devtools / tool chain | Assignee: | Edwin Zhai <edwin.zhai> |
| Status: | RESOLVED WONTFIX | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | dexuan.cui, dvhart, edwin.zhai, jessica.zhang, lianhao.lu, meta.mr.watcher, meta.watcher, nitin.a.kamble, sgw, shane.wang |
| Version: | 1.0 | ||
| Target Milestone: | 1.1 | ||
| Hardware: | x86 | ||
| OS: | x86 | ||
| Whiteboard: | could be fixed in 1 week (8/4) | ||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | --- | |
|
Description
Dexuan Cui
2011-03-09 00:00:22 UTC
(In reply to comment #0) > On qemux86, we got the following issue: > # opcontrol --init > # opcontrol --no-vmlinux > # opcontrol --start > # find / >/dev/null; ls -R / >/dev/null > # opcontrol --stop > # opreport > opreport error: No sample file found: try running opcontrol --dump > or specify a session containing sample files > Now we can find /var/lib/oprofile/samples/current/ is empty. > I tried > http://autobuilder.yoctoproject.org/nightly/20110305-4/machines/qemu/ and only > found qemux86 has the issue. qemuarm/ppc/mips are ok. > I also tried sugarbay build and it's also ok. oprofile has 2 ways for interrupt generation: timer based and event based. qemux86 emulates a cpu of Pentium II and oprofile uses event based method and it doesn't work for an unknown reason. If I force the x86 kernel to use timer based method by passing a kernel parameter oprofile.timer=1, oprofile can work fine. qemuarm/ppc/mips are using timer based method, so they are ok with oprofile. I tried qemux86 images built in Jan and last Oct and oprofile doesn't work, either. So I doubt oprofile worked on qemux86? Jessica/Nitin, did you try that? Lianhao doesn't recall he evev ran oprofile successfully in qemux86. At present I think there are 2 possible causes: 1) qemux86 doesn't emulate event based method (PMU or something similar?) or doesn't do that properly; 2) oprofile module in Linux kernel is buggy -- this is unlikely. I'm trying to analyze qemu code. And I'll ask if qemux86 supports event based method fine or not in qemu community. Dexuan, I can't be 100% sure that oprofrile was working back at last Oct. but frankly I' quite shocked to hear that it wasn't working since x86 is one of the most common tested arch in our team and there's no issue reported regarding it. Also, I'm confused by your comment, you said if you change x86 to be timer based oprofile works fine. But qemux86 is Pentium II based and support event based interruption. Isn't there some contradiction in your statement. Are you saying if you change to timer based, oproile on x86 will work but the sample data collected is in correct? (In reply to comment #2) > Dexuan, I can't be 100% sure that oprofrile was working back at last Oct. but > frankly I' quite shocked to hear that it wasn't working since x86 is one of the > most common tested arch in our team and there's no issue reported regarding it. Yes, I'm also surprised... Sorry for not catching this issue early. Actually I don't test oprofile regularly since it's stable (the last upgrade occured in last Jun). I can't be 100% sure, but my impression was it worked on qemux86 then ,e.g., I think "Bug 255 - oprofile-server can NOT be launched within Qemu through Eclipse IDE" implies we did try oprofile on qemux86. > Also, I'm confused by your comment, you said if you change x86 to be timer > based oprofile works fine. Yes. > But qemux86 is Pentium II based and support event > based interruption. Isn't there some contradiction in your statement. Since Pentirm II supports event based interruption, by default oprofile will use that, but, at present that doesn't work for some unknown cause I'm trying to figure out. > Are you saying if you change to timer based, oproile on x86 will work but > the sample data collected is in correct? If I changed to timer based interrupt, oprofile on qemux86 works and the sampled data are correct. Dexuan, since timer based interrupt works fine on x86 and seems it's a fairly easy fix, can you just change to it for 1.0 release. I appreciate you hard working on fixing the issue, so if you have other higher priority bug and other things for 1.0, I'd suggest you open another bug to track that oprofile won't work on qemux86 with even based interrupt and fix it for post 1.0. (In reply to comment #4) > Dexuan, since timer based interrupt works fine on x86 and seems it's a fairly > easy fix, can you just change to it for 1.0 release. I appreciate you hard > working on fixing the issue, so if you have other higher priority bug and other > things for 1.0, I'd suggest you open another bug to track that oprofile won't > work on qemux86 with even based interrupt and fix it for post 1.0. Ok, I agree forcing qemux86 and (qemux86-64? I need to test) to use timer based interrupt for now can be a last resort. This fix needs to modify poky-qemu script to passing 'oprofile.timer=1' to qemux86's kernel. I've changed the title of this bug to clarify this only happens on event baseed mode. So after I push the fix, we could lower the bug's priority and/or mark it post-1.0. The following commit addresses this as a workaround by usign a timer interrupt mode. http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=f11e479b1702471e3e032e8dde75a4f343949ba0 See last comment (In reply to comment #6) > The following commit addresses this as a workaround by usign a timer interrupt > mode. > http://git.pokylinux.org/cgit/cgit.cgi/poky/commit/?id=f11e479b1702471e3e032e8dde75a4f343949ba0 The bug was not really resolved. Since we changed Target Milestone to 1.1, we can reopen the bug to track it. Edwin will kindly help to look into the bug and update the Whiteboard field. After investigated qemu's source code and ia32 PMU spec, I found qemu doesn't emulate PMU at all.
According to ia32 spec, "Availability of architectural performance monitoring capabilities is enumerated using the CPUID.0AH."
From qemu side, cpu_x86_cpuid in target-i386/cpuid.c did:
switch(index) {
case 0:
..............
case 0xA:
/* Architectural Performance Monitoring Leaf */
*eax = 0;
*ebx = 0;
*ecx = 0;
*edx = 0;
break;
So target OS would not detect any PMU capability.
According to ia32 spec, "Configuring an architectural performance monitoring event involves programming performance event select registers (IA32_PERFEVTSELx MSRs,at address 186H). The result of a performance monitoring event is reported in a performance monitoring counter (IA32_PMCx MSR, at address 0C1H).
From qemu side, helper_rdmsr in target-i386/op_helper.c doesn't handle any these MSR addr.
So our target linux guest complains "Broken PMU hardware detected, software events only", and oprofile based on PMU failed. With "oprofile.timer=1", a software timer is setup to collect EIP periodically so we can see the rough CPU cycles chart, but other HW PMU data like cache miss is still unavailable.
From qemu developer's point of view, it's reasonable as qemu is _emulator_, and each target instruction is translated into many different host instructions for emulation. In such circumstance, HW PMU data like cache miss is meaningless. Furthermore, emulating PMU requires huge efforts. But for virtualization, like KVM, it's different. The host can _export_ host's PMU to guest for profiling, but need handle the PMU switching. There was one patch in KVM for this, but seems not in upstream now. (http://comments.gmane.org/gmane.comp.emulators.kvm.devel/74107). We can ask end user to do the oprofile inside KVM(already in our poky) once this patch in upstream. So final solution is KVM + PMU virtualization + documentation :) I'd like to close this bug as won't fix. Thanks, Edwin |