Bug 4099

Summary: Crosstap script check fails
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: Stoicescu Cornel <corneliux.stoicescu>
Component: meta-yoctoAssignee: Tom Zanussi <tom.zanussi>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: ning.zhang, poky.bs.watcher, poky.watcher, richard.purdie
Version: 1.4   
Target Milestone: 1.4 M6   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Stoicescu Cornel 2013-03-25 13:40:42 UTC
STATUS: FAILED
BUILD: master:9ffee9add2d0380f4025e7a678a166f25a7d659e
ENVIRONMENT: Ubuntu 12.04 x86-64
STEPS TO REPRODUCE: 
1.Create the trace_open.stp script as follows:
 

probe syscall.open
{
        printf ("%s(%d) open (%s)\n", execname(), pid(), argstr)
}

2.Bitbake a core-image-sato-sdk image and start it under qemu.
3.Make sure that the ssh service is started on the Qemu machine.
4.From the host machine poky build_dir, run "crosstap root@192.168.7.2 trace_open.stp".
NOTE: -build the image without sstate for this to work

EXPECTED OUTCOME: The script should successfully connect to the qemu machine and there 
should be presented a list of services(pid, process name) which run on 
the qemu machine.

ACTUAL OUTCOME:
"
cornel@NODE1:~/SSD/3poky/build$ crosstap root@192.168.7.2 trace_open.stp
~/SSD/3poky/build ~/SSD/3poky/build
~/SSD/3poky/build
semantic error: not accessible at this address [man error::dwarf] (0xffffffff8118b540, dieoffset: 0x146856a): identifier '$flags' at /home/cornel/SSD/3poky/build/tmp/sysroots/x86_64-linux/usr/share/systemtap/tapset/linux/syscalls2.stp:129:10
        source:         flags = $flags
                                ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff8118b540, dieoffset: 0x146855a): identifier '$filename' at :132:54
        source:                 argstr = sprintf("%s, %s, %#o", user_string_quoted($filename),
                                                                                   ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff8113bee0, dieoffset: 0x1106fbb): identifier '$flags' at :129:10
        source:         flags = $flags
                                ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff8113bee0, dieoffset: 0x1106fab): identifier '$filename' at :132:54
        source:                 argstr = sprintf("%s, %s, %#o", user_string_quoted($filename),
                                                                                   ^

Pass 2: analysis failed.  [man error::pass2]
cornel@NODE1:~/SSD/3poky/build$
"
Comment 1 Tom Zanussi 2013-03-28 20:40:44 UTC
This appears to be due to the kernel using -mfentry when ftrace is enabled on x86/x86_64 machines (haven't verified using another arch).

The systemtap version (2.1) being used in Yocto has a fix for this, but it doesn't seem to be be working - tracing through the systemtap code, the pr1523_retry_addr() dwarf lookup fails because the debuginfo check for the -mfentry string isn't finding it despite the fact that HAVE_FENTRY is true and the kernel is compiled with -mfentry: 

if (! strstr(producer, "-mfentry"))
    return 0;
Comment 2 Tom Zanussi 2013-04-15 22:40:35 UTC
    This affects the 3.8 kernel when ftrace is turned on, and can be fixed
    by adding a compile option for the kernel -record-gcc-switches.  Explanation
    below...

    Commit a2546fae [ftrace: Add -mfentry to Makefile on function tracer]
    adds support for using -mfentry when possible, whenever the ftrace
    CONFIG_FUNCTION_TRACER option is enabled.
    
    This unfortunately causes bogus dwarf debuginfo to be generated:
    
      gcc Bug 54793 - the location of a formal_parameter is not started from a
                      function entry with -mfentry
    
      http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54793
    
    This can be fixed by applying the gcc patch above, which will be
    present in gcc-4.8, but in the meantime, the problem is currently
    being worked around in different ways by various tools.  One of those
    tools that we support in Yocto, Systemtap, requires the DW_AT_producer
    string contain -mfentry, which means that it's assuming
    record-gcc-switches is also used:
    
      SystemTap Bug 15123 - workaround for bad debuginfo for -mfentry
      http://sourceware.org/bugzilla/show_bug.cgi?id=15123
    
    The lack of -grecord-gcc-switches and therefore a missing -mfentry
    string in Yocto leads to the following Yocto bug:
    
      Bug 4099 - Crosstap script check fails
      https://bugzilla.yoctoproject.org/show_bug.cgi?id=4099
    
    Which is what this patch intends to fix.
    
    It only affects architectures that HAVE_FENTRY when
    CONFIG_FUNCTION_TRACER is turned on and in any case is just adding to
    the debuginfo, so there doesn't seem to be much risk to adding it.
Comment 3 Tom Zanussi 2013-04-15 22:43:42 UTC
Just to add that SystemTap itself currently works fine if the probes don't access param values of probed functions.  It's a pretty important capability, as the fact that basic system call tapsets won't work without it demonstrates..
Comment 4 Tom Zanussi 2013-04-15 22:48:35 UTC
Just to add one more thing - the current options available to address this are:

- turn off ftrace in the kernel for x86_64
- add the gcc patch mentioned above
- wait for gcc 4.8
- add -grecord-gcc-switches when -mfentry is used

Obviously, the first three options are either impossible at this point in the release, or unpalatable (turning off ftrace), so this seems to be the best option at this point..
Comment 5 Tom Zanussi 2013-04-16 16:35:49 UTC
Patch submitted to linux-yocto mailing list.
Comment 6 Tom Zanussi 2013-04-16 20:10:29 UTC
This has been pulled into linux-yocto-3.8:

http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto-3.8/log/?h=standard/base

commit d9a45e3325030f7bd6f37947a7a0b12da7f602c3
Comment 8 Ning Zhang 2013-04-24 09:12:11 UTC
Verified in dylan branch 789b2b7e0cca404ff8ba8ed5ba823d215e6ba5b4
Comment 9 Ning Zhang 2013-04-24 09:12:59 UTC
Last reply forgot change the status