Bug 7116

Summary: GCC with ARM hard float and -mapcs-frame generates bad tailcall code
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Donn Seeley <donn.seeley>
Component: devtools / tool chainAssignee: Khem Raj <raj.khem>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: Medium CC: meta.mr.watcher, meta.watcher, sgw
Version: 1.7   
Target Milestone: 1.7.1   
Hardware: Other   
OS: arm   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
content.i none

Description Donn Seeley 2015-01-06 20:00:25 UTC
Created attachment 2313 [details]
content.i

This problem has already been submitted to the GCC bugzilla:

  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64379

Mark Hatle suggested that I should submit it here too, since it shows up with dizzy.

The overall summary: If you build for ARM with VFP/NEON, -O2 and -mapcs-frame, then a function that saves a VFP register and performs an optimized indirect tailcall may get bad code in its function epilogue that can cause segfaults.  We discovered this problem when xfsdump (and gdb) segfaulted in WR Linux 7 in a similar configuration.  I have a test program simplified from xfsdump that shows the same issue in dizzy / 1.7.

The bad code looks like this:

        sub     ip, fp, #44
        [...]
        fldmfdd ip!, {d8}
        sub     sp, fp, #36
        ldmfd   sp, {r4, r5, r6, r7, r8, r9, fp, sp, lr}
        bx      ip      @ indirect register sibling call

GCC chooses IP to hold the function pointer for an optimized indirect tailcall, but the epilogue code for -mapcs-frame explicitly uses IP to handle VFP/NEON register restores, so that when we make the indirect function call, IP has a stack address in it rather than the function pointer, and we segfault.  (The function pointer load has been optimized out.)

To re-create the problem on dizzy, I added the following to conf/local.conf:

  MACHINE = "qemuarm"
  DEFAULTTUNE = "armv7at-neon"
  DEBUG_BUILD = "1"
  DEBUG_OPTIMIZATION = "-O2 -pipe -g -feliminate-unused-debug-types -fno-omit-frame-pointer -fvisibility=default -mapcs-frame"

The modified DEBUG_OPTIMIZATION line just forces -O2, so that the tailcall optimization pass gets run.  You can generate the bad code with just '-O1 -foptimize-sibling-calls' rather than -O2.

Put the following in conf/machine/qemuarm.conf:

  require conf/machine/include/qemu.inc
  require conf/machine/include/tune-cortexa9.inc
  KERNEL_IMAGETYPE = "zImage"
  SERIAL_CONSOLE = "115200 ttyAMA0"

This change just switches to tune-cortexa9.inc so that armv7at-neon is a valid tuning.

I then ran a build.  I added the following to my environment (cut and pasted from 'bitbake -e bash'):

  export CC="arm-poky-linux-gnueabi-gcc  -march=armv7-a -marm -mthumb-interwork -mfloat-abi=softfp -mfpu=neon --sysroot=/home/donn/c/yocto/poky/build/tmp/sysroots/qemuarm"
  export CFLAGS=" -O2 -pipe -g -feliminate-unused-debug-types -fno-omit-frame-pointer -fvisibility=default -mapcs-frame"
  export PATH="/home/donn/c/yocto/poky/scripts:/home/donn/c/yocto/poky/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi:/home/donn/c/yocto/poky/build/tmp/sysroots/qemuarm/usr/bin/crossscripts:/home/donn/c/yocto/poky/build/tmp/sysroots/x86_64-linux/usr/sbin:/home/donn/c/yocto/poky/build/tmp/sysroots/x86_64-linux/usr/bin:/home/donn/c/yocto/poky/build/tmp/sysroots/x86_64-linux/sbin:/home/donn/c/yocto/poky/build/tmp/sysroots/x86_64-linux/bin:/home/donn/c/yocto/poky/scripts:/home/donn/c/yocto/poky/bitbake/bin:/home/donn/c/git/bin:/home/donn/bin:/usr/contrib/mh/bin:/sbin:/usr/sbin:/usr/contrib/sbin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"

I built the attached content.i file (reduced from dump/content.c in xfsdump) using:

  $CC $CFLAGS -S content.i

Examining the resulting content.s file shows that the bad code is present.
Comment 1 Donn Seeley 2015-01-13 21:14:33 UTC
The feedback that I got from GCC developers is that -mapcs-frame is obsolete.  It doesn't appear that Yocto uses -mapcs-frame by default, so we'll stop using it with WR Linux and we'll see what happens.
Comment 2 Saul Wold 2015-01-22 16:09:07 UTC
See prior comment