2077 2012-03-13 05:50:50 +0000 memory buffer mess up of i586-poky-linux-gdb in meta-toolchain 2012-03-23 18:59:32 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 1.2 x86 x86 RESOLVED FIXED High critical 1.2 M4 1 lianhao.lu nitin.a.kamble jessica.zhang meta.mr.watcher meta.watcher sgw --- --- --- oldest_to_newest 18977 0 lianhao.lu 2012-03-13 05:50:50 +0000 Branch: master commit id: bd2d8904181aa67258b6be8a7bbeaf8d12a8dd5a on Ubuntu 10.04 x86-64, when launching the i586-poky-linux-gdb from meta-toolchain, the following error got reported: lulianhao@llu-piketon:~/poky-build/poky$ i586-poky-linux-gdb *** glibc detected *** i586-poky-linux-gdb: double free or corruption (out): 0x00000000025cf030 *** ======= Backtrace: ========= /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6(+0x735c6)[0x7f482660b5c6] /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6(cfree+0x6c)[0x7f482660f49c] /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0(PyDict_GetItemString+0x4c)[0x7f4826049a0c] /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0(PyImport_AddModule+0x29)[0x7f48260bdbb9] /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0(Py_InitModule4_64+0xc6)[0x7f48260c4b26] /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so(initreadline+0x26)[0x7f4826395546] i586-poky-linux-gdb[0x6f7f55] i586-poky-linux-gdb[0x6f6258] i586-poky-linux-gdb[0x6f64d2] i586-poky-linux-gdb[0x6f6af2] i586-poky-linux-gdb[0x6f70fa] i586-poky-linux-gdb[0x6daf0f] i586-poky-linux-gdb[0x66c843] i586-poky-linux-gdb[0x6dcd57] i586-poky-linux-gdb[0x6dfa7b] i586-poky-linux-gdb[0x6e461f] i586-poky-linux-gdb[0x6e4772] i586-poky-linux-gdb[0x6f4f02] i586-poky-linux-gdb[0x6f5346] i586-poky-linux-gdb[0x6f6258] i586-poky-linux-gdb[0x6f64d2] i586-poky-linux-gdb[0x6f6af2] i586-poky-linux-gdb[0x6f70fa] i586-poky-linux-gdb[0x6daf0f] i586-poky-linux-gdb[0x66c843] i586-poky-linux-gdb[0x6dcd57] i586-poky-linux-gdb[0x6dfa7b] i586-poky-linux-gdb[0x6e461f] i586-poky-linux-gdb[0x6e4772] i586-poky-linux-gdb[0x6f4f02] i586-poky-linux-gdb[0x6f5346] i586-poky-linux-gdb[0x6f6258] i586-poky-linux-gdb[0x6f64d2] i586-poky-linux-gdb[0x6f6af2] i586-poky-linux-gdb[0x6f70fa] i586-poky-linux-gdb[0x6daf0f] i586-poky-linux-gdb[0x66c843] i586-poky-linux-gdb[0x6dcd57] i586-poky-linux-gdb[0x6dfa7b] i586-poky-linux-gdb[0x6e3ab0] ======= Memory map: ======== 00400000-00960000 r-xp 00000000 08:07 5657058 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/i586-poky-linux/i586-poky-linux-gdb 00b60000-00bad000 rw-p 00560000 08:07 5657058 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/i586-poky-linux/i586-poky-linux-gdb 00bad000-00bd7000 rw-p 00000000 00:00 0 0245c000-0262f000 rw-p 00000000 00:00 0 [heap] 7f4820000000-7f4820021000 rw-p 00000000 00:00 0 7f4820021000-7f4824000000 ---p 00000000 00:00 0 7f4825dac000-7f4825dc2000 r-xp 00000000 08:07 4980815 /lib/libgcc_s.so.1 7f4825dc2000-7f4825fc1000 ---p 00016000 08:07 4980815 /lib/libgcc_s.so.1 7f4825fc1000-7f4825fc2000 r--p 00015000 08:07 4980815 /lib/libgcc_s.so.1 7f4825fc2000-7f4825fc3000 rw-p 00016000 08:07 4980815 /lib/libgcc_s.so.1 7f4825fc3000-7f4826145000 r-xp 00000000 08:07 5657133 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 7f4826145000-7f4826345000 ---p 00182000 08:07 5657133 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 7f4826345000-7f4826384000 rw-p 00182000 08:07 5657133 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 7f4826384000-7f4826392000 rw-p 00000000 00:00 0 7f4826392000-7f4826396000 r-xp 00000000 08:07 5775275 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so 7f4826396000-7f4826596000 ---p 00004000 08:07 5775275 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so 7f4826596000-7f4826598000 rw-p 00004000 08:07 5775275 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so 7f4826598000-7f4826715000 r-xp 00000000 08:07 5656220 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc-2.13.so 7f4826715000-7f4826915000 ---p 0017d000 08:07 5656220 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc-2.13.so 7f4826915000-7f4826919000 r--p 0017d000 08:07 5656220 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc-2.13.so 7f4826919000-7f482691a000 rw-p 00181000 08:07 5656220 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc-2.13.so 7f482691a000-7f482691f000 rw-p 00000000 00:00 0 7f482691f000-7f4826946000 r-xp 00000000 08:07 5657031 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libexpat.so.1.5.2 7f4826946000-7f4826b45000 ---p 00027000 08:07 5657031 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libexpat.so.1.5.2 7f4826b45000-7f4826b48000 rw-p 00026000 08:07 5657031 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libexpat.so.1.5.2 7f4826b48000-7f4826b4a000 r-xp 00000000 08:07 5656228 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libutil-2.13.so 7f4826b4a000-7f4826d49000 ---p 00002000 08:07 5656228 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libutil-2.13.so 7f4826d49000-7f4826d4a000 r--p 00001000 08:07 5656228 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libutil-2.13.so 7f4826d4a000-7f4826d4b000 rw-p 00002000 08:07 5656228 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libutil-2.13.so 7f4826d4b000-7f4826d63000 r-xp 00000000 08:07 5656240 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libpthread-2.13.so 7f4826d63000-7f4826f62000 ---p 00018000 08:07 5656240 /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libpthread-2.13.soAborted 19244 1 nitin.a.kamble 2012-03-20 02:01:54 +0000 Got more symbols of back trace. When I debug the gdb binray from the build area I get this: [nitin@nbuild2 work]$ x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb *** glibc detected *** x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb: double free or corruption (out): 0x00007f11bfa05030 *** Aborted [nitin@nbuild2 work]$ gdb x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb GNU gdb (GDB) Fedora (7.3.50.20110722-10.fc16) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /build_disk/poky_build/build0/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb...done. (gdb) r Starting program: /build_disk/poky_build/build0/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libreadline.so.6 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libdl.so.2 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libncurses.so.5 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libtinfo.so.5 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libz.so.1 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libm.so.6 Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libpthread.so.0 warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libutil.so.1 warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libexpat.so.1 warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Detaching after fork from child process 14589. Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Missing separate debuginfo for /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. *** glibc detected *** /build_disk/poky_build/build0/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-x86-64-7.4-r5.3/image/opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gdb: double free or corruption (out): 0x00007ffff7e7b030 *** warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Program received signal SIGABRT, Aborted. 0x00007ffff671d8c5 in raise () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 Missing separate debuginfos, use: debuginfo-install libgcc-4.6.2-1.fc16.x86_64 (gdb) bt #0 0x00007ffff671d8c5 in raise () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 #1 0x00007ffff67209eb in abort () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 #2 0x00007ffff6752433 in ?? () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 #3 0x00007ffff675c5c6 in ?? () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 #4 0x00007ffff676049c in free () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/lib/libc.so.6 #5 0x00007ffff619aa0c in PyDict_GetItemString () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 #6 0x00007ffff620ebb9 in PyImport_AddModule () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 #7 0x00007ffff6215b26 in Py_InitModule4_64 () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/libpython2.7.so.1.0 #8 0x00007ffff64e6546 in initreadline () from /opt/poky/1.1+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/python2.7/lib-dynload/readline.so #9 0x0000000000709c05 in _PyImport_LoadDynamicModule () #10 0x0000000000707f08 in import_submodule () #11 0x0000000000708182 in load_next () #12 0x00000000007087a2 in import_module_level.isra.9 () #13 0x0000000000708daa in PyImport_ImportModuleLevel () #14 0x00000000006ecbbf in builtin___import__ () #15 0x000000000067e4f3 in PyObject_Call () #16 0x00000000006eea07 in PyEval_CallObjectWithKeywords () #17 0x00000000006f172b in PyEval_EvalFrameEx () #18 0x00000000006f62cf in PyEval_EvalCodeEx () #19 0x00000000006f6422 in PyEval_EvalCode () #20 0x0000000000706bb2 in PyImport_ExecCodeModuleEx () #21 0x0000000000706ff6 in load_source_module () #22 0x0000000000707f08 in import_submodule () #23 0x0000000000708182 in load_next () #24 0x00000000007087a2 in import_module_level.isra.9 () #25 0x0000000000708daa in PyImport_ImportModuleLevel () #26 0x00000000006ecbbf in builtin___import__ () #27 0x000000000067e4f3 in PyObject_Call () #28 0x00000000006eea07 in PyEval_CallObjectWithKeywords () #29 0x00000000006f172b in PyEval_EvalFrameEx () #30 0x00000000006f62cf in PyEval_EvalCodeEx () #31 0x00000000006f6422 in PyEval_EvalCode () #32 0x0000000000706bb2 in PyImport_ExecCodeModuleEx () #33 0x0000000000706ff6 in load_source_module () #34 0x0000000000707f08 in import_submodule () #35 0x0000000000708182 in load_next () #36 0x00000000007087a2 in import_module_level.isra.9 () #37 0x0000000000708daa in PyImport_ImportModuleLevel () #38 0x00000000006ecbbf in builtin___import__ () #39 0x000000000067e4f3 in PyObject_Call () #40 0x00000000006eea07 in PyEval_CallObjectWithKeywords () #41 0x00000000006f172b in PyEval_EvalFrameEx () #42 0x00000000006f5760 in PyEval_EvalFrameEx () #43 0x00000000006f5760 in PyEval_EvalFrameEx () #44 0x00000000006f62cf in PyEval_EvalCodeEx () #45 0x00000000006f6422 in PyEval_EvalCode () #46 0x0000000000706bb2 in PyImport_ExecCodeModuleEx () #47 0x0000000000706ff6 in load_source_module () #48 0x0000000000707f08 in import_submodule () #49 0x0000000000708182 in load_next () #50 0x00000000007087a2 in import_module_level.isra.9 () #51 0x0000000000708daa in PyImport_ImportModuleLevel () ---Type <return> to continue, or q <return> to quit--- 19261 2 nitin.a.kamble 2012-03-20 19:08:11 +0000 BTW I notice this issue also present in 1.2_M3 19263 3 nitin.a.kamble 2012-03-20 22:45:32 +0000 Currently I am doing git bisect to pinpoint the issue: Looking at current status of bisect, this commit is likely a culprit: commit 1cb4614f1d5e482b88ea372d1841a6c313a49941 Author: Eric BENARD <eric@eukrea.com> Date: Sun Feb 26 08:49:42 2012 +0100 gdb-cross-canadian: build gdb with python support * python support is needed for providing finer scripting control in gdb, and for remote controling gdb for example from qtcreator * gdb/configure tries to autodetect python using python executable with --includes --ldflags --exec-prefix to get the right flags As it's difficult to achieve in OE's context, we generate a script which will return the right values and make gdb's build system happy. This idea was taken from the following article : http://www.mentby.com/doug-evans-2/python-enabled-gdb-on-windows-and-relocation.html * tested using angstrom & armv7 target & qtcreator 201005 & 2.4.0 * MJ: updated paths to python as disscussed here http://lists.linuxtogo.org/pipermail/openembedded-core/2012-February/018222.html now it's easier with cross-canadian staging dir and SDKPATH fixed :) (From OE-Core rev: 73a90b88541c373aa5b83a939dd3b5f1fe05b9b8) Signed-off-by: Eric Bénard <eric@eukrea.com> Cc: fcooper@ti.com Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 19287 4 nitin.a.kamble 2012-03-21 16:38:01 +0000 sent a commit to oe-core mailing list for reverting the culprit commit http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=nitin/bugfix&id=bfe0a5400cad2436b7be7d35a173c62349e47c0a 19439 5 nitin.a.kamble 2012-03-23 18:59:32 +0000 khem pushed a fix for this: commit 45bc21c2752bdfe46a2ae262fec5acd0da185853 Author: Khem Raj <raj.khem@gmail.com> Date: Fri Mar 23 05:50:41 2012 -0700 gdb-cross-canadian: Fix linking issues when python support is enabled gdb-cross-canadian was linking with static version of libpython and looking for build host python when run. This could be mismatch between python-nativesdk and build host provided python. So we make it consistent to always use python-nativesdk Instead of adding bunch of -L we add -rpath and -rpath-link so that gdb finds all needed libraries in SDK and does not reach out into host This should fix yocto PR# 2077 as well. (From OE-Core rev: d56177c59b5e237a501281d806724885456f0193) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>