Bug 13722

Summary: Debugging With the GNU Project Debugger enhancements
Product: [Documentation] Development Manual Reporter: anon2313
Component: developmentAssignee: Sundeep KOKKONDA <sundeep.kokkonda>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: akuster, john.kaldas.enpj, michael.opdenacker, pgowda.cve, randy.macleod, richard.purdie, sundeep.kokkonda
Version: 3.0.1   
Target Milestone: 4.1 M1   
Hardware: x86   
OS: Multiple   
Whiteboard: NIR 04/15 - NEWCOMER
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description anon2313 2019-12-22 22:28:28 UTC
I really appreciate that these steps are included in the documentation. I made some mistakes and other parts I didn't understand why it would be necessary and maybe could be clarified.

3.30.13. Debugging With the GNU Project Debugger (GDB) Remotely

I think the tip "install debug (-dbg) packages for the applications you are going to debug." belongs to section "3.30.14. Debugging with the GNU Project Debugger (GDB) on the Target".

1) Configure your build system to construct the companion debug filesystem:

Is this necessary if you build and use the SDK?  I think the SDK already includes debug files without setting these values.  IMAGE_GEN_DEBUGFS isn't in the reference guide so it's hard to know what it is supposed to do.

2) Configure the system to include gdbserver in the target filesystem:

Mention the following also includes the debugger
EXTRA_IMAGE_FEATURES_append = " tools-debug"

4) Set up the debugfs

Not necessary when using the SDK as it has sources in it? But when not using the SDK untaring the wrong file system like "core-image-minimal-initramfs-genericx86-64-dbg.tar.bz2" can happen.  I did this the first time through and it took awhile for me to understand the mistake I made. Maybe just a note to prevent new users from making the stupid mistake I made.

There is a typo in the documentation "build-dir/tmp-glibc/deploy/images/machine/image.rootfs.tar.bz2". "tmp-glibc" should just be "tmp".


5) Set up GDB

Modify the sentence to read "Sourcing the environment file puts the SDK in your PATH environment variable and sets $GDB to the SDK's debugger".  gdb by itself is the host tool.

7) Debug a program:

"set substitute-path /usr/src/debug debugfs/usr/src/debug".  I don't know what circumstances this works with but not with any application I tried debugging.  With a fresh Poky build all applications I tried debugging uses relative paths.  "substitue-path" doesn't work with files that start with relative paths (there doesn't seem to be a way for gdb to strip relative paths (https://stackoverflow.com/questions/37330688/how-can-i-set-a-gdb-source-mapping-for-a-relative-path).

For example debugging less:
(gdb) l
68	../less-551/main.c: No such file or directory.


But the "dir" command does work when adding the programs build directory as a base directory to use.  This seems to be primarily the case when something is built with autotools with a separate build directory.  The file path is relative from the build folder to the source folder.  

8) Deploying without a full image rebuild:

When using "devtool modify recipe" and "devtool build recipe" the "-src" package and folder is empty.  I guess this is due to using external sources.  Again for this to work with gdb I had to use the "dir" command so that the sources could be found.  Also maybe the SDK manual should point to these sections in "Deploy the Build Output" if you need to perform debugging.

The major problem I had was trying to find a front-end to gdb that worked.  I don't know what the problem is but the debugger keeps having issues with a failure.  When using VS Code or Eclipse with the debugger in Jethro worked fine but with Zeus I keep seeing the error and it kills the debug session:

warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code. 

I mention this because local.conf has "eclipse-debug" but I didn't understand from the documentation if it was still relevant as eclipse support was said to be removed.

I eventually found that gdbgui works without any problems with the debugger that is built with Yocto. I guess that gdb has a builtin interface called tui but it is disabled.
Comment 1 anon2313 2019-12-22 22:45:50 UTC
Also Maybe the tip to make a .gdbinit file should include that the commands can be in any file and use $GDB -ix file.init.  I used this instead because .gdbinit is run for every every use of gdb (Yocto related or not).
Comment 2 anon2313 2019-12-23 14:51:45 UTC
Maybe repeat the DEBUG_BUILD = "1" tip from the bottom of 7.30.14 in section 7.30.13.  I think it's kind of easy to miss.
Comment 3 Armin Kuster 2020-01-09 15:41:22 UTC
Thanks for providing the details.
Comment 4 Randy MacLeod 2020-06-19 10:49:24 UTC
I don't think this is done and M1 is over so bumping to M2.
Comment 5 John Kaldas 2021-02-16 15:46:25 UTC
I would like to work on this ticket.
Comment 6 Randy MacLeod 2021-02-16 15:59:31 UTC
Thanks for signing up John.

Are you on the docs list: https://lists.yoctoproject.org/g/docs
Have you cloned: http://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/
I haven't worked with the new sphinx docs but I hear that it's easier and likely even documented somewhere. Just ask if you need help and I'll find someone who is better informed on docs than I am! -- Randy
Comment 7 Randy MacLeod 2021-02-25 17:12:56 UTC
Any news or problems John?
Comment 8 John Kaldas 2021-03-09 16:30:21 UTC
I had a couple of questions which were more related to the workflow and docs generation, I have had answers to it through the mail list.

I will keep the ticket updated with the progress.
Comment 9 Randy MacLeod 2021-08-19 15:14:10 UTC
Any news John? Do you need help?
Comment 10 Randy MacLeod 2021-11-11 16:05:38 UTC
Add Naveen who might have time to work on this in a month or so. 

John, Any news?
Comment 11 Sundeep KOKKONDA 2022-03-15 07:57:38 UTC
Hello,


I analyzed the suggested doc changes, a few changes can be considered in doc update and for a few I need some more details. See my analysis/comments below for each doc change requested (Each number corresponds to the item number used by the issue reporter in the initial bug ticket).

1) IMAGE_GEN_DEBUGFS is in the documentation. The doc says 'set IMAGE_GEN_DEBUGFS to “1”, which will generate a companion debug image containing unstripped binaries and associated debug sources alongside the image.'

2) Doc will be updated as suggested.

4) The 'tmp' directory is configurable in conf/local.conf with TMPDIR switch. Anyway, the doc will be updated with default directory name 'tmp'.

5) Doc will be updated as suggested.

7) I followed the exact steps in document and I'm able to establish the gdbserver connection for debugging.

Regarding the 'substitute-path' usage refer the below page.
https://sourceware.org/gdb/onlinedocs/gdb/Source-Path.html#:~:text=set%20substitute%2Dpath%20is%20also,at%20the%20rewritten%20location%20instead.

Let me know what is the exact issue and what change is expected in doc?

8) Can you provide here more detailed steps of your issue. I presume you're facing an issue with debugging. Can you use below switches in local.conf and try to work with gdb? (These pkgs appends debugging related info into the generated image)

IMAGE_INSTALL:append = " gcc g++ binutils libgcc libgcc-dev libstdc++ libstdc++-dev libstdc++-staticdev nfs-export-root libc-mtrace gdb gdbserver openssh openssl kexec-tools"
EXTRA_IMAGE_FEATURES:append = " dbg-pkgs debug-tweaks dev-pkgs eclipse-debug package-management src-pkgs tools-debug tools-sdk"


Let me know your inputs for my analysis to finalize the doc changes.
Comment 12 Sundeep KOKKONDA 2022-03-28 09:41:12 UTC
Hi,

Can you update me on listed queries.
Comment 13 Sundeep KOKKONDA 2022-04-18 05:29:16 UTC
Hello,

The docs corrections which can be corrected/updated are considered and below is the patch.
https://lists.yoctoproject.org/g/docs/message/2778
Comment 14 Sundeep KOKKONDA 2022-05-01 17:32:21 UTC
https://lists.yoctoproject.org/g/docs/message/2786
Changes are in 'master' branch.
Comment 15 Sundeep KOKKONDA 2022-05-04 04:40:42 UTC
Commit ID - 7116c349b5074913df37aa3a7633226f42480f0d

bft1@bft1-OptiPlex-7060:~/skokkonda/docs/yocto-docs$ git branch -a --contains 7116c349b5074913df37aa3a7633226f42480f0d
* master
Comment 16 Sundeep KOKKONDA 2022-05-04 14:05:25 UTC
The patch not taken to master.
Comment 17 Sundeep KOKKONDA 2022-05-11 05:59:12 UTC
Patch taken to master & master-next branch. Below is the commit links.
https://git.yoctoproject.org/yocto-docs/commit/?id=48c96378fd5edd587025f3ca138bc5114e927f2f