Bug 1783

Summary: dosfstools mkdosfs -d option crashes with subdirectories
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Darren Hart <dvhart>
Component: devtools / tool chainAssignee: Nitin Kamble <nitin.a.kamble>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: meta.mr.watcher, meta.watcher, otavio, richard.purdie
Version: 1.0   
Target Milestone: ---   
Hardware: x86   
OS: x86   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Darren Hart 2011-11-22 14:47:18 UTC 
When building images with bootimg.bbclass, we use the -d argument provided by mkdosfs-dir.patch for dosfstools. This allows us to populate the fs as we create it. We currently place all the files in the top level directory. If we add subdirectories, the mkdosfs command in bootimg.bbclass crashes with output at the end of this comment.

For proper live image creation for EFI booting we should be using an EFI/boot directory structure. So we need a way to create the image with directories - it may be that an alternate means is now available as many distributions now create EFI live images.


*** glibc detected *** mkdosfs: malloc(): memory corruption: 0x00000000006441f0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x78a96)[0x7f4a7dc3da96]
/lib/x86_64-linux-gnu/libc.so.6(+0x7a5f8)[0x7f4a7dc3f5f8]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x6d)[0x7f4a7dc4177d]
mkdosfs[0x401be7]
mkdosfs[0x402258]
mkdosfs[0x402499]
mkdosfs[0x4044f0]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f4a7dbe630d]
mkdosfs[0x4012c9]
======= Memory map: ========
00400000-00407000 r-xp 00000000 09:00 3304324                            /build/poky/n450/tmp/sysroots/x86_64-linux/usr/sbin/mkdosfs
00607000-00608000 r--p 00007000 09:00 3304324                            /build/poky/n450/tmp/sysroots/x86_64-linux/usr/sbin/mkdosfs
00608000-00609000 rw-p 00008000 09:00 3304324                            /build/poky/n450/tmp/sysroots/x86_64-linux/usr/sbin/mkdosfs
00609000-0060d000 rw-p 00000000 00:00 0
00638000-00659000 rw-p 00000000 00:00 0                                  [heap]
7f4a78000000-7f4a78021000 rw-p 00000000 00:00 0
7f4a78021000-7f4a7c000000 ---p 00000000 00:00 0
7f4a7d58e000-7f4a7d5a3000 r-xp 00000000 08:01 786887                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a7d5a3000-7f4a7d7a2000 ---p 00015000 08:01 786887                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a7d7a2000-7f4a7d7a3000 r--p 00014000 08:01 786887                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a7d7a3000-7f4a7d7a4000 rw-p 00015000 08:01 786887                     /lib/x86_64-linux-gnu/libgcc_s.so.1
7f4a7d7a4000-7f4a7d7bc000 r-xp 00000000 08:01 786910                     /lib/x86_6
4-linux-gnu/libpthread-2.13.so
7f4a7d7bc000-7f4a7d9bb000 ---p 00018000 08:01 786910                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7f4a7d9bb000-7f4a7d9bc000 r--p 00017000 08:01 786910                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7f4a7d9bc000-7f4a7d9bd000 rw-p 00018000 08:01 786910                     /lib/x86_64-linux-gnu/libpthread-2.13.so
7f4a7d9bd000-7f4a7d9c1000 rw-p 00000000 00:00 0
7f4a7d9c1000-7f4a7d9c3000 r-xp 00000000 08:01 786898                     /lib/x86_64-linux-gnu/libdl-2.13.so
7f4a7d9c3000-7f4a7dbc3000 ---p 00002000 08:01 786898                     /lib/x86_64-linux-gnu/libdl-2.13.so
7f4a7dbc3000-7f4a7dbc4000 r--p 00002000 08:01 786898                     /lib/x86_64-linux-gnu/libdl-2.13.so
7f4a7dbc4000-7f4a7dbc5000 rw-p 00003000 08:01 786898                     /lib/x86_64-linux-gnu/libdl-2.13.so
7f4a7dbc5000-7f4a7dd5a000 r-xp 00000000 08:01 786896                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a7dd5a000-7f4a7df59000 ---p 00195000 08:01 786896                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a7df59000-7f4a7df5d000 r--p 00194000 08:01 786896                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a7df5d000-7f4a7df5e000 rw-p 00198000 08:01 786896                     /lib/x86_64-linux-gnu/libc-2.13.so
7f4a7df5e000-7f4a7df64000 rw-p 00000000 00:00 0
7f4a7df64000-7f4a7df95000 r-xp 00000000 09:00 3428306                    /build/poky/n450/tmp/sysroots/x86_64-linux/usr/lib/pseudo/lib64/libpseudo.so
7f4a7df95000-7f4a7e194000 ---p 00031000 09:00 3428306                    /build/poky/n450/tmp/sysroots/x86_64-linux/usr/lib/pseudo/lib64/libpseudo.so
7f4a7e194000-7f4a7e195000 r--p 00030000 09:00 3428306                    /build/poky/n450/tmp/sysroots/x86_64-linux/usr/lib/pseudo/lib64/libpseudo.so
7f4a7e195000-7f4a7e197000 rw-p 00031000 09:00 3428306                    /build/poky/n450/tmp/sysroots/x86_64-linux/usr/lib/pseudo/lib64/libpseudo.so
7f4a7e197000-7f4a7e19f000 rw-p 00000000 00:00 0
7f4a7e19f000-7f4a7e1c0000 r-xp 00000000 08:01 786908                     /lib/x86_64-linux-gnu/ld-2.13.so
7f4a7e3aa000-7f4a7e3ae000 rw-p 00000000 00:00 0
7f4a7e3bc000-7f4a7e3bf000 rw-p 00000000 00:00 0
7f4a7e3bf000-7f4a7e3c0000 r--p 00020000 08:01 786908                     /lib/x86_64-linux-gnu/ld-2.13.so
7f4a7e3c0000-7f4a7e3c2000 rw-p 00021000 08:01 786908                     /lib/x86_64-linux-gnu/ld-2.13.so
7ffff43d3000-7ffff43f5000 rw-p 00000000 00:00 0                          [stack]
7ffff43ff000-7ffff4400000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Comment 1 Otavio Salvador 2011-12-06 06:13:59 UTC 
This is a blocker for bootimg and also to remove loopback use of sdcard_image class on meta-texasinstruments.
Comment 2 Darren Hart 2011-12-13 09:44:03 UTC 
This is still an issue after the recent update to dosfstools:

995a09e dosfstools: Add patch to disable fat32 autoselection and behave as 2.10
0c45ecb dosfstools: update native to 2.11

(Nitin requested a retest)
Comment 3 Otavio Salvador 2011-12-13 09:58:23 UTC 
Yes it does fail.
Comment 4 Nitin Kamble 2011-12-13 10:01:19 UTC 
dave, 
I am able to reproduce the issue:
mkdosfs 2.11 (12 Mar 2005)
ADD fri2-live-fs/ldlinux.sys
ADD fri2-live-fs/initrd
ADD fri2-live-fs/syslinux.cfg
ADD fri2-live-fs/EFI/BOOT/initrd
*** glibc detected *** image/builddisk/build/build0/tmp/sysroots/x86_64-linux/usr/sbin/mkdosfs: malloc(): memory corruption: 0x000000000195aa40 ***


BTW do you know when it was working, and which patch broke it?

Thanks,
Nitin
Comment 5 Darren Hart 2011-12-13 10:42:05 UTC 
(In reply to comment #4)
 
> BTW do you know when it was working, and which patch broke it?

It was never working to my knowledge. This is why bootimg crams everything into the root directory. EFI requires live images to use the EFI/BOOT/bootia32.efi directory structure. We can't do that because dosfstools breaks when using a non-flat directory structure.
Comment 6 Richard Purdie 2011-12-13 11:52:40 UTC 
For reference that code is also broken with fat32 (-F 32).
Comment 7 Nitin Kamble 2011-12-13 15:28:12 UTC 
I have a patch to fix this issue, will send the commit to mailing list soon.
Nitin
Comment 8 Nitin Kamble 2011-12-15 09:05:35 UTC 
This commit went into master to fix the issue.


commit 8413a9b4e80b65e0ad0ddac659cc6abbd305ba35
Author: Nitin A Kamble <nitin.a.kamble@intel.com>
Date:   Tue Dec 13 15:26:22 2011 -0800

    dosfstools: fix populated image creation with dirs
    
    This fixes bug: [YOCTO #1783]
    
    Fix populated image creation. Earlier subdirectories support
    was broken, and files can only be placed in the root directory.
    Now directory hirarchy is supported in the image. Also support
    for long names is extended to directory names.
    
    There are some outstanding issues as documented in the patch
    header, these issues can be worked around by running
    dosfsck tool after populated image creation. The dosfsck tool
    is also part of this package.
    
    (From OE-Core rev: 2de8f008b304017de7ca116aa79ef778ab40362d)
    
    Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>