Bug 11709 - Support EFI System partition with 4K FAT clusters
Summary: Support EFI System partition with 4K FAT clusters
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: Scripts and Tools (show other bugs)
Version: 2.4
Hardware: x86 Multiple
: Medium+ major
Target Milestone: 2.4 M2
Assignee: Ed Bartosh
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-06-23 08:27 UTC by jonathan.yong
Modified: 2017-07-24 07:31 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jonathan.yong 2017-06-23 08:27:31 UTC
Currently, the WIC and HDDIMG images produced by yocto fails to boot when written to a storage device that presents itself as having 4096 byte logical (and physical) sectors.

Changing the FAT cluster size from 512 to 4096 in meta/classes/image-live.bbclass allows a valid file system on such a device.

4K sectors/cluster breaks legacy BIOS boot, so an additional MACHINE_FEATURES_remove = "pcbios" is required in local.conf.

Make a new MACHINE_FEATURES entry for 4K cluster EFI system partitions?
Comment 1 Ed Bartosh 2017-06-26 11:11:13 UTC
On which hardware can I test this? Is it possible to reproduce in qemu?
Comment 2 jonathan.yong 2017-06-28 00:20:25 UTC
In our case, we're testing on an embedded 64GB UFS disk, I'm not sure how to get those on the market, high end Android phones? Perhaps with proper qemu device models? I'm not familiar with qemu internals.

The only part that alerted me was the dmesg claiming that the logical blocks were 4096 bytes.

The kernel refuses to mount it if formatted with 512 byte clusters, depending on EFI bootloader firmware implementations, some might work anyway.
Comment 3 Ed Bartosh 2017-06-28 14:47:22 UTC
If there is no way for me to reproduce this I'd need your help in testing my patches.

First attempt (remove -S 512 in a hope that mkdosf will decide correct sector size):

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 5aa68c9..0633589 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -312,8 +312,8 @@ class Partition():
         if self.fstype == 'msdos':
             size_str = "-F 16" # FAT 16
 
-        dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
-                                                       rootfs, rootfs_size)
+        dosfs_cmd = "mkdosfs %s %s -C %s %d" % (label_str, size_str,
+                                                rootfs, rootfs_size)
         exec_native_cmd(dosfs_cmd, native_sysroot)
 
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)


Let me know please if this works for you or not.
Comment 4 jonathan.yong 2017-06-29 01:21:59 UTC
Looks like it still uses 512 byte clusters, and then errors out with "Attempting to create a too large filesystem" with larger volumes that would have succeeded if created with 4K clusters manually.
Comment 5 Ed Bartosh 2017-06-29 08:17:34 UTC
OK, next attempt then:

diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 5aa68c9..64f622f 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -312,8 +312,8 @@ class Partition():
         if self.fstype == 'msdos':
             size_str = "-F 16" # FAT 16
 
-        dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
-                                                       rootfs, rootfs_size)
+        dosfs_cmd = "mkdosfs %s -S 512 -s 8 %s -C %s %d" % (label_str, size_str,
+                                                            rootfs, rootfs_size)
         exec_native_cmd(dosfs_cmd, native_sysroot)
 
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)

BTW, you mentioned that you've changed -S 512 to -S 4096 in image-live.bbclass. However, -S specifies size of logical sector. That would not always means that cluster size will be 4096. So, I left logical sector 512 and specified amount of sectors per cluster. That should ensure that cluster size will be 4096. Let me know if this works and if this is what you wanted.
Comment 6 jonathan.yong 2017-06-29 08:48:37 UTC
Nice, -S 512 -s 8 is working too, I was confused with the description of -s and -S in the man page.
Comment 7 Ed Bartosh 2017-06-29 08:55:18 UTC
OK, then I implement --mkfs-opts option for .wks and you can specify any options you want. Let me know if it would work for you.
Comment 8 Stephen K Jolley 2017-06-29 14:38:44 UTC
Please answer Ed's question.
Comment 9 jonathan.yong 2017-06-30 00:21:38 UTC
Hi, how do I get your new --mkfs-opts changes to test them? I don't see a patch. Btw, hddimg also needs to be handled.
Comment 10 jonathan.yong 2017-07-05 02:09:30 UTC
Hold on, no "-S 512 -s 8" does not work, found out it was still booting the image as an emulated traditional USB drive image with 512-byte sectors due to a typo in my test.

Looks like it actually needs -S 4096.
Comment 11 Ross Burton 2017-07-05 10:47:05 UTC
Changing to a proper state and reassigning to Ed, hopefully that's right.
Comment 12 Ed Bartosh 2017-07-05 11:19:37 UTC
> Hold on, no "-S 512 -s 8" does not work

Can you tell me what does work? You mentioned you've changed 512 to 4096 in image-live.bbclass, but you didn't show the change.

Is it something like this?

--- a/meta/classes/image-live.bbclass
+++ b/meta/classes/image-live.bbclass
@@ -217,10 +217,10 @@ build_fat_img() {
        fi
 
        if [ -z "${HDDIMG_ID}" ]; then
-               mkdosfs ${FATSIZE} -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${FATIMG} \
+               mkdosfs ${FATSIZE} -n ${BOOTIMG_VOLUME_ID} -S 4096 -C ${FATIMG} \
                        ${BLOCKS}
        else
-               mkdosfs ${FATSIZE} -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${FATIMG} \
+               mkdosfs ${FATSIZE} -n ${BOOTIMG_VOLUME_ID} -S 4096 -C ${FATIMG} \
                ${BLOCKS} -i ${HDDIMG_ID}
        fi


Here is a similar patch for wic:
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -312,7 +312,7 @@ class Partition():
         if self.fstype == 'msdos':
             size_str = "-F 16" # FAT 16
 
-        dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str,
+        dosfs_cmd = "mkdosfs %s -S 4096 %s -C %s %d" % (label_str, size_str,
                                                        rootfs, rootfs_size)
         exec_native_cmd(dosfs_cmd, native_sysroot)


Let me know if that works for you and I'll implement --mkfs-opts for wic and something similar for hddimg.
Comment 13 jonathan.yong 2017-07-05 23:25:11 UTC
Yes, that is exactly what I changed to get the kernel to mount the filesystem properly.
Comment 14 Ed Bartosh 2017-07-06 08:56:03 UTC
Sent first patch for review: http://lists.openembedded.org/pipermail/openembedded-core/2017-July/139172.html
Comment 15 Ed Bartosh 2017-07-06 11:46:14 UTC
sent second patchset for review: http://lists.openembedded.org/pipermail/openembedded-core/2017-July/139178.html
Comment 16 Ed Bartosh 2017-07-24 07:31:47 UTC
both patchsets have been merged to poky master.