Since commit 58589bc5382b5c77825bc963a1c84a4e8f54498f Author: Ricardo Ribalda Delgado <ricardo@ribalda.com> Date: Thu Jul 18 15:13:56 2019 +0200 wic: Fix (again) partition files UIDs on multi rootfs images rootfs/etc/fstab is modified in place during do_image_wic. If other image types are enabled (such as ext4), the respective do_image_* tasks create image files at the same time as this modification is happening, which leads to unpredictable outcome (fstab may or may not be modified by wic in the output). I do not have a quick solution (inserting task dependencies does not sound appealing; making a rootfs copy is slow and is causing pseudo issues that the commit above is aiming to solve), but wanted to have a record for the problem neverthless. One possible option is to mount the wic-made rootfs file after it has been created, then modify fstab there and only there.
Ricardo, FYI.
I believe that after the latest changes https://git.openembedded.org/openembedded-core/commit/?id=fd739c15cdba221d9d497d3402b7d929c0be2ca4 (among others) Pseudo should be playing nice with wic, but as you mentioned, that involves copying the partition and that is slow. Mounting the wic rootfs after its creations seems like the best cadidate, although I do not know if all the filesystem will allow you to do that.
I was just hitting this as well here. Spuriously b/c of the parallelism of do_image_ext4 + do_image_tar + do_image_wic running at the same time. I think the mounting is the better way, but if that is not feasible, why not make do_image_wic run *last* e.g. after do_image_ext4 and do_image_tar ?
e.g. add to image_types_wic.bbclass: # make sure that ext4 and tar are completely written out before we modify files to create the wic format (e.g. /etc/fstab !) do_image_wic[deptask] += "${@bb.utils.contains('IMAGE_FSTYPES', 'ext4', '%s:do_image_ext4' % d.getVar('PN'), '', d)}" do_image_wic[deptask] += "${@bb.utils.contains('IMAGE_FSTYPES', 'tar', '%s:do_image_tar' % d.getVar('PN'), '', d)}" Not sure what other image-fstype would need it. Still mounting before modification is indeed better.
Hardcoding task dependencies like this is too brittle, as there is no mechanism to ensure all possible image types are accounted for, and there's quite a bit more than ext4 and tar (see classes/image_types.bbclass).
Agree, hardcoding is not the way to go.
I can see 3 possible approaches here, all imperfect: 1) We can easily modify fstab inside the filesystem image after it is created using mcopy (for vfat) or debugfs (for ext2/3/4). But I can't see an equivalent tool for btrfs. 2) We can copy the rootfs into a temporary directory so that fstab can be modified without interfering with other do_image_* tasks. This must be done as a full copy though along with copying the pseudo db, we can't use copyhardlinktree for the same reason explained in the comments added in https://git.openembedded.org/openembedded-core/commit/?id=e8c21c6ebaebde88151697381bdb2452f1171090. So this option would use more space and slow down the build. 3) The final alternative is to ensure that no other do_image_* tasks can run in parallel with do_image_wic. Rather than hardcoding dependencies we could use lockfiles for this though it would have the unfortunate side effect of making all do_image_* functions run in series even if no fstab modification occurs. We don't have support in bitbake for a more complex read/write lock arrangement. I think my preference would be to check the fstype and use approach (1) for vfat & ext2/3/4, falling back to approach (2) for btrfs. Any thoughts Ricardo or Alex?
I think approach 3, if documented / sufficiently clear would also help expose a more generic way to fix other race conditions that can occur (for example when using dm-verity and also needing to deal with that fitImage+initramfs being in an image).
(In reply to comment #8) > I think approach 3, if documented / sufficiently clear would also help > expose a more generic way to fix other race conditions that can occur (for > example when using dm-verity and also needing to deal with that > fitImage+initramfs being in an image). I am with Tony, approach 1) will require a fix for every single rfs type and 2) is a tremendous penalty to the overall performance Also, a disclaimer here, I just changed jobs so while I get up to the speed with the new role I would not have much time to work on this sorry :(
(In reply to comment #9) > (In reply to comment #8) > > I think approach 3, if documented / sufficiently clear would also help > > expose a more generic way to fix other race conditions that can occur (for > > example when using dm-verity and also needing to deal with that > > fitImage+initramfs being in an image). > > I am with Tony, approach 1) will require a fix for every single rfs type and > 2) is a tremendous penalty to the overall performance > > Also, a disclaimer here, I just changed jobs so while I get up to the speed > with the new role I would not have much time to work on this sorry :( The issue with approach (3) is that it can also add a huge penalty to rebuilds when very little has changed but images need to be recreated. On a previous client project I saw rebuild time for minimal changes (i.e. just the date stamp in os-release changed) go from 5min to 7min (+40% build time) when do_image_tar and do_image_wic had to run in series. My plan right now is to implement (2) as the default solution and then optimise this where possible with (1), i.e. for vfat and ext2/3/4.
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=36575a949399953f98fc1cfda4a3dcb5a295b02c