Bug 13898

Hi Diego


I have sent a series to fix the uid/gid problems on wic. Unfortunately it dit not make it on time to dunfell.

The last patch of the series is 01619c9acbf8ac5c66817a5cdfe9a8819891989b which is on master ( http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=01619c9acbf8ac5c66817a5cdfe9a8819891989b )

If you can use master please give it a try. 


I do not now if this will be backported to LTS. I am not sure about that process.

Thanks!

Hi Ricardo,

I added Steve Sakoman to this bug CC list, he maintains denfell.

Patch backport guidelines are here:
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Backporting_fixes_to_stable_releases

So a subject line such as:
[oe-core][dunfell] package: brief description goes here.
would make sense

(In reply to comment #1)
Hi Ricardo,

I am actually running master, I just verified it and also pull the latest status to make sure I was.

The bug is still there even with your patches.

Let me know if you need me to run any tests.

Hi Diego

That is interesting, we even have a very similar selftest covering your usecase http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/selftest/cases/wic.py?id=01619c9acbf8ac5c66817a5cdfe9a8819891989b#n611 

Can you share the exact hash number of your version?

How do you use the wks, from bitbake or using wic?


Thanks

Hi Ricardo,

Here is the last commit:

git log -n 1 
commit 344deae2004356d121e4302640d57ff579cc1dc6 (HEAD -> master, origin/master, origin/HEAD)
Author: Khem Raj <raj.khem@gmail.com>
Date:   Tue May 12 16:46:31 2020 -0700

    wayland: fix condition for strndup detection
    
    current check does not work with gcc10
    
    (From OE-Core rev: ce6c50ea68c34e02b487d54fdd9288e04833cdef)
    
    Signed-off-by: Khem Raj <raj.khem@gmail.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

I invoke WIC from bitbake, this is my environment:

WKS_FILE = "edge.wks"
WIC_CREATE_EXTRA_ARGS = "--no-fstab-update"

It would be great if you could:


bitbake -c cleansstate $IMAGE

bitbake $IMAGE


And then share all the files under:

tmp/work/$MACHINE/$IMAGE/$VERSION/tmp


Hopefully that is enough and we do not need to look into

tmp/work/$MACHINE/$IMAGE/$VERSION/build-wic 

Thanks!

(In reply to comment #6)
> It would be great if you could:
> 
> 
> bitbake -c cleansstate $IMAGE
> 
> bitbake $IMAGE
> 
> 
> And then share all the files under:
> 
> tmp/work/$MACHINE/$IMAGE/$VERSION/tmp
> 
> 
> Hopefully that is enough and we do not need to look into
> 
> tmp/work/$MACHINE/$IMAGE/$VERSION/build-wic 
> 
> Thanks!

Now this is weired, I did a mistake and the permissions were set correctly:

part /          --source rootfs                                        --ondisk mmcblk0 --fstype=ext4 --label rootfsA --align 1024 --fixed-size 370M --exclude-path=etc/certs

The previous line produces a rootfs with the permissions set ok

part /          --source rootfs                                        --ondisk mmcblk0 --fstype=ext4 --label rootfsA --align 1024 --fixed-size 370M --exclude-path=etc/certs/

Produces a rootfs with the permissions set wrong (please not the / at the end of the line).

I am attaching the output you asked for in the second case. 

As a reference here is what Wic's help says about that last /: "If the path ends with a slash, only the content of the directory is omitted, not the directory itself."

Created attachment 4674 [details]
Contents of tmp/work/$MACHINE/$IMAGE/$VERSION/tmp

Thanks for the files.

I do not like the output of log.do_flush_pseudodb

DEBUG: Executing shell function do_flush_pseudodb
Server refused shutdown.  Remaining client fds: 1
Client pids: 210685 
Server will shut down after all clients exit.
DEBUG: Shell function do_flush_pseudodb finished


It should not show any remaining clients. It is probably because it is running on parallel do_image_ext4. That will also explain why it is not very consistent the error.

I am trying to replicate the bug here. And then send a new patch to fix the tasks order.

On the meanwhile try to have only wic on IMAGE_FSTYPES and see if that fixes your issue and please report back ;).

I managed to replicate the bug with:



ricardo@neopili:/tmp$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Enumerating objects: 475079, done.
remote: Counting objects: 100% (475079/475079), done.
remote: Compressing objects: 100% (112228/112228), done.
remote: Total 475079 (delta 355624), reused 474667 (delta 355274)
Receiving objects: 100% (475079/475079), 163.34 MiB | 2.77 MiB/s, done.
Resolving deltas: 100% (355624/355624), done.
ricardo@neopili:/tmp$ cd poky/
ricardo@neopili:/tmp/poky$ git checkout 344deae2004356d121e4302640d57ff579cc1dc6
Note: switching to '344deae2004356d121e4302640d57ff579cc1dc6'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 344deae200 wayland: fix condition for strndup detection
ricardo@neopili:/tmp/poky$ gvim ./meta/recipes-core/images/core-image-minimal.bb
ricardo@neopili:/tmp/poky$ cat meta/recipes-core/images/core-image-minimal.bb
SUMMARY = "A small image just capable of allowing a device to boot."

IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL}"

IMAGE_LINGUAS = " "

LICENSE = "MIT"

inherit core-image

IMAGE_ROOTFS_SIZE ?= "8192"
IMAGE_ROOTFS_EXTRA_SPACE_append = "${@bb.utils.contains("DISTRO_FEATURES", "systemd", " + 4096", "" ,d)}"

WKS_FILE = "edge.wks"
IMAGE_FSTYPES = "ext4 wic"
ricardo@neopili:/tmp/poky$ gvim ./scripts/lib/wic/canned-wks/edge.wks
ricardo@neopili:/tmp/poky$ cat ./scripts/lib/wic/canned-wks/edge.wks
part /  --source rootfs --ondisk mmcblk0 --fstype=ext4 --label rootfsA --exclude-path=etc/certs/ 
part /  --source rootfs --ondisk mmcblk0 --fstype=ext4 --label rootfsB
ricardo@neopili:/tmp/poky$ docker run --rm -it -v $PWD:/workdir crops/poky:debian-9 --workdir=/workdir
pokyuser@6247199d1783:/workdir$ . oe-init-build-env 
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you with some default values. To add additional metadata layers
into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
    http://www.openembedded.org/


### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

Other commonly useful commands are:
 - 'devtool' and 'recipetool' handle common recipe tasks
 - 'bitbake-layers' handles common layer tasks
 - 'oe-pkgdata-util' handles common target package tasks
pokyuser@6247199d1783:/workdir/build$ bitbake core-image-minimal

This fixes the issue for me:


Can you give it a try? I will send later today a patch to the mailing list


diff --git a/meta/classes/image_types_wic.bbclass b/meta/classes/image_types_wic.bbclass
index 96ed0473ee..7b1db50a28 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -142,7 +142,7 @@ python do_rootfs_wicenv () {
     depdir = d.getVar('IMGDEPLOYDIR')
     bb.utils.copyfile(os.path.join(outdir, basename) + '.env', os.path.join(depdir, basename) + '.env')
 }
-addtask do_flush_pseudodb after do_image before do_image_wic
+addtask do_flush_pseudodb after do_rootfs before do_image do_image_qa
 addtask do_rootfs_wicenv after do_image before do_image_wic
 do_rootfs_wicenv[vardeps] += "${WICVARS}"
 do_rootfs_wicenv[prefuncs] = 'set_image_size'

It is fixed!

Thanks so much Ricardo.

Thanks for your report. I have sent the patch. https://patchwork.openembedded.org/series/24197/

And now it is in master
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=81e481b0177d3280f46c204e107f20dfd3ebc0f6

So I guess it is ok to close this up. Thanks for the report