Bug 12107

Summary: useradd-staticids: groupadd: GID already exists
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Juro Bystricky <juro.bystricky>
Component: coreAssignee: Eilís Ní Fhlannagáin <pidge>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: alex.kiernan, ecordonnier, ernstp, JPEWhacker, lukas.durfina, maxin.john, meta.mr.watcher, meta.watcher, pidge, randy.macleod, richard.purdie
Version: 2.4.1   
Target Milestone: 4.3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
log showing the error
none
/etc/group renamed
none
/etc/passwd renamed none

Description Juro Bystricky 2017-09-20 20:04:07 UTC
Created attachment 4017 [details]
log showing the error

Run 
$ bitbake-core-image-sato-sdk-ptest

this will create files /etc/group and /etc/passwd
Note that running the above command several times may result in different PID/UID as they are assigned based on first come bases and order is not guaranteed.
To achieve a reproducible result, we need to use static IDs via

USERADDEXTENSION = "useradd-staticids"

So one would think that this would work as intended, giving deterministic GID/UIDs (reproducible-passwd, reproducible-group are renamed files created before: /etc/group etc/passwd)

local conf:

USERADDEXTENSION = "useradd-staticids"
USERADD_ERROR_DYNAMIC ??= "warn"
USERADD_UID_TABLES += "conf/distro/include/reproducible-passwd"
USERADD_GID_TABLES += "conf/distro/include/reproducible-group"

Place the attached files reproducible-password, reproducible-group in meta-poky/conf/distro/include and run:

$ bitbake-core-image-sato-sdk-ptest

But we get an error:

GID to multiple UID, getting the error:
groupadd: GID '997' already exists.


We also get the following warnings:
WARNING: /data/master/poky-contrib/meta/recipes-support/user-creation/xuser-account_0.1.bb: xuser-account - xuser-account: username xuser does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-graphics/wayland/weston_2.0.0.bb: weston - weston: groupname weston-launch does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-graphics/builder/builder_0.1.bb: builder - builder: username builder does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-extended/cups/cups_2.2.4.bb: cups - cups: groupname lpadmin does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-devtools/distcc/distcc_3.2.bb: distcc - distcc: groupname distcc does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: username systemd-timesync does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: username systemd-network does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: username systemd-resolve does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: username polkitd does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: groupname lock does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd: groupname systemd-journal does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-core/systemd/systemd_234.bb: systemd - systemd-extra-utils: username systemd-bus-proxy does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-connectivity/ppp-dialin/ppp-dialin_0.1.bb: ppp-dialin - ppp-dialin: username ppp does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-connectivity/dhcp/dhcp_4.3.6.bb: dhcp - dhcp-server: username dhcp does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-connectivity/bind/bind_9.10.5-P3.bb: bind - bind: username bind does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-connectivity/avahi/avahi_0.6.32.bb: avahi - avahi-autoipd: username avahi-autoipd does not have a static ID defined.
WARNING: /data/master/poky-contrib/meta/recipes-connectivity/avahi/avahi-ui_0.6.32.bb: avahi-ui - avahi-autoipd: username avahi-autoipd does not have a static ID defined.
Comment 1 Juro Bystricky 2017-09-20 20:05:28 UTC
Created attachment 4018 [details]
/etc/group renamed
Comment 2 Juro Bystricky 2017-09-20 20:06:06 UTC
Created attachment 4019 [details]
/etc/passwd renamed
Comment 3 Ross Burton 2017-09-26 13:38:24 UTC
Has this been fixed with oe-core fc3a86ae68919cec72c1a8ae0f9ba1f98ae13f0d?
Comment 4 Juro Bystricky 2017-09-26 15:45:06 UTC
(In reply to comment #3)
> Has this been fixed with oe-core fc3a86ae68919cec72c1a8ae0f9ba1f98ae13f0d?

distcc problem seems fixed, definitely an improvement.
However, I still get 
WARNING: apt - apt: username _apt does not have a static ID defined.
WARNING: avahi - avahi-autoipd: username avahi-autoipd does not have a static ID defined.

(possibly some other problems). 

In a nutshell, this is what I want to do:

1. build an image, image creates etc/passwd etc/group
2. use these very same etc/passwd etc/group files as "templates" for static ids when building the same image the next time.

This should work without errors or warnings, n'est-ce pas?
Comment 5 Maxin B. John 2018-01-16 17:22:58 UTC
Hi Juro,


Tried the build with similar setup and got the following error as mentioned here. Still, avahi recipe has the following line in avahi.inc. 

USERADD_PARAM_avahi-autoipd = "--system --home /run/avahi-autoipd \
                               --no-create-home --shell /bin/false \
                                --user-group \
                                -c \"Avahi autoip daemon\" \
                                avahi-autoipd"

We don't have a corresponding entry for "avahi-autopid" in the "reproducible-passwd" file. I think we should have that entry in the "reproducible-passwd" file as well. Please tell me if I missed any other details here.


WARNING: avahi - avahi-autoipd: system username avahi-autoipd does not have a static ID defined. Add avahi-autoipd to one of these files: /home/maxin/poky/build/conf/distro/include/reproducible-passwd
ERROR: avahi-0.7-r0 do_prepare_recipe_sysroot: avahi: groupadd command did not succeed.
ERROR: avahi-0.7-r0 do_prepare_recipe_sysroot: Function failed: useradd_sysroot (log file is located at /home/maxin/poky/build/tmp/work/core2-64-poky-linux-musl/avahi/0.7-r0/temp/log.do_prepare_recipe_sysroot.35068)
ERROR: Logfile of failure stored in: /home/maxin/poky/build/tmp/work/core2-64-poky-linux-musl/avahi/0.7-r0/temp/log.do_prepare_recipe_sysroot.35068
ERROR: Task (/home/maxin/poky/meta/recipes-connectivity/avahi/avahi_0.7.bb:do_prepare_recipe_sysroot) failed with exit code '1'
NOTE: Tasks Summary: Attempted 5209 tasks of which 4768 didn't need to be rerun and 1 failed.
Comment 6 Ernst Persson 2018-11-22 09:10:10 UTC
So for me the problem is when having a workspace without staticids, and then adding it, etc/group and etc/passwd needs to be rebuilt from scratch. But nothing triggers that rebuild, I have fc3a86ae68919cec72c1a8ae0f9ba1f98ae13f0d also.

I think you need to trigger a rebuild with something like this:

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index 124becd082..0509ba0a82 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -171,6 +171,7 @@ SSTATEPREINSTFUNCS_append_class-target = " useradd_sysroot_sstate"
 
 do_package_setscene[depends] += "${USERADDSETSCENEDEPS}"
 do_populate_sysroot_setscene[depends] += "${USERADDSETSCENEDEPS}"
+do_prepare_recipe_sysroot[vardeps] += "USERADDEXTENSION"
 USERADDSETSCENEDEPS_class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene"
 USERADDSETSCENEDEPS = "
Comment 7 Ernst Persson 2018-11-22 11:05:34 UTC
Ok that fix didn't really work, but this does:

recipes-core/base-passwd/base-passwd_%.bbappend
do_prepare_recipe_sysroot[vardeps] += "USERADDEXTENSION"
Comment 8 Ernst Persson 2018-12-21 09:10:00 UTC
My previous comment is also wrong, I still couldn't work around it.
Comment 9 Joshua Watt 2019-05-23 14:58:28 UTC
To recap (please correct me if I'm wrong), the issue here is that when you switch between using not using static user/group ID and using them, the change is not correctly detected to cause the core passwd and group to be rebuilt.

It is unlikely that I'm going to be able to get around to this, so I'm going to make it unassigned.
Comment 10 Richard Purdie 2023-12-15 13:54:44 UTC
https://git.yoctoproject.org/poky/commit/?id=e70f491b7eb2df47be7f9fe919704b5e32b626e5 adds a test for this which passes so we're assuming this is resolved.