In Fido & Jethro(?) if one used useradd-staticids and USERADD_UID_TABLES dynamically added users do not get added to the images and are just silently ignored. This was apparent for recipes in a way that do_install coudln't find users/groups. This was solved in some recipies by defining them by numerical values on meta-openembedded so we did try to use the same mechanism. Easy to demonstrate use case is postgresql recipe from meta-openembedded. In postgresql.inc's do_install there is chown postgresql:postresql. This was failing because postgresql user/group cannot be found even thou it has been defined in recipe. | chown: invalid user: 'postgres:postgres' | WARNING: /oe/zc702-krogoth/poky/build-ept-demo-zc702-ept/tmp/work/cortexa9hf-neon-vaisala-linux-gnueabi/postgresql/9.4.5-r0.0/temp/run.do_install.31609:1 exit 1 from 'chown -R postgres:postgres /oe/zc702-krogoth/poky/build-ept-demo-zc702-ept/tmp/work/cortexa9hf-neon-vaisala-linux-gnueabi/postgresql/9.4.5-r0.0/image/var/lib/postgresql' If I just changed that chown line to be with ${usernum} and ${groupnum} variables then it compiled nicely but users did not get propagated to the device (like it did with fido). Next I tried enabling USERADD_ERROR_DYNAMIC but that didn't complain a thing. I believe the problem was introduced with: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=krogoth&id=4f2c3525807f79d4e264d39bb70e0b8e4096355b Where in meta/classes/useradd-staticids.bbclass:def rewrite_useradd(params): if uaargs.LOGIN not in users: continue This just skips out of the loop and does not cause useradd string to be generated from recipe entry. To debug this issue a bit I did following change: diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass index a9b506d..88b41da 100644 --- a/meta/classes/useradd-staticids.bbclass +++ b/meta/classes/useradd-staticids.bbclass @@ -112,6 +112,7 @@ def update_useradd_static_config(d): users = merge_files(get_passwd_list(d), 7) if uaargs.LOGIN not in users: + bb.warn("%s: Login '%s' not in users, verify configuration files!" % (d.getVar('PN', True), uaargs.LOGIN)) continue field = users[uaargs.LOGIN] @@ -244,6 +245,7 @@ def update_useradd_static_config(d): groups = merge_files(get_group_list(d), 4) if gaargs.GROUP not in groups: + bb.warn("%s: Group '%s' not in groups, verify configuration files!" % (d.getVar('PN', True), gaargs.GROUP)) continue field = groups[gaargs.GROUP] With this I get output like following during recipe parsing time: WARNING: ..../poky/meta/recipes-connectivity/dhcp/dhcp_4.3.3.bb: dhcp: Login 'dhcp' not in users, verify configuration files! WARNING: ..../poky/meta/recipes-connectivity/bind/bind_9.10.3-P3.bb: bind: Login 'bind' not in users, verify configuration files! WARNING: ..../poky/meta/recipes-connectivity/avahi/avahi_0.6.32.bb: avahi: Login 'avahi' not in users, verify configuration files! and during build time: WARNING: postgresql: Login 'postgres' not in users, verify configuration files! WARNING: postgresql: Group 'postgres' not in groups, verify configuration files! Workaround for the problem is to add necessary entries to files pointed by USERADD_UID_TABLES and USERADD_GID_TABLES. I believe this bug/change was introduced when refactoring of the useradd-staticids code. If this a intentional change then I recommend adding a warning during build time and documenting the new behavior.
Sorry for typo in first lines: "In Fido & Jethro(?) if one used useradd-staticids and USERADD_UID_TABLES dynamically added users do not get added to the images and are just silently ignored." -> "In Fido & Jethro(?) if one used useradd-staticids and USERADD_UID_TABLES dynamically added users in recipies did get added properly into the image. With krogoth this is no longer true and they do not get added."
If a user is added by a recipe but doesn't exist in the static map it's silently skipped?
(In reply to comment #2) > If a user is added by a recipe but doesn't exist in the static map it's > silently skipped? Yes. And thus there is no useradd during image creation.
(In reply to comment #3) > (In reply to comment #2) > > If a user is added by a recipe but doesn't exist in the static map it's > > silently skipped? > > Yes. And thus there is no useradd during image creation. Apologies, posted an incomplete comment. This is an interesting one, the behaviour change is clear and arguably a regression. It's clear that silently ignoring the requested user creation is not desirable. I'd argue that if you're using staticids you don't want dynamic users created either (the old behaviour). Based on that premise, is a warning sufficient when it will result in errors later? Perhaps this should be an error when a recipe tries to add a user which doesn't exist in the static map? (Adding Peter to CC as he worked on recent changes here)
(In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > If a user is added by a recipe but doesn't exist in the static map it's > > > silently skipped? > > > > Yes. And thus there is no useradd during image creation. > > Apologies, posted an incomplete comment. This is an interesting one, the > behaviour change is clear and arguably a regression. > > It's clear that silently ignoring the requested user creation is not > desirable. > > I'd argue that if you're using staticids you don't want dynamic users > created either (the old behaviour). Based on that premise, is a warning > sufficient when it will result in errors later? > > Perhaps this should be an error when a recipe tries to add a user which > doesn't exist in the static map? > > (Adding Peter to CC as he worked on recent changes here) You could have essential services/users that are writing to disk as static ids. Then when you do software updates for your devices they stay the same. If you have services that are using ramdisk or just for runtime ACL then we don't really care what their UID/GID numerically is -- you only care about matching then. So there is a place for dynamic UID/GIDs in systems. If disallowing dynamic UID/GIDs when using static ids is expected change then I suppose minimum is to give warning/error about it and document it in manual. However I would recommend support dynamic even if static maps are defined. Then static maps would be looked up for uids/gids for found users/groups and otherwise default uid/gid generation would be used. If user defines USERADD_ERROR_DYNAMIC then that would disable dynamic generation. If fido you just created users and groups dynamically and then you paid the price of uid/gid possibly changing in the device. In postgresql case the uid/gid wasn't event dynamic -- they were defined in recipe with exact uid/gid numbers. I think this should still be supported. One problem thou is what if postgresql uid/gid was different on your previous system -- can you override the uid/gid with static map?
Thanks for the feedback. I think we should default to raising an error when a user/group is added which doesn't exist in the static map, but allow a configuration option to be set which turns this into a warning and creates a dynamic user/group instead.
Ok, this is clearly my fault. I do not know how I could have missed this case when I rewrote the code, since we rely on all users having static UIDs/GIDs and thus have USERADD_ERROR_DYNAMIC set. I will send a patch that restores the pre-Krogoth behavior of raising an exception if no numeric ID is found for a user or group and USERADD_ERROR_DYNAMIC is set. Changing the exception to a warning in case USERADD_ERROR_DYNAMIC is not set may be a good idea, but I think it needs to be a feature that is explicitly enabled (because one gets a huge number of warnings per missing user or group as the code is executed multiple times...) My suggestion is to change USERADD_ERROR_DYNAMIC to take "error" (or "1" for backwards compatibility) and "warn" as values to control the behavior. I will do this as a second commit.
oe-core adc0f830a695c417b4d282fa580c5231e1f0afbe was merged into master. Looks serious enough that someone should backport and verify this to krogoth.
I backported and tested this. The patch has been sent to the oe-core mailing list for krogoth. (cc'd Armin)