Bug 13419

Summary: recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Alexander Kanavin <alex.kanavin>
Component: configurationAssignee: Eilís Ní Fhlannagáin <pidge>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: jermain.horsman, JPEWhacker, louis.rannou, niko.mauno, pidge, randy.macleod, richard.purdie, tgamblin
Version: unspecified   
Target Milestone: 4.3 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Alexander Kanavin 2019-06-26 12:43:57 UTC 
Scenario:

recipe A creates a group some-group

recipe B depends on A and adds a user some-user that belongs to some-group


This works when building from scratch, but breaks down when both recipes are populated from sstate-cache, e.g. this way:

bitbake B
rm -rf tmp/
bitbake -c package B

will produce:

NOTE: Executing SetScene Tasks
ERROR: B-git-r0 do_package_setscene: B: useradd command did not succeed.
WARNING: Logfile for failed setscene task is /home/alexander/development/mbient/build-thud-jetson/tmp/work/aarch64-mbient-linux/B/git-r0/temp/log.do_package_setscene.43058
WARNING: Setscene task (/home/alexander/development/mbient/build-thud-jetson/../sources/meta-mbient/meta-mbient/recipes-media/B/B_git.bb:do_package_setscene) failed with exit code '1' - real task will be run instead

Adding setscene dependencies to B does not help, as sstate code actually overrides them, except for some basic packages (e.g. base-passwd and shadow).

A workaround is to re-create the needed groups from B, but this should also Just Work.
Comment 1 Richard Purdie 2019-10-17 15:21:22 UTC 
Adding a pseudo-native dependency to target recipes may resolve this.
Comment 2 Richard Purdie 2019-12-04 13:59:16 UTC 
Qqestion is whether http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?id=911d784d9c838567dea40b94156e436d70b61c20 helps this at all or not. I now suspect not.

Do we have the full error from the failing command? I'm wondering what exactly is missing...
Comment 3 Niko Mauno 2019-12-04 14:57:41 UTC 
I just tried http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?id=911d784d9c838567dea40b94156e436d70b61c20 in thud (yocto-2.6.2) but still can every time reproduce the failure as thus far by resorting to

  rm -rf build/tmp-glibc
  bitbake -c package foobar

once sstate-cache has been populated. Relevant error message lines look like

  Running useradd commands...
  NOTE: foobar: Performing useradd with [--root .../foobar/1.0.0-r0/recipe-sysroot --home-dir / --gid foobar --groups foo1,foo2,foo3,upstream-recipe- 
 group1,upstream-recipe-group2 --system --shell /bin/false --uid 1234 foobar]
  useradd: group 'upstream-recipe-group1' does not exist
  useradd: group 'upstream-recipe-group2' does not exist
  ERROR: foobar: useradd command did not succeed.

for the record, for the time being as a workaround for this issue we're using following additional line in foobar recipe

  GROUPADD_PARAM_${PN}_append = ";upstream-recipe-group1;upstream-recipe-group2"
Comment 4 Randy MacLeod 2023-07-26 21:21:58 UTC 
Bulk move of 64 bugs to 4.3 M3 after a quick review.
If a bug is actually fixed, please add a commit link and resolve it.
-- Randy for YP bug team.
Comment 5 Trevor Gamblin 2023-09-14 16:35:28 UTC 
Moved to M4.
Comment 6 Eilís Ní Fhlannagáin 2023-11-02 13:10:23 UTC 
This is an issue with the useradd.bbclass. We're not seeing this for users because we obviously have USERADD_PARAM and friends in a recipe (recipe B), but as we're relying on the GROUPADD_PARAM from recipe A, useradd_preinst doesn't find any GROUPADD_PARAM, doesn't add the groups and fails out of the setscene task and just runs do_package.

So, interestingly,

bitbake B
rm -rf tmp/
bitbake -c package B (Errors, but runs the actual do_package)

In the tests I'm writing for this, my test recipe spits out (WARNING: Setscene task (/home/pidge/poky/meta-selftest/recipes-test/usergrouptests/creategroupdep.bb:do_package_setscene) failed with exit code '1' - real task will be run instead)

So, running 'bitbake -c package B' immediately after the above without deleting tmp will succeed because the actual do_package ran and created the group.

The question here is how to fix this correctly. We could look at the USERADD_PARAM, see if a non-system group is specified, check if there is a GROUP_PARAM for that specific group and if not create the appropriate GROUP_PARAM, which is how I'm leaning.
Comment 7 Jermain Horsman 2023-11-02 14:32:10 UTC 
This has generally been my experience as well.

- from a certain sstate the warning "WARNING: Setscene task (somerecipe.bb:do_package_setscene) failed with exit code '1' - real task will be run instead" will occur
- bitbake will continue and finish building everything (successfully), but exit with an error due to the previous warning
- sstate is updated and a subsequent run will be fine
Comment 8 Eilís Ní Fhlannagáin 2023-11-19 04:19:44 UTC 
I have a patch for this that adds each dependancies do_populate_sysroot_setscene to USERADDSETSCENEDEPS (and selftests). 

This seems to fix it, however I want to see how this works with some of the suggested suggestions for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084 and https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904 as these seem related or at least close enough to each other that I want to see how these all work together.

One thing mentioned in the other bug is the question of should a recipe that relies on anothers group have that recipe as a DEPENDS. I believe it should and we should document it.