Bug 4739

Created attachment 1282 [details]
useradd.bbclass: Run do_package_setscene per-MACHINE

Here is an RFC patch that fixes the issue for my builds.  I've also tested it against libnss-mdns/avahi in openembedded-core to confirm that staging of the avahi user still works correctly.  I don't think our multi-MACHINE configuration is represented in openembedded-core so I don't think I can provide a better upstream test case.

I think this is a use case we're not currently supported or tested (recipe A using users and groups setup by recipe B).

do_package is certainly not meant to be machine specific, that would make no sense and cause untold other problems.

I'm wondering if this is as simple to fix as changing useradd_sysroot() in useradd.bbclass to list both package_setscene and populate_sysroot_setscene in the conditional?

Yeah, my first workaround (in the "users-and-groups" recipe) was:

useradd_sysroot_sstate () {
	if [ "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
	then
		useradd_sysroot
	fi
}

do_populate_sysroot_setscene[depends] = "${USERADDSETSCENEDEPS}"

but that conflicted with a change you had made to check "package_setscene" instead, so I wasn't sure if manipulating do_populate_sysroot_setscene was a good idea.

I could test a patch like:

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index 1b419bd..fe64b7f 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -129,7 +129,7 @@ useradd_sysroot () {
 }
 
 useradd_sysroot_sstate () {
-	if [ "${BB_CURRENTTASK}" = "package_setscene" ]
+	if [ "${BB_CURRENTTASK}" = "package_setscene" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
 	then
 		useradd_sysroot
 	fi

and resubmit if this is a use-case you think is worth supporting.  Otherwise I can maintain it locally; I filed here in case others are seeing this in their configurations.

Give it a go and see if it works?

OK, will do.

(In reply to comment #6)
> OK, will do.

Any update on whether this worked out of interest?

(In reply to comment #7)
> (In reply to comment #6)
> > OK, will do.
> 
> Any update on whether this worked out of interest?

Yes, sorry for the delay.  Our builds take a long time, and I got caught by the Python 2.7 update on oe-core master.  I'll be upgrading my RHEL-5.5-based environment to something more modern soon.  For these tests I went back to testing bitbake master and openembedded-core master as of 2013-06-13.  But I want to test the patch against tip too, so I'll probably end up building Python 2.7 from source.

As for preliminary results, the patch in comment #4 worked for our build, but in oe-core it caused a do_configure error in avahi:

NOTE: Executing glib-gettextize --force --copy
/distro/build/pure-oe-master-sstate/tmp-eglibc/work/i586-oe-linux/avahi/0.6.31-r6.1/temp/run.do_configure.1288: line 143: glib-gettextize: command not found
ERROR: Function failed: do_configure (log file is located at /distro/build/pure-oe-master-sstate/tmp-eglibc/work/i586-oe-linux/avahi/0.6.31-r6.1/temp/log.do_configure.1288)

Adding the do_populate_sysroot_setscene[depends] line seems to fix the error.  I can't explain why though:

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index e50c889..1aafe06 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -129,7 +129,7 @@ useradd_sysroot () {
 }
 
 useradd_sysroot_sstate () {
-	if [ "${BB_CURRENTTASK}" = "package_setscene" ]
+	if [ "${BB_CURRENTTASK}" = "package_setscene" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
 	then
 		useradd_sysroot
 	fi
@@ -151,6 +151,7 @@ USERADDSETSCENEDEPS_virtclass-cross = ""
 USERADDSETSCENEDEPS_class-native = ""
 USERADDSETSCENEDEPS_class-nativesdk = ""
 do_package_setscene[depends] += "${USERADDSETSCENEDEPS}"
+do_populate_sysroot_setscene[depends] += "${USERADDSETSCENEDEPS}"
 
 # Recipe parse-time sanity checks
 def update_useradd_after_parse(d):

Without the extra do_populate_sysroot_setscene[depends] line, avahi's do_populate_sysroot_setscene task fails:

[...]
Running useradd commands...
grep: /distro/build/pure-oe-master-sstate/tmp-eglibc/sysroots/qemux86/etc/passwd: No such file or directory
pseudo: pseudo: can't run useradd: Permission denied
[...]

and so avahi's do_configure task is attempted:

  12019:WARNING: Setscene task 61 (/distro/sources/openembedded-core/meta/recipes-connectivity/avahi/avahi_0.6.31.bb, do_populate_sysroot_setscene) failed with exit code '1' - real task will be run instead

but glib-2.0-native hasn't been staged by the time it's run, so glib-gettextize isn't available.

I still have to retest the dependency addition on my build and on oe-core master.

Created attachment 1320 [details]
useradd.bbclass: Run useradd_sysroot during populate_sysroot_setscene

I set up a Python 2.7-based environment and tested this new patch by building libnss-mdns normally and then with shared state, checking the state of avahi users in the sysroot, and comparing with logs from an unpatched build.

The patch introduces two changes to the build process: 1) the ordering of avahi _setscene tasks is slightly different to accomodate avahi:do_populate_sysroot_setscene's base-passwd/shadow-native/shadow-sysroot task dependencies. 2) the avahi users are staged immediately by do_populate_sysroot_setscene instead of on-demand by do_package_setscene (e.g. when libnss-mdns is rev-bumped).  I think these changes should be harmless.

I also tested the patch against our internal distro and it enables the desired use case, where one recipe uses users and groups set up by another recipe, even across multiple MACHINE builds.

Sorry for the delayed reply. I'm going to test this patch a little locally and post it on the OE-Core mailing list for review. I think it should be reasonable to merge it.

Thanks Richard, I'll watch for discussion on the mailing list.

Patch merged to master, thanks! http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=f24dda3de90696c5dc33170bcdaa83137cafff9d