After having used the new recipe specific sysroots (RSS) for a while a serious problem occurred for me today. Suddenly recipes started to fail to build all over the place complaining about non-existing users/groups. After some debugging it seems that what is happening is that something has caused base-passwd to be invalidated. This leads to a message like this appearing in the log.do_install file for the failing recipes:

NOTE: base-passwd exists in sysroot, but is stale (base-passwd.5c851c4442756f2ec9415e966858e933 vs. base-passwd.68b489dd6c4657f8ec785e48cdb5bd78), removing.
DEBUG: Removing manifest: /home/pkj/dists/cvp-master/builds/q3505/tmp/work/all-poky-linux/useraddtest/1.0-r0/recipe-sysroot/etc/passwd
DEBUG: Removing manifest: /home/pkj/dists/cvp-master/builds/q3505/tmp/work/all-poky-linux/useraddtest/1.0-r0/recipe-sysroot/etc/group
...

And then I guess the base-passwd files are reinstalled in the sysroot (but there are no log messages about this?). However, since the entire sysroot is not recreated (i.e., do_prepare_recipe_sysroot is not run), the useradd/groupadd commands are not rerun and the user/group that the recipe had previously created in the sysroot are gone and thus do_install() fails when it tries to use them.

To reproduce:

1) Create the following recipe as meta/recipes-core/useraddtest/useraddtest_1.0.bb:

LICENSE = "CLOSED"

inherit allarch useradd

do_install () {
	mkdir -p ${D}${datadir}
	echo test > ${D}${datadir}/test
	chown test:test ${D}${datadir}/test
}

USERADD_PACKAGES = "${PN}"

GROUPADD_PARAM_${PN} = "--system test"
USERADD_PARAM_${PN} = "--system --gid test test"

FILES_${PN} = "${datadir}/test"

2) bitbake useraddtest

3) Make any trivial changes to the base-passwd and useraddtest recipes to cause them to rebuild.

4) bitbake useraddtest

The build fails with:

| DEBUG: Executing shell function do_install
| chown: invalid user: `test:test'

My take on this is that if anything requires anything in the RSS to be updated, then the entire RSS needs to be rebuilt. It is not enough to only remove the files of the stale package and reinstall it, as some function of some other recipe may have modified them (i.e., what happens with the passwd/group files installed by base-passwd but modified by any recipe using the useradd bbclass). Currently it seems that useradd.bbclass is the only one doing something like this (at least in OE-Core), but since any recipe can hook into the same functions the problem is not limited to that.