Bug 11007

Summary: build fails after user add/delete commands
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Humberto Ibarra <humberto.ibarra.lopez>
Component: coreAssignee: Patrick Ohly <patrick.ohly>
Status: VERIFIED FIXED QA Contact: oscar lopez arandas <oscarx.lopez.arandas>
Severity: normal    
Priority: Medium+ CC: jose.perez.carranza, meta.mr.watcher, meta.watcher, patrick.ohly, ross.burton
Version: 2.3   
Target Milestone: 2.3 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)
Attachments:
Description Flags
do_rootfs
none
passwd
none
passwd-
none
group
none
group-
none
shadow
none
shadow-
none
gshadow
none
gshadow- none

Description Humberto Ibarra 2017-02-01 18:18:42 UTC 
Created attachment 3614 [details]
do_rootfs

I was running Testcase 799 (https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=799), which consists in issuing user/group commands at build, and the build process failed. These are the steps contained in 799 TC:


1. Download the poky source and set the environment

2. Add the following lines in conf/local.conf

INHERIT += "extrausers"

EXTRA_USERS_PARAMS = "\
useradd -p '' tester1; \
useradd -p 'vz4kBbiMF9z6E' tester2; \
groupadd developers; \
userdel games; \
groupdel video; \
groupmod -g 1020 developers; \
usermod -s /bin/sh tester1; \
usermod -g users -G developers,sys tester2; \
"

The above settings do the following things:
a. Add a user tester1 with empty password
b. Add a user tester2 with password 123456
c. Add a group developers
d. Delete user games
e. Delete group video
f. Set the gid of developers to 1020
g. For tester1, set the login shell to /bin/sh
h. For tester2, set the initial group to users. Set the supplementary groups to developers and sys.

3. Build the image


EXPECTED RESULTS:

The build should end successfully

ACTUAL RESULTS:

Got a exception in python code, with the sort_passwd function. (See attachments)

---

I believe this has something to do with the 'games' key in a dictionary, which is being accessed but does not exist
Comment 1 Ross Burton 2017-02-01 20:33:12 UTC 
Moving to Maxin who understands the useradd stuff far more than I do!
Comment 2 Maxin B. John 2017-02-02 15:33:52 UTC 
Setting this in the "local.conf" file helps here.

SORT_PASSWD_POSTPROCESS_COMMAND=""

This error seems to be introduced by this commit (adding Patrick in cc):

commit 753471db4534f121fe03b5d495fc3ec44f7207e7
Author: Patrick Ohly <patrick.ohly@intel.com>
Date:   Mon Jan 9 11:09:42 2017 +0100

    rootfs-postcommands.bbclass: sort passwd entries
Comment 3 Maxin B. John 2017-02-02 15:38:29 UTC 
Also setting Documentation change required as "yes" since we don't have documentation for "SORT_PASSWD_POSTPROCESS_COMMAND" variable.
Comment 4 Patrick Ohly 2017-02-02 15:58:29 UTC 
What is the content of the /etc passwd, group, shadow and related backup files at the time of the failure?
Comment 5 Maxin B. John 2017-02-02 16:17:45 UTC 
Created attachment 3615 [details]
passwd
Comment 6 Maxin B. John 2017-02-02 16:18:54 UTC 
Created attachment 3616 [details]
passwd-
Comment 7 Maxin B. John 2017-02-02 16:19:41 UTC 
Created attachment 3617 [details]
group
Comment 8 Maxin B. John 2017-02-02 16:20:00 UTC 
Created attachment 3618 [details]
group-
Comment 9 Maxin B. John 2017-02-02 16:20:29 UTC 
Created attachment 3619 [details]
shadow
Comment 10 Maxin B. John 2017-02-02 16:21:46 UTC 
Created attachment 3620 [details]
shadow-
Comment 11 Maxin B. John 2017-02-02 16:22:49 UTC 
Created attachment 3621 [details]
gshadow
Comment 12 Maxin B. John 2017-02-02 16:23:06 UTC 
Created attachment 3622 [details]
gshadow-
Comment 13 Patrick Ohly 2017-02-02 16:29:40 UTC 
There is an inconsistency in these files: "shadow-" contains a "games" entry, "passwd-" does not.

Is that normal?

My expectation was that the backup files represent the state before the last operation, so I'd expect "games" to be in "passwd-" and "shadow-" directly after removing the "games" user.

Regardless whether the expectation is right or wrong, I suggest to simply delete the backup files instead of sorting them, as they can't be sorted when they are inconsistent and, more importantly, do not really belong into the target rootfs.
Comment 14 Patrick Ohly 2017-02-10 12:05:40 UTC 
Fixed by removing the backup files instead of sorting, as discussed with Maxin.

See "rootfspostcommands: remove shadow backup files instead of trying to sort".
Comment 15 Ross Burton 2017-02-21 17:16:32 UTC 
Merged in oe-core e5628c80a52f3caeea9d9dc7f67d1b8a61222aef.
Comment 16 oscar lopez arandas 2017-04-20 16:16:10 UTC 
build with success after user add/delete commands

Build Configuration:
BB_VERSION        = "1.33.3"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i686-poky-linux"
MACHINE           = "genericx86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2+snapshot-20170420"
TUNE_FEATURES     = "m32 core2"
COMMIT:           = "HEAD:eff56e4f0d59b1d965a68e4f009b7f07717b7edd"

STEPS:

1. Download the poky source and set the environment

2. Add the following lines in conf/local.conf

INHERIT += "extrausers"

EXTRA_USERS_PARAMS = "\
useradd -p '' tester1; \
useradd -p 'vz4kBbiMF9z6E' tester2; \
groupadd developers; \
userdel games; \
groupdel video; \
groupmod -g 1020 developers; \
usermod -s /bin/sh tester1; \
usermod -g users -G developers,sys tester2; \
"

3. Build the image


EXPECTED OUTCOME:

The build should end successfully

ACTUAL OUTCOME:
The build  end successfully

NOTES:
----snip----
poky-linux/core-image-minimal/1.0-r0/rootfs -g users -G developers,sys tester2]
usermod: no changes
DEBUG: Shell function set_user_group finished
NOTE: Executing ssh_disable_dns_lookup ...
DEBUG: Executing shell function ssh_disable_dns_lookup
DEBUG: Shell function ssh_disable_dns_lookup finished
NOTE: Executing sort_passwd ...
DEBUG: Executing python function sort_passwd
DEBUG: Python function sort_passwd finished
NOTE: Executing write_image_manifest ...
DEBUG: Executing python function write_image_manifest
DEBUG: Python function write_image_manifest finished
NOTE: Added oe-repo repo from file:///home/oscar/11007/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo
6 files removed

NOTE: Executing: ldconfig -r/home/oscar/11007/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs-c new -v
DEBUG: Python function do_rootfs finished
----snip----