I updated poky from 3.1.1 to 3.1.2 and now I frequently see things like these in my builds: 4: expect-5.45.4-r0 do_package - 5s (pid 302) [290A[J[1;31mERROR[0m: [31mexpect-5.45.4-r0 do_package: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_report_unihash(d) 0003: File: '/workdir/sources/poky/meta/classes/sstate.bbclass', lineno: 840, function: sstate_report_unihash 0836: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) 0837: 0838: if report_unihash: 0839: ss = sstate_state_fromvars(d) *** 0840: report_unihash(os.getcwd(), ss['task'], d) 0841:} 0842: 0843:# 0844:# Shell function to decompress and prepare a package for installation File: '/workdir/sources/poky/bitbake/lib/bb/siggen.py', lineno: 555, function: report_unihash 0551: 0552: if "." in self.method: 0553: (module, method) = self.method.rsplit('.', 1) 0554: locs['method'] = getattr(importlib.import_module(module), method) *** 0555: outhash = bb.utils.better_eval('method(path, sigfile, task, d)', locs) 0556: else: 0557: outhash = bb.utils.better_eval(self.method + '(path, sigfile, task, d)', locs) 0558: 0559: try: File: '/workdir/sources/poky/bitbake/lib/bb/utils.py', lineno: 420, function: better_eval 0416: if extraglobals: 0417: ctx = copy.copy(ctx) 0418: for g in extraglobals: 0419: ctx[g] = extraglobals[g] *** 0420: return eval(source, ctx, locals) 0421: 0422:@contextmanager 0423:def fileslocked(files): 0424: """Context manager for locking and unlocking file locks.""" File: '<string>', lineno: 1, function: <module> File "<string>", line 1, in <module> File: '/workdir/sources/poky/meta/lib/oe/sstatesig.py', lineno: 585, function: OEOuthashBasic 0581: 0582: update_hash("\n") 0583: 0584: # Process this directory and all its child files *** 0585: process(root) 0586: for f in files: 0587: if f == 'fixmepath': 0588: continue 0589: process(os.path.join(root, f)) File: '/workdir/sources/poky/meta/lib/oe/sstatesig.py', lineno: 548, function: process 0544: add_perm(stat.S_IXOTH, 'x') 0545: 0546: if include_owners: 0547: try: *** 0548: update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name) 0549: update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name) 0550: except KeyError: 0551: bb.warn("KeyError in %s" % path) 0552: raise Exception: KeyError: 'getpwuid(): uid not found: 1000' [0m Waiting for 7 running tasks to finish: I just erased sstate and the build dir and try again.
At least for the built I ran before it helped to erase sstate and the build dir.
Any idea how we'd reproduce this? We're not seeing it on the project autobuilder and nobody else as reported it yet :/
Hi Richard, I have no reproduce able way to create this problem but a quick google search reveals that I am not the only one seeing it: https://lists.yoctoproject.org/g/yocto/message/48244 https://lists.yoctoproject.org/g/yocto/message/49572 https://patches.openembedded.org/patch/168690/ Hehe and I got you ;) https://www.yoctoproject.org/irc/%23yocto.2020-01-14.log.html#t2020-01-14T18:36:58
The error means something in expect installed a file with the wrong ownership (1000 instead of what probably should be root). What I mean is we don't have other reports of expect doing this. We have seen the error before, the cause is probably different here though :/.
Created attachment 4744 [details] Build error log and environment poky.diff: the patch I used to keep the build system running Dockerfile: The docker file used to create the docker image to build the software buildscript: The script which is run inside the docker container to build the software console: The console output.
I ran into the same problem. My guess is that at some point there is a chroot into a directory without a /etc/passwd, since the docker image I run the build within has a passwd entry for the user. I hope the attached archive makes it possible to reproduce the bug.
I've also run into this a couple of times now when building 3.1 Dunfell in a Docker image.
Do you also see this with master or gatesgarth, or is this dunfell specific?
I'm more and more starting to suspect that this was an issue with our docker setup. We're running with -v /etc/passwd:/etc/passwd After some changes and upgrades this doesn't seem to happen anymore...
I don't see those anymore and the reason might be the way I kill bitbake/start a new build. I always call a script that does this: #!/bin/bash # kill all instances of bitbake ps -ef | grep bitbake for pid in $(ps -ef | awk '/\/bitbake\/bin\/bitbake/ {print $2}'); do echo "do kill -9 ${pid}"; kill -9 $pid; done ps -ef | grep bitbake echo "rm -f ${BUILDDIR}/hashserve.sock" rm -f ${BUILDDIR}/hashserve.sock
(In reply to comment #9) > I'm more and more starting to suspect that this was an issue with our docker > setup. We're running with -v /etc/passwd:/etc/passwd > > After some changes and upgrades this doesn't seem to happen anymore... We also copied the passwd file into the container.
This error has nothing to do with your /etc/passwd on host machine or in docker. Pseudo uses it's own passwd/group files that are under directory provided by PSEUDO_PASSWD variable. This passwd files contain only users/groups that will be installed on target machine. So when running "pwd.getpwuid(s.st_uid).pw_name" it's trying to get the user info of the owner of the file/path "s" (s = os.lstat(path)). If it cannot find this user in PSEUDO_PASSWD it most likely means there is a host user contamination. I would suggest determining which file is causing the problem: --- a/meta/lib/oe/sstatesig.py +++ b/meta/lib/oe/sstatesig.py @@ -510,6 +510,7 @@ def OEOuthashBasic(path, sigfile, task, d): def process(path): s = os.lstat(path) + bb.warn('path ' + str(path)) if stat.S_ISDIR(s.st_mode): update_hash('d') and make sure this file is owned by root (or other user/group that will be on target). Now to install files as root you need to use fakeroot. Outside fakeroot you won't have permission to run "chown". I guess this file is created with do_install that already uses it. If you have some custom task creating this path/file you need to make it use fakeroot, create your task with: - do_custom_task() { + do_custom_task[depends] += "virtual/fakeroot-native:do_populate_sysroot" + fakeroot do_custom_task() { or add this lines to the recipe: do_custom_task[fakeroot] = "1" do_mytask[depends] += "virtual/fakeroot-native:do_populate_sysroot" Now since the task already uses fakeroot install them with "install" or run "chown 0:0 -R <PATH>" and the problem should be gone. Anyway it's possible another issue will appear: chown: changing ownership of '<PATH>': Operation not permitted It means you still have no permission despite being in fakeroot environment. Most likely it means you're trying to use "fakeroot powers" under the path that is set in PSEUDO_IGNORE_PATHS. Please note this variable not only disables the "pseudo path mismatch" error but also prevents fakeroot from executing commands under this path. First, make sure you're not adding some incorrect paths to PSEUDO_IGNORE_PATHS. Second, make sure you're not trying to execute fakeroot commands under any path that matches any value set in PSEUDO_IGNORE_PATHS variable. Note that this paths might not be specific, for example "${WORKDIR}/deploy-" will affect "${WORKDIR}/deploy-foo". If this is the case try not to remove these paths from PSEUDO_IGNORE_PATHS (they're there for a reason), instead move your work to another directory that doesn't match any value in PSEUDO_IGNORE_PATH and then execute your fakeroot commands. I think we should add more descriptive error message here.
Is anyone else still seeing this issue? I haven't seen this in any of my testing.
Closing since no one seems to be encountering this in the past few months.