14811 2022-05-23 07:57:21 +0000 Incorrect path matching in pseudo cause package failure in yocto 2026-10-01 15:48:12 +0000 1 1 6 Yocto Project Subprojects Pseudo pseudo unspecified x86 Multiple NEW Medium+ normal 6.2 1 fede.linux richard.purdie fede.linux richard.purdie yp.pseudo.watcher yp.watcher --- --- No (bug/feature does not impact docs) oldest_to_newest 93299 0 4868 fede.linux 2022-05-23 07:57:21 +0000 Created attachment 4868 Possible patch Preconditions/Environment ------------------------- Building a recipe which create files in a path that's a string prefix of one of the build / packaging directories. Here is the offending recipe: ------ SUMMARY = "Powerboard support module" LICENSE = "CLOSED" inherit module SRC_URI = "file://linux_module_4.1.0.tar.gz " S = "${WORKDIR}/pkg" ---- where `linux_module_4.1.0.tar.gz` is a tarball with the following structure: pkg/Makefile pkg/source1.c pkg/source2.c pkg/.. Triggering Action/Cause ----------------------- bitbake kernel-module-powerboard Expectation ----------- The recipe should build correctly Actual Result ------------- We alternate between one of the following results: ------------------- 0565: File: '/home/fede/yocto/sources/poky/meta/lib/oe/sstatesig.py', lineno: 555, function: process 0551: add_perm(stat.S_IXOTH, 'x') 0552: 0553: if include_owners: 0554: try: *** 0555: update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name) 0556: update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name) 0557: except KeyError as e: 0558: bb.warn("KeyError in %s" % path) 0559: msg = ("KeyError: %s\nPath %s is owned by uid %d, gid %d, which doesn't match " Exception: Exception: KeyError: 'getpwuid(): uid not found: 1000' Path ./pkgdata/kernel-module-powerboard is owned by uid 1000, gid 1000, which doesn't match any user/group on target. This may be due to host contamination. ------------------- or ------------------- File: '/usr/lib/python3.6/subprocess.py', lineno: 311, function: check_call 0307: if retcode: 0308: cmd = kwargs.get("args") 0309: if cmd is None: 0310: cmd = popenargs[0] *** 0311: raise CalledProcessError(retcode, cmd) 0312: return 0 0313: 0314: 0315:def check_output(*popenargs, timeout=None, **kwargs): Exception: subprocess.CalledProcessError: Command '['rm', '-rf', '/home/fede/yocto/build-delta80t/tmp/work/delta80_vt-poky-linux-gnueabi/kernel-module-powerboard/1.0-r0/sstate-build-package/']' died with <Signals.SIGABRT: 6>. ------------------- Reproducibility --------------- always Explanation --------------- I've found that the code that process the ignore list in pseudo is incorrect. In this example yocto run the packaging phase with PSEUDO_PATH_IGNORE=`pwd`/pkg plus other paths, `pkg` is inserted because it's the value of "${S}", but this way pseudo also ignore pkgdata causing the packaging process to run out of fakeroot and the errors above. the culprit is the following function in pseudo: int pseudo_client_ignore_path_chroot(const char *path, int ignore_chroot) { 26 char *next = strchr(p, ','); 27 if (next) 28 *next++ = '\0'; 29 - if (*p && !strncmp(path, p, strlen(p))) { 30 - pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path); 31 - ret = 1; 32 - break; the strcmp() matches both /pkg and /pkgdata I've attacched a possible fix as a patch. 94019 1 richard.purdie 2022-09-09 15:47:42 +0000 The trouble is that code was specifically designed to allow partial prefix matching and we do rely on that behaviour to simplify other areas of code. Whilst this patch will fix a specific corner case, I suspect it will break other usage. 94067 2 fede.linux 2022-09-26 08:38:34 +0000 If you have some concrete example it might be possible to tune the check to allow the required usecase without allowing for a wildcard match, or fix the other parts adding a * to explicitly allow a wider matching. The issue it's not blocking since I've avoided the problem simply renaming the folder inside the tar.gz but it has been pretty tricky and time consuming to find; maybe we could just detect the case during unpacking and generate a build error. 95074 3 richard.purdie 2023-03-07 13:03:36 +0000 Id support spotting problematic patterns and erroring if found. 4868 2022-05-23 07:57:21 +0000 2022-05-23 07:57:21 +0000 Possible patch 0001-Avoid-matching-on-prefixes-if-they-are-not-a-full-pa.patch text/plain 2531 fede.linux RnJvbSA4MmFjZjZkNzI2MWNkOWRkMmRhZThkMzgxNGY5OGNkZTc2YzMyYzcxIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBGZWRlcmljbyBHaW92YW5hcmRpIDxmZ2lvdmFuYXJkaUBhcmFn bmV0LmNvbT4KRGF0ZTogTW9uLCAyMyBNYXkgMjAyMiAwOToxMjo1MSArMDIwMApTdWJqZWN0OiBb UEFUQ0hdIEF2b2lkIG1hdGNoaW5nIG9uIHByZWZpeGVzIGlmIHRoZXkgYXJlIG5vdCBhIGZ1bGwg cGF0aAoKRml4IHRoZSBQU0VVRE9fSUdOT1JFX1BBVEggcHJvY2Vzc2luZyBsb29wIHRvIGRpc2Nh cmQKbWF0Y2hlcyBvbiBzdHJpbmcgcHJlZml4ZXMgYnV0IGluc3RlYWQgcGVyZm9ybSBtYXRjaGVz Cm9uIHBhdGggcHJlZml4ZXMuCgpGaXggYW4gZXJyb3IgaW4gY2FzZSBvZiBhIHJlY2lwZSBjcmVh dGUgc29tZSBmaWxlIGluIGEgZm9sZGVyCm5hbWVkIGBwa2dgLCBieSBkZWZhdWx0IHRoZSBgJHtT fWAgZm9sZGVyIGlzIGlnbm9yZSBpbiB5b2N0bwpzbyBgcGtnYCBlbmRzIGJlaW5nIGlnbm9yZS4g V2l0aG91dCB0aGUgcGF0Y2ggdGhlIGxvb3AKYWxzbyBpZ25vcmUgYHBrZ2RhdGFgIHJlc3VsdGlu ZyBpbiBhIGZhaWx1cmUgZHVyaW5nIHJlY2lwZQpwYWNrYWdpbmcuCi0tLQogcHNldWRvX2NsaWVu dC5jICAgICB8IDE0ICsrKysrKysrKystLS0tCiB0ZXN0L3Rlc3QtaWdub3JlLnNoIHwgMjYgKysr KysrKysrKysrKysrKysrKysrKysrKysKIDIgZmlsZXMgY2hhbmdlZCwgMzYgaW5zZXJ0aW9ucygr KSwgNCBkZWxldGlvbnMoLSkKIGNyZWF0ZSBtb2RlIDEwMDc1NSB0ZXN0L3Rlc3QtaWdub3JlLnNo CgpkaWZmIC0tZ2l0IGEvcHNldWRvX2NsaWVudC5jIGIvcHNldWRvX2NsaWVudC5jCmluZGV4IGEw M2Q2YjEuLmM0YjFjYTQgMTAwNjQ0Ci0tLSBhL3BzZXVkb19jbGllbnQuYworKysgYi9wc2V1ZG9f Y2xpZW50LmMKQEAgLTE1ODcsMTAgKzE1ODcsMTYgQEAgaW50IHBzZXVkb19jbGllbnRfaWdub3Jl X3BhdGhfY2hyb290KGNvbnN0IGNoYXIgKnBhdGgsIGludCBpZ25vcmVfY2hyb290KSB7CiAJCWNo YXIgKm5leHQgPSBzdHJjaHIocCwgJywnKTsKIAkJaWYgKG5leHQpCiAJCQkqbmV4dCsrID0gJ1ww JzsKLQkJaWYgKCpwICYmICFzdHJuY21wKHBhdGgsIHAsIHN0cmxlbihwKSkpIHsKLQkJCXBzZXVk b19kZWJ1ZyhQREJHRl9QQVRIIHwgUERCR0ZfVkVSQk9TRSwgImlnbm9yaW5nIHBhdGg6ICclcydc biIsIHBhdGgpOwotCQkJcmV0ID0gMTsKLQkJCWJyZWFrOworICAgICAgICB1bnNpZ25lZCBpbnQg bGVuID0gc3RybGVuKHApOworCQlpZiAoKnAgJiYgIXN0cm5jbXAocGF0aCwgcCxsZW4pKSB7Cisg ICAgICAgICAgICAvKldlIHNob3VkIGF2b2lkIG1hdGNoaW5nIHByZWZpeGVzIGlmIHRoZSBkb24n dCB0ZXJtaW5hdGUgaW4gYSBkaXIqLworICAgICAgICAgICAgLyppZiBwYXR0ZXJuIGlzIC9wa2cg d2Ugc2hvdWxkIG1hdGNoIC9wa2cvdGVzdCBidXQgbm90IC9wa2dkYXRhICAgKi8KKyAgICAgICAg ICAgIGlmKCBwYXRoW2xlbl0gPT0gJy8nIHx8IHBhdGhbbGVuXSA9PSAnXDAnKQorICAgICAgICAg ICAgeworICAgICAgICAgICAgICAgIHBzZXVkb19kZWJ1ZyhQREJHRl9QQVRIIHwgUERCR0ZfVkVS Qk9TRSwgImlnbm9yaW5nIHBhdGg6ICclcydcbiIsIHBhdGgpOworICAgICAgICAgICAgICAgIHJl dCA9IDE7CisgICAgICAgICAgICAgICAgYnJlYWs7CisgICAgICAgICAgICB9CiAJCX0KIAkJcCA9 IG5leHQ7CiAJfQpkaWZmIC0tZ2l0IGEvdGVzdC90ZXN0LWlnbm9yZS5zaCBiL3Rlc3QvdGVzdC1p Z25vcmUuc2gKbmV3IGZpbGUgbW9kZSAxMDA3NTUKaW5kZXggMDAwMDAwMC4uYzQwZThiMgotLS0g L2Rldi9udWxsCisrKyBiL3Rlc3QvdGVzdC1pZ25vcmUuc2gKQEAgLTAsMCArMSwyNiBAQAorIyEg L2Jpbi9zaAorCisjIFRlc3QgaWdub3JlIHBhdHRlcm4KK3NldCAtZQorc2V0IC14CitURVNUX1BB VEg9JChwd2QpL3Rlc3QvdGVzdC1pZ25vcmUKKyN0ZXN0IHBzZXVkbyBPSworZnVuY3Rpb24gdGVz dF9maWxlX293bmVyKCkKK3sKKyAgICBybSAtcmYgJFRFU1RfUEFUSAorICAgIG1rZGlyIC1wICQo ZGlybmFtZSAiJDEiKQorICAgIHRvdWNoICIkMSIKKyAgICBGTj0kKHN0YXQgLWMgJXUgIiQxIikK KyAgICBpZiBbICIkPyIgLW5lIDAgLW8gIiRGTiIgIT0gIiQyIiBdOworICAgIHRoZW4KKyAgICAg ICAgZWNobyAiSWdub3JlIHBhdGggZXJyb3IiCisgICAgICAgIGV4aXQgLTEKKyAgICBmaQorfQor VVNFUl9JRD0kKFBTRVVET19ESVNBQkxFRD0xIGlkIC11KQorCisjdGVzdCBiYXNpYyBwYXRoCit0 ZXN0X2ZpbGVfb3duZXIgJFRFU1RfUEFUSCAwCisjaWdub3JlIFBLRyBzaG91bGQgbWF0Y2gsIFBL R0RBVEEgc2hvdWxkIG5vdCBtYXRjaAorUFNFVURPX0lHTk9SRV9QQVRIUz0kVEVTVF9QQVRIL3Br ZyB0ZXN0X2ZpbGVfb3duZXIgJFRFU1RfUEFUSC9wa2cvZmlsZSAkVVNFUl9JRAorUFNFVURPX0lH Tk9SRV9QQVRIUz0kVEVTVF9QQVRIL3BrZyB0ZXN0X2ZpbGVfb3duZXIgJFRFU1RfUEFUSC9wa2dk YXRhL2ZpbGUgMAotLSAKMi4zNi4xCgo=