15498 2024-05-28 04:36:40 +0000 random staging issues since UNPACKDIR introduction 2024-08-06 08:34:43 +0000 1 1 7 Build System, Metadata & Runtime OE-Core oe-core other unspecified x86 Multiple RESOLVED FIXED Medium+ major 5.1 M3 1 kweihmann richard.purdie tgamblin --- --- Don't know oldest_to_newest 99096 0 kweihmann 2024-05-28 04:36:40 +0000 Since the changes related to `UNPACKDIR` I see random hard/softlink issues with staging. E.g. File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:extend_recipe_sysroot(d) 0003: File: '/opt/build/sources/poky/meta/classes-global/staging.bbclass', lineno: 539, function: extend_recipe_sysroot 0535: continue 0536: 0537: msg_adding.append(c) 0538: *** 0539: os.symlink(c + "." + taskhash, depdir + "/" + c) 0540: 0541: manifest, d2 = oe.sstatesig.find_sstate_manifest(c, setscenedeps[dep][2], "populate_sysroot", d, multilibs) 0542: if d2 is not d: 0543: # If we don't do this, the recipe sysroot will be placed in the wrong WORKDIR for multilibs Exception: FileNotFoundError: [Errno 2] No such file or directory: 'python3-build-native.a02f3a216f80e615a959ea4fb1273cebb0e26a3228154905281ee93bb45da422' -> '/opt/build/build/tmp/work/x86_64-linux/python3-shellexeclist-native/1.10.2/recipe-sysroot-native/installeddeps/python3-build-native' ERROR: Logfile of failure stored in: /opt/build/build/tmp/work/x86_64-linux/python3-shellexeclist-native/1.10.2/temp/log.do_prepare_recipe_sysroot.104702 NOTE: recipe python3-shellexeclist-native-1.10.2-r0: task do_prepare_recipe_sysroot: Failed ERROR: python3-shellexeclist-native-1.10.2-r0 do_unpack: CalledProcessError(1, ['rm', '-rf', '/opt/build/build/tmp/work/x86_64-linux/python3-shellexeclist-native/1.10.2']) NOTE: recipe python3-shellexeclist-native-1.10.2-r0: task do_unpack: Failed The recipes in question are random, but all of them so far are -native recipes. With a local patch that retries the symlinking with a little backoff time everything works out much more nicely. The same has been seen with this code line https://git.yoctoproject.org/poky/tree/meta/classes-global/staging.bbclass#n165 In before (the UNPACKDIR change) the same code base was working flawlessly. Seen locally and in Github cloud 99108 1 richard.purdie 2024-05-31 16:26:31 +0000 Looking at this, the rm -rf in do_unpack is removing WORKDIR. Can you look at the python3-shellexeclist-native recipe and see what S is set to please? I've like to understand why it is removing WORKDIR. We should be erroring for S = WORKDIR now so that shouldn't happen. 99109 2 kweihmann 2024-05-31 17:16:13 +0000 S = "${UNPACKDIR}/git" that's what set in the recipe. The thing I'm worried about is that seem to affect random recipes, even if they all share the same S pattern 99113 3 kweihmann 2024-06-01 08:10:36 +0000 OK I think I figured it out. For pre-styhead the layer in question contained a weak fallback definition of UNPACKDIR. Set to UNPACKDIR ??= WORKDIR. Just send a patch to error out on this. Would it be possible to define UNPACKDIR ?= ... in base? Like that ??= as a fallback definition would not override the default, this is btw what I learned today about ??= behavior 99116 4 richard.purdie 2024-06-03 14:22:00 +0000 https://lists.openembedded.org/g/openembedded-core/topic/106405273#msg200061 I can see arguments both ways on this one :/. Thanks for the sanity test patch though, I'll queue that and get that in. 99495 5 richard.purdie 2024-08-06 08:34:43 +0000 Since https://git.yoctoproject.org/poky/commit/?id=3bb4c6bd18a088f20ee343cf7c96aabe73fa8ca0 has merged, we shouldn't run into this issue any more so marking as resolved.