Created attachment 4659 [details]
Log master

Under certain (unknown) conditions cache files from python3 or python3-site-packages get lost while populating or stating, leading to stack traces like
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 551, function: extend_recipe_sysroot
     0547:                    dest = newmanifest[l]
     0548:                    if l.endswith("/"):
     0549:                        staging_copydir(l, targetdir, dest, seendirs)
     0550:                        continue
 *** 0551:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
     0552:
     0553:    bb.note("Installed into sysroot: %s" % str(msg_adding))
     0554:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
     0555:
File: '/build/poky/meta/classes/staging.bbclass', lineno: 152, function: staging_copyfile
     0148:        os.symlink(linkto, dest)
     0149:        #bb.warn(c)
     0150:    else:
     0151:        try:
 *** 0152:            os.link(c, dest)
     0153:        except OSError as err:
     0154:            if err.errno == errno.EXDEV:
     0155:                bb.utils.copyfile(c, dest)
     0156:            else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/build/poky/build/tmp/sysroots-components/x86_64/python3-msgcheck-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc' -> '/build/poky/build/tmp/work/qemux86_64-mine-linux/core-image-minimal-mine/1.0-r0/recipe-sysroot-native/usr/lib/python3.7/site-packages/msgcheck/__pycache__/__init__.cpython-37.pyc'

This affects only packages with python cache files and seems to be most likely to happen on very low-spec-systems with a lot of python-native packages to be injected into a workspace.

Very often this happens in a Github Actions VM using a Ubuntu 18.04 based container, but happens occasionally even on more powerful systems.

Specs of the mostly used systems
2-core CPU
7 GB of RAM memory
14 GB of SSD disk space
Ubuntu 18.04 based docker container running on Ubuntu 18.04

The actual missing cache file is pretty random (see the different attachments for more info) and this issue can be reproduced on all branches since thud (TBH branches before I didn't test).

Current workaround for me is to delete all cache files from every package before do_package (for python3-native also after preprocess of do_package), which fixes the symptoms but doesn't really cure the underlying issue