Incremental build in Docker on macOS BigSur fails with errors of the kind reproduced below. The error does not occur when shutil.move is used instead of os.rename, please see https://stackoverflow.com/questions/42392600/oserror-errno-18-invalid-cross-device-link. ERROR: core-image-base-1.0-r0 do_deploy_source_date_epoch: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/home/pi/yocto-raspberry-pi/poky/meta/classes/sstate.bbclass', lineno: 770, function: sstate_task_postfunc 0766: 0767: omask = os.umask(0o002) 0768: if omask != 0o002: 0769: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0770: sstate_package(shared_state, d) 0771: os.umask(omask) 0772: 0773: sstateinst = d.getVar("SSTATE_INSTDIR") 0774: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/home/pi/yocto-raspberry-pi/poky/meta/classes/sstate.bbclass', lineno: 677, function: sstate_package 0673: if sharedworkdir in plain: 0674: pdir = plain.replace(sharedworkdir, sstatebuild) 0675: bb.utils.mkdirhier(plain) 0676: bb.utils.mkdirhier(pdir) *** 0677: os.rename(plain, pdir) 0678: 0679: d.setVar('SSTATE_BUILDDIR', sstatebuild) 0680: d.setVar('SSTATE_INSTDIR', sstatebuild) 0681: Exception: OSError: [Errno 18] Invalid cross-device link: '/home/pi/yocto-raspberry-pi/build/tmp/work/raspberrypi0_wifi-poky-linux-gnueabi/core-image-base/1.0-r0/deploy-source-date-epoch' -> '/home/pi/yocto-raspberry-pi/build/tmp/work/raspberrypi0_wifi-poky-linux-gnueabi/core-image-base/1.0-r0/sstate-build-deploy_source_date_epoch//deploy-source-date-epoch' ERROR: Logfile of failure stored in: /home/pi/yocto-raspberry-pi/build/tmp/work/raspberrypi0_wifi-poky-linux-gnueabi/core-image-base/1.0-r0/temp/log.do_deploy_source_date_epoch.77 ERROR: Task (/home/pi/yocto-raspberry-pi/poky/meta/recipes-core/images/core-image-base.bb:do_deploy_source_date_epoch) failed with exit code '1' NOTE: Tasks Summary: Attempted 4634 tasks of which 4633 didn't need to be rerun and 1 failed.
Created attachment 4776 [details] Use shutil.move instead of os.rename
The rename is faster but we could trap the error and fall-back to the move if the host/filesystem has an error. Can you send a patch?
According to documentation, shutil.move falls back to os.rename when source and destination are on the same filesystem, but trapping error with os.rename and trying shutil.move looks safer. I'll attach a patch.
Created attachment 4777 [details] Use shutil.move when os.rename fails
Thanks Devendra, Are you able to send he patch to the oe-core email list, like this: git send-email -M -1 --to="openembedded-core@lists.openembedded.org" See: https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Sending_using_git-send-email
Thanks, Randy. I've submitted the patch via git send-email.
Hello Devendra, I didn't see the email. Did I miss it? Did it get merged? ../Randy
Sorry, looks like I didn't have an smtp server setup and the mail didn't go out. Sent it out again, and confirmed it got sent at https://lists.openembedded.org/g/openembedded-core/topic/patch_use_shutil_move_when/81698791.
Patches have been integrated.