During the "Publishing Artifacts" step of the 2.2 rc4 build in nightly-arm64 a bunch of nativesdk deb packages apparently already existed at the destination which shouldn't be possible: --------- snip --------- cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-it_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-de_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-vi_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-doc_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-liblzma5_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-dbg_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-pl_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-cs_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-dev_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-fr_5.2.2-r0.0_amd64.deb': File exists --------- snip --------- Link to the log: https://autobuilder.yoctoproject.org/main/builders/nightly-arm64/builds/586/steps/Publishing%20Artifacts/logs/stdio The previous build was interrupted which may have something to do with it, but still that shouldn't have caused this.
I have a suspicion that multiple builds write this same set of artefacts. If one writes them, then another, one will simply overwrite the other. If two write them at the same time, I suspect one set of cp commmands will error as the file didn't exist when it started the operation but does exist part way through. Some simple tests with some cp commands running in parallel will likely confirm how cp behaves in these cases. A -n option to the cp commands on the autobuilder will likely expose the full extent of the problem (but will make it fail badly).
(In reply to comment #1) > I have a suspicion that multiple builds write this same set of artefacts. If > one writes them, then another, one will simply overwrite the other. If two > write them at the same time, I suspect one set of cp commmands will error as > the file didn't exist when it started the operation but does exist part way > through. I suspect you're right. All of the architecture builders (nightly-arm, nightly-arm64, nightly-mips, nightly-mips64, nightly-ppc, nightly-x86-64, nightly-x86) and nightly-world publish the package artefacts for each package-manager backend, if two of those happen to be scheduled to PublishArtifacts at the same time we could easily run into this issue.
We haven't seen this for a while, but let's push to M4. We understand what's occurring but I don't have a clean solution yet.
We haven't seen this for a while. As we know what the issue is we'll aim to resolve it as part of bug #10437.