10448 2016-10-18 07:27:59 +0000 Package files already existed at Publishing Artifacts step 2017-03-30 15:11:46 +0000 1 1 5 Infrastructure AutoBuilder autobuilder unspecified x86 x86_64 RESOLVED WONTFIX Medium+ major 2.3 M4 1 bluelightning joshuagloe infras.ab.watcher Infras.watcher pidge richard.purdie --- --- No (bug/feature does not impact docs) oldest_to_newest 67320 0 bluelightning 2016-10-18 07:27:59 +0000 During the "Publishing Artifacts" step of the 2.2 rc4 build in nightly-arm64 a bunch of nativesdk deb packages apparently already existed at the destination which shouldn't be possible: --------- snip --------- cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-it_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-de_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-vi_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-doc_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-liblzma5_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-dbg_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-pl_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-cs_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-dev_5.2.2-r0.0_amd64.deb': File exists cp: cannot create regular file '/srv/www/vhosts/autobuilder.yoctoproject.org/pub/releases/yocto-2.2.rc4/deb/x86_64-nativesdk/nativesdk-xz-locale-fr_5.2.2-r0.0_amd64.deb': File exists --------- snip --------- Link to the log: https://autobuilder.yoctoproject.org/main/builders/nightly-arm64/builds/586/steps/Publishing%20Artifacts/logs/stdio The previous build was interrupted which may have something to do with it, but still that shouldn't have caused this. 67440 1 richard.purdie 2016-10-21 07:07:42 +0000 I have a suspicion that multiple builds write this same set of artefacts. If one writes them, then another, one will simply overwrite the other. If two write them at the same time, I suspect one set of cp commmands will error as the file didn't exist when it started the operation but does exist part way through. Some simple tests with some cp commands running in parallel will likely confirm how cp behaves in these cases. A -n option to the cp commands on the autobuilder will likely expose the full extent of the problem (but will make it fail badly). 68965 2 joshuagloe 2016-12-08 22:44:24 +0000 (In reply to comment #1) > I have a suspicion that multiple builds write this same set of artefacts. If > one writes them, then another, one will simply overwrite the other. If two > write them at the same time, I suspect one set of cp commmands will error as > the file didn't exist when it started the operation but does exist part way > through. I suspect you're right. All of the architecture builders (nightly-arm, nightly-arm64, nightly-mips, nightly-mips64, nightly-ppc, nightly-x86-64, nightly-x86) and nightly-world publish the package artefacts for each package-manager backend, if two of those happen to be scheduled to PublishArtifacts at the same time we could easily run into this issue. 71206 3 joshuagloe 2017-03-13 14:38:38 +0000 We haven't seen this for a while, but let's push to M4. We understand what's occurring but I don't have a clean solution yet. 71912 4 joshuagloe 2017-03-30 15:11:46 +0000 We haven't seen this for a while. As we know what the issue is we'll aim to resolve it as part of bug #10437.