15696 2024-12-19 14:57:38 +0000 AB_INT: resulttool: File exists 2026-03-12 23:00:23 +0000 1 1 10 QA/Testing Tooling tooling unspecified x86 Multiple RESOLVED FIXED AB-INT Medium normal 6.0 1 mathieu.dubois-briand yoann.congal richard.purdie tim.orling yoann.congal --- --- Done (doc changes complete) oldest_to_newest 100455 0 mathieu.dubois-briand 2024-12-19 14:57:38 +0000 Probable AB-INT We had two builds with a similar error: 277 INFO: Creating tag mathieu/master-next/76713-gd2218d353b5d7becad9610f997c57b8099ea3753/0 278 Traceback (most recent call last): 279 File "/srv/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 83, in <module> 280 sys.exit(main()) 281 ^^^^^^ 282 File "/srv/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 77, in main 283 ret = args.func(args, logger) 284 ^^^^^^^^^^^^^^^^^^^^^^^ 285 File "/srv/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/store.py", line 89, in store 286 shutil.copytree(tempdir, logdir) 287 File "/usr/lib64/python3.12/shutil.py", line 600, in copytree 288 return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks, 289 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 290 File "/usr/lib64/python3.12/shutil.py", line 498, in _copytree 291 os.makedirs(dst, exist_ok=dirs_exist_ok) 292 File "<frozen os>", line 225, in makedirs 293 FileExistsError: [Errno 17] File exists: '/srv/autobuilder/valkyrie.yocto.io/pub/non-release/20241217-82/testresults/../../testresult-logarchives/mathieu/master-next/76713-gd2218d353b5d7becad9610f997c57b8099ea3753/0' 294 Traceback (most recent call last): 295 File "/srv/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 281, in <module> 296 send_qa_email() 297 File "/srv/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 191, in send_qa_email 298 subprocess.check_call([resulttool, "store", "--revision", revision, "-l", args.results_dir + "/../../testresult-logarchives" , args.results_dir, tempdir]) 299 File "/usr/lib64/python3.12/subprocess.py", line 413, in check_call 300 raise CalledProcessError(retcode, cmd) 301 subprocess.CalledProcessError: Command '['/srv/pokybuild/yocto-worker/a-full/build/scripts/resulttool', 'store', '--revision', 'd2218d353b5d7becad9610f997c57b8099ea3753', '-l', '/srv/autobuilder/valkyrie.yocto.io/pub/non-release/20241217-82/testresults/../../testresult-logarchives', '/srv/autobuilder/valkyrie.yocto.io/pub/non-release/20241217-82/testresults', '/tmp/sendqaem 301 ail.wzqr2sik']' returned non-zero exit status 1. https://valkyrie.yoctoproject.org/#/builders/29/builds/643/steps/29/logs/stdio https://valkyrie.yoctoproject.org/#/builders/29/builds/661/steps/29/logs/stdio 100459 1 mathieu.dubois-briand 2024-12-19 15:18:15 +0000 a-full ubuntu2204-vk-2 mathieu/master-next completed at 2024-12-14T10:21:05Z https://valkyrie.yoctoproject.org/#/builders/29/builds/643/steps/29/logs/stdio a-full fedora40-vk-3 mathieu/master-next completed at 2024-12-17T19:59:03Z https://valkyrie.yoctoproject.org/#/builders/29/builds/661/steps/29/logs/stdio 100495 2 mathieu.dubois-briand 2024-12-24 09:53:45 +0000 a-full ubuntu2204-vk-3 mathieu/master-next completed at 2024-12-20T12:04:25Z https://valkyrie.yoctoproject.org/#/builders/29/builds/685/steps/29/logs/stdio 101223 3 mathieu.dubois-briand 2025-02-25 12:47:28 +0000 a-full fedora41-vk-1 mathieu/master-next completed at 2025-02-21T20:39:34Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1064/steps/29/logs/stdio 101267 4 mathieu.dubois-briand 2025-03-03 16:06:44 +0000 a-full opensuse156-vk-1 mathieu/master-next completed at 2025-02-27T21:00:33Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1097/steps/31/logs/stdio 101649 5 mathieu.dubois-briand 2025-04-15 09:20:24 +0000 a-full debian11-vk-3 mathieu/master-next completed at 2025-04-13T21:01:56Z https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1391/steps/31/logs/stdio 101805 6 randy.macleod 2025-05-01 14:07:06 +0000 Bulk move of all unassigned 5.2 medium importance bugs to 5.3. 102443 7 mathieu.dubois-briand 2025-07-10 12:48:20 +0000 a-full alma8-vk-2 mathieu/master-next completed at 2025-07-09 18:09:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1986/steps/33/logs/stdio 102497 8 mathieu.dubois-briand 2025-07-23 11:38:49 +0000 a-full rocky9-vk-3 mathieu/master-next completed at 2025-07-22 13:17:31+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2058/steps/31/logs/stdio 102663 9 mathieu.dubois-briand 2025-08-22 09:40:39 +0000 a-quick debian12-vk-8 kraj/clang-merge-2 completed at 2025-04-20 06:58:02+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/62/builds/79/steps/31/logs/stdio a-quick debian11-vk-1 kraj/clang-merge-2 completed at 2025-05-02 10:53:37+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/62/builds/105/steps/31/logs/stdio 102851 10 mathieu.dubois-briand 2025-09-30 08:46:21 +0000 a-full fedora41-vk-1 mathieu/master-next completed at 2025-09-26 18:04:01+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2446/steps/31/logs/stdio 103298 11 mathieu.dubois-briand 2025-11-14 10:11:02 +0000 a-full ubuntu2404-vk-1 master completed at 2025-11-14 07:26:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2721/steps/37/logs/stdio 103304 12 mathieu.dubois-briand 2025-11-17 10:28:01 +0000 a-full fedora42-vk-1 master completed at 2025-11-15 11:11:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2724/steps/37/logs/stdio a-full debian12-vk-1 master completed at 2025-11-16 05:50:23+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2728/steps/37/logs/stdio 103328 13 mathieu.dubois-briand 2025-11-19 16:08:22 +0000 a-full ubuntu2404-vk-2 master completed at 2025-11-18 10:34:55+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2737/steps/37/logs/stdio 103431 14 mathieu.dubois-briand 2025-11-28 08:06:07 +0000 a-full fedora39-vk-1 master completed at 2025-11-28 02:23:24+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2797/steps/37/logs/stdio 103453 15 mathieu.dubois-briand 2025-12-02 09:23:52 +0000 a-full debian12-vk-1 master completed at 2025-11-29 04:13:24+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2803/steps/37/logs/stdio a-full fedora42-vk-1 master completed at 2025-11-30 04:11:30+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2806/steps/37/logs/stdio a-full rocky8-vk-1 master completed at 2025-12-01 04:09:28+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2807/steps/39/logs/stdio a-full rocky9-vk-2 master completed at 2025-12-02 04:54:34+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2810/steps/37/logs/stdio 103487 16 mathieu.dubois-briand 2025-12-04 09:32:42 +0000 a-full ubuntu2204-vk-1 master completed at 2025-12-04 00:23:12+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2818/steps/37/logs/stdio 103640 17 mathieu.dubois-briand 2025-12-23 10:52:14 +0000 a-full ubuntu2204-vk-1 mathieu/master-next completed at 2025-12-23 00:26:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2913/steps/37/logs/stdio 103650 18 mathieu.dubois-briand 2025-12-29 09:11:49 +0000 a-full stream9-vk-1 mathieu/master-next completed at 2025-12-25 20:49:44+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2938/steps/37/logs/stdio a-full alma9-vk-2 mathieu/master-next completed at 2025-12-26 15:19:26+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2946/steps/37/logs/stdio 103667 19 mathieu.dubois-briand 2025-12-30 13:42:26 +0000 So the issue here is in resulttool, when archiving logs: - we create a local tag, making sure it does not clash with a tag already existing on the remote. - we copy the logs in shared NFS, folder name does correspond to this tag, e.g. /srv/autobuilder/valkyrie.yocto.io/pub/non-release/20251129-12/testresults/../../testresult-logarchives/master/65319-gac7327583aef83fd400190506f3147623de38b68/0 - we push the tag But, if for any reason the tag is not pushed, the same tag name will be reused on the next build of the same oe-core version. And if the logs were already copied, the log copy of this next build will fail. I believe the solution here is to make sure to push the tag before copying any log file. Also, for some branches like mine, the tag is never pushed: rebuilding the same oe-core version on such a branch will always trigger this failure. I believe copying logs while not pushing the tag does not make sense, so we should just make sure to only copy the logs when we do push tags. I'm going to send a few patches trying to fix this. 103711 20 yoann.congal 2026-01-06 11:52:02 +0000 Also on whinlatter-nut: https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2997#bb-step-37 103712 21 yoann.congal 2026-01-06 11:57:28 +0000 Patches sent by Mathieu: *[PATCH 0/2] oeqa/gitarchive: Push tag before copying log files https://lists.openembedded.org/g/openembedded-core/message/228687 * [PATCH 1/2] oeqa/gitarchive: Fix git push URL parameter https://lists.openembedded.org/g/openembedded-core/message/228688 * [PATCH 2/2] oeqa/gitarchive: Push tag before copying log files https://lists.openembedded.org/g/openembedded-core/message/228689 103713 22 mathieu.dubois-briand 2026-01-06 12:24:52 +0000 (In reply to Yoann Congal from comment #21) > Patches sent by Mathieu: > *[PATCH 0/2] oeqa/gitarchive: Push tag before copying log files > https://lists.openembedded.org/g/openembedded-core/message/228687 > * [PATCH 1/2] oeqa/gitarchive: Fix git push URL parameter > https://lists.openembedded.org/g/openembedded-core/message/228688 > * [PATCH 2/2] oeqa/gitarchive: Push tag before copying log files > https://lists.openembedded.org/g/openembedded-core/message/228689 Thanks, I forgot to list these here. Also, we have one on the autobuilder-helper side: [PATCH yocto-autobuilder-helper] send_qa_email: Only copy logs when we are pushing tags https://lists.yoctoproject.org/g/yocto-patches/message/2885 103777 23 mathieu.dubois-briand 2026-01-09 09:12:20 +0000 a-full ubuntu2404-vk-3 whinlatter-next&stable/whinlatter-nut&stable/2.16-nut completed at 2026-01-08 10:30:05+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/3010/steps/37/logs/stdio 103875 24 yoann.congal 2026-01-13 23:02:52 +0000 Stable build hit: whinlatter a-full fedora40-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/2995 103954 25 yoann.congal 2026-01-17 22:46:31 +0000 OE-Core commits merged on master: oeqa/gitarchive: Fix git push URL parameter https://git.openembedded.org/openembedded-core/commit/?id=024f08629feeec8198d1e489633e475959754cfe oeqa/gitarchive: Push tag before copying log files https://git.openembedded.org/openembedded-core/commit/?id=7d869c9c5aaeeda9fa476bfe6b05ded6e225379d scripts/oe-git-archive: Ensure new push parameter is specified https://git.openembedded.org/openembedded-core/commit/?id=983cb2171e53564bc9dd188136439f3e2ad9e188 (but not the yocto-autobuilder-helper part, yet) 103958 26 yoann.congal 2026-01-18 22:33:06 +0000 a-full fedora40-vk-3 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/3082 Note: This is a weird build: yocto-autobuilder-helper stayed on default (master?) but layers/bitbake where on stable/whinlatter-nut 103959 27 yoann.congal 2026-01-19 07:50:27 +0000 whinlatter a-full ubuntu2204-vk-4 https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/3083 103984 28 yoann.congal 2026-01-21 09:30:20 +0000 Patch merged in yocto-autobuilder-helper/master: send_qa_email: Only copy logs when we are pushing tags https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=904d5722ced282b58474e3b605ce87897994f4f7 Also, maybe related: scripts/send_qa_email: Show git processing commands used in logs - yocto-autobuilder-helper - Decoupled release build scripts https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=612fb877fef9885b7793117556a5152e312de4da scripts/send_qa_email.py: Add missing log parameter - yocto-autobuilder-helper - Decoupled release build scripts https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=e47be6614f777dd8f670618250645c2265c391d0 scripts/send_qa_email.py: Fix master branch comparision for regression report - yocto-autobuilder-helper - Decoupled release build scripts https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=ec17abb0b2d759b800cefec4fcac61cee74dbefc 104057 29 yoann.congal 2026-01-23 13:21:56 +0000 whinlatter a-full rocky8-vk-1 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/29/builds/3112 104295 30 mathieu.dubois-briand 2026-02-13 12:46:29 +0000 Last commit was merged in autobuilder-helper master branch: https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=904d5722ced282b58474e3b605ce87897994f4f7 @Yoann: What about stable branches? Is the issue still present? 104298 31 yoann.congal 2026-02-13 12:56:34 +0000 (In reply to Mathieu Dubois-Briand from comment #30) > Last commit was merged in autobuilder-helper master branch: > https://git.yoctoproject.org/yocto-autobuilder-helper/commit/ > ?id=904d5722ced282b58474e3b605ce87897994f4f7 > > @Yoann: What about stable branches? Is the issue still present? I've not seen this one recently but not everything is merged. You can attribute this bug to me and I'll close it once everything is merged on their respective branches. For example: This is not merged: send_qa_email: Only copy logs when we are pushing tags https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?h=contrib/ycongal/whinlatter-nut&id=b72ff5eb03a2296287b39c82443ef048d6a6db4b 104619 32 richard.purdie 2026-03-12 22:50:02 +0000 Yoann to close when everything backported 104620 33 yoann.congal 2026-03-12 23:00:23 +0000 openembedded-core/whinlatter: * oeqa/gitarchive: Fix git push URL parameter * https://git.openembedded.org/openembedded-core/commit/?h=whinlatter&id=97a510e319286cf5cd86d3a905bb1f8311d3db7e * oeqa/gitarchive: Push tag before copying log files * https://git.openembedded.org/openembedded-core/commit/?h=whinlatter&id=412c9c6388d449e9a455d1b1961fe17d81b8528f * scripts/oe-git-archive: Ensure new push parameter is specified * https://git.openembedded.org/openembedded-core/commit/?h=whinlatter&id=003b6951b58e732dd5e780c3624fd1966a3db5c5 yocto-autobuilder-helper/whinlatter: * send_qa_email: Only copy logs when we are pushing tags * https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?h=whinlatter&id=be29ed1e75ef15b9bf395c576a3a0f519d2b2013 Everything merged, closing!