Bug 15098

Summary: AB-INT-NET: network failure when fetching repositories
Product: [Infrastructure] AutoBuilder Reporter: Alexandre Belloni <alexandre.belloni>
Component: autobuilderAssignee: Unassigned <unassigned>
Status: RESOLVED WORKSFORME QA Contact:
Severity: normal    
Priority: Medium CC: alexandre.belloni, infras.ab.watcher, Infras.watcher, jon.mason, luca.ceresoli, mathieu.dubois-briand, pidge, randy.macleod, ross.burton
Version: unspecified   
Target Milestone: 5.3   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT AB-INT-NET
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Alexandre Belloni 2023-04-06 14:29:59 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5130/steps/4/logs/stdio

Checking for stash at: /home/pokybuild/git/mirror/bitbake
Cloning from stash to /home/pokybuild/tmp/shared-repo-temp-uqkmilxz/bitbake...
Cloning into '/home/pokybuild/tmp/shared-repo-temp-uqkmilxz/bitbake'...
Updating from origin...
kex_exchange_identification: Connection closed by remote host
Connection closed by 104.236.212.160 port 22
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/prepare-shared-repos", line 47, in <module>
    utils.fetchgitrepo(tempdir, repo, repos[repo], stashdir)
  File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/utils.py", line 259, in fetchgitrepo
    subprocess.check_call(["git", "fetch", "origin"] + fetchopt, cwd=sharedrepo)
  File "/usr/lib64/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'fetch', 'origin']' returned non-zero exit status 128.
program finished with exit code 1
elapsedTime=2.538095
Comment 2 Alexandre Belloni 2023-06-09 17:49:40 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/5212/steps/11/logs/stdio

oe-selftest-debian debian11-ty-1

ERROR: test_that_the_tarball_contents_does_not_leak_info (bb.tests.fetch.CleanTarballTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/tests/fetch.py", line 669, in test_that_the_tarball_contents_does_not_leak_info
    fetcher.download()
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/fetch2/__init__.py", line 1799, in download
    raise FetchError("Unable to fetch URL from any source.", u)
bb.fetch2.FetchError: Fetcher failure for URL: 'git://git.openembedded.org/bitbake;protocol=https'. Unable to fetch URL from any source.
Stdout:
URL: git://git.openembedded.org/bitbake;protocol=https does not set any branch parameter. The future default branch used by tools and repositories is uncertain and we will therefore soon require this is set in all git urls.
Cloning into bare repository '/tmp/bitbake-fetch-7f7dlp9p/download/git2/git.openembedded.org.bitbake'...
error: Unable to get pack file https://git.openembedded.org/bitbake/objects/pack/pack-29d020480cbf7050bc5600cb67db896589ea0cf3.pack
GnuTLS recv error (-9): Error decoding the received TLS packet.
error: Unable to find b801e442b0163a73fe4624e3a7dec4bf7bb63f3e under https://git.openembedded.org/bitbake
Cannot obtain needed object b801e442b0163a73fe4624e3a7dec4bf7bb63f3e
while processing commit 4dbecf6059e495246267b09d0f43086d51e6df2c.
error: fetch failed.
Failed to fetch URL git://git.openembedded.org/bitbake;protocol=https, attempting MIRRORS if available
Fetcher failure: Fetch command export PSEUDO_DISABLED=1; LANG=C git -c gc.autoDetach=false -c core.pager=cat clone --bare --mirror https://git.openembedded.org/bitbake /tmp/bitbake-fetch-7f7dlp9p/download/git2/git.openembedded.org.bitbake --progress failed with exit code 128, no output
Comment 3 Alexandre Belloni 2023-07-18 17:48:49 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5352/steps/11/logs/stdio

oe-selftest-fedora fedora38-ty-1
Comment 4 Alexandre Belloni 2023-07-20 10:00:12 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5492/steps/11/logs/stdio

oe-selftest-fedora fedora37-ty-3
Comment 6 Alexandre Belloni 2023-07-20 11:43:51 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5280/steps/11/logs/stdio

oe-selftest-fedora fedora38-ty-2
Comment 11 Luca Ceresoli 2023-08-21 15:58:05 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/1944/steps/12/logs/stdio

oe-selftest-armhost on ubuntu1804-arm-1
Comment 13 Alexandre Belloni 2023-09-07 21:25:27 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/5632/steps/11/logs/stdio

oe-selftest-debian debian11-ty-1
Comment 14 Alexandre Belloni 2023-09-21 16:05:15 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/2058/steps/11/logs/stdio

oe-selftest-armhost ubuntu2204-arm-1
Comment 16 Randy MacLeod 2023-10-30 15:37:31 UTC 
Build move to 5.0 -- ../Randy
Comment 17 Alexandre Belloni 2024-04-04 21:23:52 UTC 
bb.fetch2.FetchError: Fetcher failure for URL: 'git://git.openembedded.org/bitbake;branch=master;protocol=https'. Unable to fetch URL from any source.

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6605/steps/11/logs/stdio
Comment 18 Alexandre Belloni 2024-04-05 16:25:35 UTC 
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6495/steps/14/logs/stdio

oe-selftest-centos alma9-ty-2
Comment 19 Randy MacLeod 2024-04-25 01:23:51 UTC 
Build move to 5.1
Comment 24 Ross Burton 2024-10-18 10:29:06 UTC 
Our CI is seeing the same failures: a fetch of meta-openembedded that doesn't have the 'styhead' branch for example, or commits that are on master apparently not being present.

Obviously this is in containers inside k8s inside CI, so I can't just poke at the repo...
Comment 25 Ross Burton 2024-10-18 10:36:37 UTC 
Hunch: the European mirror of git.openembedded.org is struggling now that it's being hit a lot more by valkyrie.  Our CI is physically in Europe too.
Comment 26 Jon Mason 2024-10-18 13:09:08 UTC 
Not sure if it is related, but I'm seeing missing commits in my gitlab ci runners.
...
2024-10-18 06:18:39 - ERROR    - Branch "master" in repository "poky" does not contain commit "066108377dbbfb8705b7bbaf6effcfbe23adeec7"
...

See https://gitlab.com/jonmason00/meta-arm/-/jobs/8120420587

I am able to rerun it and have it work.  Also, I am seeing it on different runners
Comment 27 Jon Mason 2024-10-21 12:53:24 UTC 
Seeing another git server issue in my nightly CI
https://gitlab.com/jonmason00/meta-arm/-/jobs/8129616662

fatal: unable to access 'https://git.openembedded.org/meta-openembedded/': Failed to connect to git.openembedded.org port 443 after 131176 ms: Couldn't connect to server
Comment 29 Mathieu Dubois-Briand 2024-11-05 11:07:23 UTC 
oe-selftest-armhost ubuntu2404-vk-arm1
https://valkyrie.yoctoproject.org/#/builders/23/builds/376/steps/11/logs/stdio

oe-selftest rocky9-vk-2
https://valkyrie.yoctoproject.org/#/builders/69/builds/33/steps/11/logs/stdio
Comment 30 Mathieu Dubois-Briand 2024-11-06 17:08:19 UTC 
oe-selftest-fedora fedora40-vk-1 kirkstone
https://valkyrie.yoctoproject.org/#/builders/48/builds/320/steps/15/logs/stdio
Comment 31 Mathieu Dubois-Briand 2025-01-14 10:06:29 UTC 
oe-selftest-ubuntu ubuntu2410-vk-1 master-next completed at 2025-01-03T14:03:00Z
https://valkyrie.yoctoproject.org/#/builders/54/builds/690/steps/14/logs/stdio

oe-selftest-armhost ubuntu2204-vk-arm1 master completed at 2025-01-06T03:36:37Z
https://valkyrie.yoctoproject.org/#/builders/23/builds/786/steps/11/logs/stdio

oe-selftest-centos alma9-vk-2 master completed at 2025-01-06T02:14:28Z
https://valkyrie.yoctoproject.org/#/builders/76/builds/711/steps/11/logs/stdio

oe-selftest-debian debian12-vk-8 master completed at 2025-01-06T02:07:16Z
https://valkyrie.yoctoproject.org/#/builders/35/builds/716/steps/14/logs/stdio

qemux86 alma9-vk-2 master completed at 2025-01-06T01:38:56Z
https://valkyrie.yoctoproject.org/#/builders/30/builds/740/steps/14/logs/stdio

oe-selftest-ubuntu ubuntu2204-vk-2 master completed at 2025-01-06T02:30:06Z
https://valkyrie.yoctoproject.org/#/builders/54/builds/701/steps/14/logs/stdio

qemuarm64 fedora39-vk-2 master-next completed at 2025-01-06T13:22:46Z
https://valkyrie.yoctoproject.org/#/builders/36/builds/748/steps/14/logs/stdio

qemumips ubuntu2404-vk-2 master-next completed at 2025-01-06T13:02:44Z
https://valkyrie.yoctoproject.org/#/builders/39/builds/740/steps/14/logs/stdio

qemuppc debian11-vk-2 master-next completed at 2025-01-06T13:03:43Z
https://valkyrie.yoctoproject.org/#/builders/31/builds/743/steps/14/logs/stdio

qemux86-64 alma8-vk-2 master-next completed at 2025-01-06T13:14:30Z
https://valkyrie.yoctoproject.org/#/builders/68/builds/783/steps/15/logs/stdio

oe-selftest-fedora fedora39-vk-1 abelloni/master-next completed at 2025-01-08T20:34:09Z
https://valkyrie.yoctoproject.org/#/builders/48/builds/690/steps/14/logs/stdio

qemuarm64-armhost ubuntu2404-vk-arm2 master-next completed at 2025-01-09T00:50:55Z
https://valkyrie.yoctoproject.org/#/builders/8/builds/781/steps/16/logs/stdio

qemuarm alma8-vk-1 master-next completed at 2025-01-09T01:00:05Z
https://valkyrie.yoctoproject.org/#/builders/16/builds/771/steps/15/logs/stdio

qemuarm64 debian12-vk-9 master-next completed at 2025-01-09T00:51:23Z
https://valkyrie.yoctoproject.org/#/builders/36/builds/770/steps/14/logs/stdio

qemumips fedora40-vk-4 master-next completed at 2025-01-09T00:31:55Z
https://valkyrie.yoctoproject.org/#/builders/39/builds/762/steps/14/logs/stdio

qemuppc debian12-vk-9 master-next completed at 2025-01-09T00:34:55Z
https://valkyrie.yoctoproject.org/#/builders/31/builds/765/steps/14/logs/stdio

oe-selftest-debian debian11-vk-3 master-next completed at 2025-01-09T01:21:22Z
https://valkyrie.yoctoproject.org/#/builders/35/builds/740/steps/11/logs/stdio
Comment 32 Mathieu Dubois-Briand 2025-01-20 08:38:59 UTC 
oe-selftest-armhost ubuntu2404-vk-arm2 mathieu/master-next completed at 2025-01-15T15:27:05Z
https://valkyrie.yoctoproject.org/#/builders/23/builds/854/steps/11/logs/stdio

oe-selftest-fedora fedora40-vk-2 master completed at 2025-01-16T02:58:37Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/736/steps/14/logs/stdio

oe-selftest-armhost ubuntu2410-vk-arm1 mathieu/master-next completed at 2025-01-17T15:01:33Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/866/steps/14/logs/stdio
Comment 33 Mathieu Dubois-Briand 2025-01-27 13:39:08 UTC 
oe-selftest-debian debian12-vk-4 mathieu/master-next completed at 2025-01-26T17:36:52Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/857/steps/11/logs/stdio
Comment 34 Mathieu Dubois-Briand 2025-02-13 15:17:10 UTC 
oe-selftest-armhost ubuntu2004-vk-arm1 master-next completed at 2025-02-13T12:08:55Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1047/steps/11/logs/stdio

oe-selftest-centos alma8-vk-2 master-next completed at 2025-02-13T10:27:12Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/76/builds/961/steps/12/logs/stdio

oe-selftest-fedora fedora40-vk-4 master-next completed at 2025-02-13T10:39:19Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/921/steps/11/logs/stdio

oe-selftest-ubuntu ubuntu2204-vk-3 master-next completed at 2025-02-13T10:22:04Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/54/builds/963/steps/11/logs/stdio

oe-selftest-debian debian12-vk-4 master-next completed at 2025-02-13T10:20:48Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/973/steps/11/logs/stdio
Comment 35 Mathieu Dubois-Briand 2025-02-17 13:53:38 UTC 
oe-selftest-debian debian11-vk-1 kirkstone completed at 2025-02-11T14:22:41Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/961/steps/14/logs/stdio

build-appliance ubuntu2204-vk-2 master completed at 2025-02-15T01:23:39Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/1016/steps/11/logs/stdio

build-appliance alma9-vk-1 mathieu/master-next completed at 2025-02-17T09:41:05Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/1022/steps/11/logs/stdio
Comment 36 Mathieu Dubois-Briand 2025-02-25 12:47:01 UTC 
build-appliance debian12-vk-4 master completed at 2025-02-18T01:21:03Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/1028/steps/11/logs/stdio
Comment 37 Mathieu Dubois-Briand 2025-03-10 09:42:59 UTC 
build-appliance debian12-vk-7 mathieu/master-next completed at 2025-03-06T16:55:28Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/1164/steps/11/logs/stdio

build-appliance debian11-vk-2 master completed at 2025-03-07T11:51:52Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/78/builds/1170/steps/11/logs/stdio

oe-selftest-armhost ubuntu2004-vk-arm2 master-next completed at 2025-03-08T10:37:00Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/1193/steps/11/logs/stdio
Comment 38 Mathieu Dubois-Briand 2025-03-20 14:11:29 UTC 
qemuarm64-ltp ubuntu2004-vk-arm2 mathieu/master-next completed at 2025-03-19T09:35:16Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/98/builds/1148/steps/2/logs/stdio

qemuarm64 fedora39-vk-2 master-next completed at 2025-03-19T16:03:27Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/36/builds/1216/steps/14/logs/stdio
Comment 39 Randy MacLeod 2025-05-01 14:07:11 UTC 
Bulk move of all unassigned 5.2 medium importance bugs to 5.3.
Comment 40 Mathieu Dubois-Briand 2025-05-20 13:26:07 UTC 
a-full fedora39-vk-2 mathieu/master-next completed at 2025-05-19T00:11:05Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1612/steps/25/logs/stdio
a-full rocky9-vk-2 mathieu/master-next completed at 2025-05-19T17:18:44Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1616/steps/25/logs/stdio

a-full alma9-vk-2 mathieu/master-next completed at 2025-05-19T09:42:29Z
https://autobuilder.yoctoproject.org/valkyrie/#/builders/29/builds/1614/steps/25/logs/stdio
Comment 41 Randy MacLeod 2025-08-14 14:52:28 UTC 
Likely due to the infamous AI 2025 web spam issue and likely resolved now.