15948 2025-08-06 07:23:24 +0000 PREMIRROR does not work with git lfs 2026-08-20 12:53:06 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 5.0.15 x86 x86_64 IN PROGRESS REVIEW Medium+ normal 6.1 M1 1 Oliver.Feilner unassigned mathieu.dubois-briand poky.bs.watcher poky.watcher randy.macleod sivakumar.bs --- --- No (bug/feature does not impact docs) oldest_to_newest 102547 0 Oliver.Feilner 2025-08-06 07:23:24 +0000 When fetching a repo using git lfs from a MIRROR / PREMIRROR configured in the local.conf the fetch failed. PREMIRRORS = " git://.*\.xxx\.com.*/(yyyy|zzzz)/([^/]+)\.git git://dev.azure.com/project/Synchronization/_git/yyyy.\2;protocol=https;user=${ACCESSTOKEN} \n" PREMIRRORS =+ "${MY_MIRRORS}" MIRRORS =+ "${MY_MIRRORS}" ``` ERROR: prebuild-libs-8.30.6885.501-r0 do_fetch: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:base_do_fetch(d) 0003: File: '/home/myBuild/autobuild/../../poky/meta/classes-global/base.bbclass', lineno: 150, function: base_do_fetch 0146: return 0147: 0148: try: 0149: fetcher = bb.fetch2.Fetch(src_uri, d) *** 0150: fetcher.download() 0151: except bb.fetch2.BBFetchException as e: 0152: bb.fatal("Bitbake Fetcher Error: " + repr(e)) 0153:} 0154: File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1851, function: download 1847: done = True 1848: elif m.try_premirror(ud, self.d): 1849: logger.debug("Trying PREMIRRORS") 1850: mirrors = mirror_from_string(self.d.getVar('PREMIRRORS')) *** 1851: done = m.try_mirrors(self, ud, self.d, mirrors) 1852: if done: 1853: try: 1854: # early checksum verification so that if the checksum of the premirror 1855: # contents mismatch the fetcher can still try upstream and mirrors File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1648, function: try_mirrors 1644: def try_mirrors(self, fetch, urldata, d, mirrors, check=False): 1645: """ 1646: Try to use a mirror 1647: """ *** 1648: return bool(try_mirrors(fetch, d, urldata, mirrors, check)) 1649: 1650: def checkstatus(self, fetch, urldata, d): 1651: """ 1652: Check the status of a URL File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1163, function: try_mirrors 1159: 1160: uris, uds = build_mirroruris(origud, mirrors, ld) 1161: 1162: for index, uri in enumerate(uris): *** 1163: ret = try_mirror_url(fetch, origud, uds[index], ld, check) 1164: if ret: 1165: return ret 1166: return None 1167: File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/__init__.py', lineno: 1061, function: try_mirror_url 1057: return found 1058: return False 1059: 1060: if not verify_donestamp(ud, ld, origud) or ud.method.need_update(ud, ld): *** 1061: ud.method.download(ud, ld) 1062: if hasattr(ud.method,"build_mirror_data"): 1063: ud.method.build_mirror_data(ud, ld) 1064: 1065: if not ud.localpath or not os.path.exists(ud.localpath): File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/git.py', lineno: 494, function: download 0490: # on the bare clonedir, but that operation requires a working copy on some 0491: # releases of Git LFS. 0492: with tempfile.TemporaryDirectory(dir=d.getVar('DL_DIR')) as tmpdir: 0493: # Do the checkout. This implicitly involves a Git LFS fetch. *** 0494: Git.unpack(self, ud, tmpdir, d) 0495: 0496: # Scoop up a copy of any stuff that Git LFS downloaded. Merge them into 0497: # the bare clonedir. 0498: # File: '/home/myBuild/poky/bitbake/lib/bb/fetch2/git.py', lineno: 636, function: unpack 0632: destdir = ud.destdir = os.path.join(destdir, destsuffix) 0633: if os.path.exists(destdir): 0634: bb.utils.prunedir(destdir) 0635: if not ud.bareclone: *** 0636: ud.unpack_tracer.unpack("git", destdir) 0637: 0638: need_lfs = self._need_lfs(ud) 0639: 0640: if not need_lfs: Exception: AttributeError: 'FetchData' object has no attribute 'unpack_tracer' ERROR: Logfile of failure stored in: /home/myBuild/autobuild/build/tmp/work/armv7vet2hf-neon-vfpv4-yaskawa-linux-gnueabi/prebuild-libs/8.30.6885.501/temp/log.do_fetch.3715563 ERROR: Task (/home/myBuild/autobuild/../../meta-arp/recipes-arp/prebuild-libs/prebuild-libs_8.30.6885.501.bb:do_fetch) failed with exit code '1' ``` Without using a mirror download is called with an object from type Fetch. Which has the attribute unpack_tracer. In case of mirrors download is called with an object from the type FetchData. Which has no attribute unpack_tracer. The behavior could be fixed by adding the unpack_tracer also to the mirror ud From f8e3d48ea9cc609c0c9dd96be90a34cd52822e86 Mon Sep 17 00:00:00 2001 From: Oliver Feilner <oliver.feilner@yaskawa.eu> Date: Wed, 6 Aug 2025 09:16:13 +0200 Subject: [PATCH] Copy unpack_tracker also mirror uds --- bitbake/lib/bb/fetch2/__init__.py | 1 + 1 file changed, 1 insertion(+) diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index e698fde19c..aed5a28c69 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -1020,6 +1020,7 @@ def build_mirroruris(origud, mirrors, ld): except UnboundLocalError: pass continue + newud.unpack_tracer = ud.unpack_tracer uris.append(newuri) uds.append(newud) -- 2.34.1 102557 1 randy.macleod 2025-08-07 14:43:49 +0000 Oliver, Can you send the patch to the oe-core email list please? https://docs.yoctoproject.org/contributor-guide/index.html -- YP bug review / Randy 102828 2 randy.macleod 2025-09-25 15:05:12 +0000 Oliver, We don't have many git lfs users so unless you can send a patch or convince someone else to do that, this issue won't be fixed. If you need help getting started, see: https://docs.yoctoproject.org/contributor-guide/index.html and/or ask me. ../Randy 106095 3 sivakumar.bs 2026-07-03 11:20:17 +0000 Patch submitted to the bitbake-devel mailing list: [bitbake-devel][PATCH] fetch2: give every FetchData a default unpack_tracer https://lists.openembedded.org/g/bitbake-devel/topic/120097900 [add Patchwork link after sending] Thanks Oliver for the report, the root-cause analysis, and the proposed patch. I took a slightly different approach than the patch attached here. Copying the tracer onto the mirror ud in build_mirroruris() (newud.unpack_tracer = ud.unpack_tracer) assumes the origin ud always has an unpack_tracer, but that isn't guaranteed: Fetch() only sets it on the URLs it manages, so bare FetchData objects don't have it. Applying that change makes build_mirroruris() raise the same AttributeError and breaks all six existing bb.tests.fetch.MirrorUriTest tests. Instead, the fix initialises unpack_tracer to a DummyUnpackTracer in FetchData.init(), so the attribute always exists. Fetch() still overrides it with the real (or BB_UNPACK_TRACER_CLASS-configured) tracer for managed URLs; the mirror FetchData objects only do an internal, throwaway git-lfs checkout that shouldn't be traced anyway. Verified: bb.tests.fetch.MirrorUriTest: 6 pass on master, all 6 fail with the originally-proposed patch, 7 pass with this fix (added a regression test asserting mirror uds carry an unpack_tracer) Broader offline fetcher suite: 45 tests pass 106580 4 mathieu.dubois-briand 2026-08-20 12:53:06 +0000 Patch got merged: https://git.openembedded.org/bitbake/commit/?id=f0c9cf8d3885c5b1c2ba448f064421dae476fcd0 . Oliver, can you confirm we can close the bug?