Bug 4091 - Add support to bitbake fetcher to fetch files from bare repos for PREMIRRORS
Summary: Add support to bitbake fetcher to fetch files from bare repos for PREMIRRORS
Status: RESOLVED OBSOLETE
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: All Multiple
: Medium enhancement
Target Milestone: Future
Assignee: Robert Yang
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-03-25 02:16 UTC by Jeff Polk
Modified: 2025-10-02 15:05 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff Polk 2013-03-25 02:16:58 UTC
WR ships our releases and updates via bare git repositories.  We also ship complete PREMIRRORS so that no network access or downloads are required but those directories and updates for them, while maintained internally as repositories, must currently be updated as regular directories.  We'd like to be able to harmonize the mechanisms used for our distributions and updates by directly supporting bare git repositories as PREMIRRORS.
Comment 1 Richard Purdie 2013-03-25 14:33:21 UTC
Doesn't this work already in master?
Comment 2 Richard Purdie 2013-06-20 21:12:25 UTC
I'm struggling to believe this doesn't work with current master since everything in DL_DIR exists as bare git repos and the mirror tarballs are even bare git repos. I'm closing this as WORKSFORME since I can't see the problem here. If there is a problem, please repoen with more details.
Comment 3 Stoicescu Cornel 2013-10-11 10:59:02 UTC
Inside bitbake/lib/bb/tests/fetch.py I see:

self.d.setVar("PREMIRRORS", "git://someserver.org/bitbake git://git.openembedded.org/bitbake \n")

Marking this as VERIFIED.
Comment 4 Mark Hatle 2014-06-23 19:02:37 UTC
I'm reopening this.  I found the missing piece of information.

The request is not to mirror bare repositories.  The request is to mirror -files- -from- bare repositories.

In otherwords, if I have a git repository of all of the downloads.  I shouldn't have to git clone meta-downloads, and then point the PREMIRRORS to 'file://...meta-downloads'.

Instead I should be able to point the PREMIRRORS to 'git://...meta-downloads.git;branch=daisy'  It will then get a PREMIRROR call and attempt to extract just the requested file(s) from the bare git repository.
Comment 5 Richard Purdie 2014-09-24 14:33:27 UTC
Hi Mark, I hadn't seen this until now. The fetcher is "dumb" in some senses, in particular it doesn't know anything about the content of the git repo. This sounds like you want the fetcher to optionally fetch the repo, then unpack it and see if it contains something else which looks like an artefact (either another repo or tarball) which can then be passed back to the original "fetch".

I'm not sure this recursive fetching is something we want to support or encourage to be quite honest. If you did want to do it, I'd suggest a different fetcher module for it using a different url type to distinguish it from the "normal" git fetcher. That backend would get calls in from the premirrors code and it would be up to it how it dealt with them.

As such, this isn't something I'm planning to work on though and I am not sure how feasible this is, just thinking out loud for anyone who may want to try implementing it.
Comment 6 Stephen K Jolley 2015-11-19 16:06:54 UTC
This was REOPENED
Comment 7 Mark Hatle 2017-08-03 19:15:04 UTC
Just so it's clear, I'm still tracking this.  Even though it is in a future state.
Comment 8 Randy MacLeod 2019-07-25 15:32:29 UTC
Mark, it's been 2 years, are you still tracking this?
Comment 9 Mark Hatle 2019-07-26 18:34:11 UTC
Yes I am.
Comment 10 Randy MacLeod 2021-09-23 15:25:03 UTC
Robert, do we still want this bug opened or has it been fixed?
Comment 11 Randy MacLeod 2023-09-28 15:08:23 UTC
Robert, do we still care about this ER ?
Comment 12 Randy MacLeod 2025-10-02 15:05:03 UTC
No activity so declaring obsolete.