Bug 11831 - Proposal to replace/rewrite gitsm
Summary: Proposal to replace/rewrite gitsm
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: Future
Assignee: Mark Hatle
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-07-24 22:08 UTC by Christopher Larson
Modified: 2019-05-15 11:47 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christopher Larson 2017-07-24 22:08:52 UTC
The current gitsm implementation is naive at best, broken at worst, and lets git itself do the fetching of the submodules, so we can't use our mirror tarballs or shallow tarballs for them, among other issues, such as the already reported issue with license.bbclass.

This issue is to track its replacement with a version that uses the bitbake fetcher to fetch the submodules recursively, and provides the urls of all the submodules to the metadata (likely stored as a member of the fetcher object, so anyone that needs it will have to isinstance() and grab it from there, as we can't get that information at parse time, only after the download()).

We can implement direct logic to do this, or we can try implementing a fetch helper to let git call back to us to do the repository fetch for it.
Comment 1 Christopher Larson 2017-08-29 15:19:37 UTC
I've created a thus-far working prototype of a rewritten gitsm that uses the bitbake fetcher for the submodules, and am currently testing it, mulling over how to address shallow support, and starting on thorough unit tests which don't all rely on network connectivity (we can construct the repositories we need on the fly, the way we do in the shallow unit tests). See https://gist.github.com/kergoth/9c163433c33cf883b21256459d743a98 for the current implementation.
Comment 2 Mark Jonas 2018-12-18 09:41:24 UTC
My team is affected by this issue as well. We are looking forward to a solution. :)
Comment 3 Mark Hatle 2019-05-15 11:47:03 UTC
The gitsm fetcher has been refactored and rewritten to address this, locking and other issues.  The code changes went into the thud version (1.40).
Comment 4 Mark Hatle 2019-05-15 11:47:28 UTC
resolved