Bug 11830

Summary:	gitsm is fetching from upstream in do_unpack
Product:	[Build System, Metadata & Runtime] BitBake	Reporter:	Christopher Larson <kergoth>
Component:	bitbake	Assignee:	Mark Hatle <mark.hatle>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	mark.hatle, mark.jonas, poky.bs.watcher, poky.watcher, randy.macleod, rebecca.swee.fun.chang, sgw
Version:	unspecified
Target Milestone:	Future
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Christopher Larson 2017-07-24 21:49:29 UTC

The gitsm download method jumps through hoops (pretending it's a non-bare repo) to get the submodules fetched into the 'modules' directory relative to ud.clonedir, then the unpack() method ignores the modules directory entirely and lets `submodule update` re-clone the submodules from upstream in *do_unpack* rather than do_fetch. This behavior was introduced by "fetch2/gitsm: Fix when repository change submodules", prior to this commit the modules were being copied correctly.

If the modules in the clone directory don't match the branch we need to check out, then the download method should be fixed to correctly update the modules directory, rather than ignoring it entirely.

With the current behavior, it's not possible to build with a fully populated DL_DIR and no network connectivity available.

Comment 1 Richard Purdie 2017-07-27 15:00:01 UTC

There was discussion about this on the bitbake list, seems there was a patch identified which caused this: http://lists.openembedded.org/pipermail/bitbake-devel/2017-July/008835.html

Comment 2 Christopher Larson 2017-07-28 17:45:04 UTC

I think it's clear the specified commit should be reverted, but we don't want that to reintroduce the issue that commit was trying to fix, so we'll need to do some testing with that and potentially introduce a unit test or two to go along with the revert.

Comment 3 Christopher Larson 2017-08-21 16:58:36 UTC

2 patches sent to the bitbake list for this.

Comment 4 Christopher Larson 2017-08-29 15:13:46 UTC

Having rethought and tested the patches I submitted, there's just no viable way to work around this without reworking how gitsm works. Including submodules inside the git repo will *always* make assumptions about exactly what modules are included and from where, which can change not just between branches, but between srcrevs, and the repository and mirror tarball can be used for any number of those, so either we lock down both the repo and mirror tarball to srcrev, or we stop shipping submodules inside the git repository.

Given this, the only viable path forward for this issue is Issue #11831. I'm keeping the issue open, as clearly this bug still exists, and will need verification.

Comment 5 Mark Hatle 2019-05-15 11:46:03 UTC

The gitsm fetcher has been refactored over the last year.  See the thud (1.40) and newer branches.