Bug 8323

Summary: fetch2/hg mirror tarballs are kinda broken
Product: [Build System, Metadata & Runtime] BitBake Reporter: Richard Tollerton <rich.tollerton>
Component: bitbakeAssignee: Pavel Zhukov <pavel>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: pavel, poky.bs.watcher, poky.watcher, randy.macleod, rich.tollerton, richard.purdie
Version: unspecified   
Target Milestone: Future   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Richard Tollerton 2015-09-17 23:41:37 UTC 
The hg fetcher generates mirror tarballs as hg.localfile, cf bitbake/lib/bb/fetch2/hg.py:Hg.urldata_init(). But there are severe problems with them:

(1) They require SRC_URI[md5sum] or SRC_URI[sha256sum] checksums be supplied in recipes, in bitbake >1.26 and when BB_STRICT_CHECKSUM is set. These ought not to be necessary in the first place, because Mercurial commits are merkle trees; when repos are properly cloned, commit ids provide an equivalent guarantee of integrity. 

And of course you can't really reproduce these tarballs outside the fetcher itself, so editing SRC_URI becomes an unnecessarily two-step process.

(2) AFAIK, they appear to be created but never extracted. They are not connected at all to the BB_GENERATE_MIRROR_TARBALLS facility in fetch2/git.

Note that fetch2/git does not suffer from these problems, because (a) Git.supports_checksum() returns False, and (b) Git.unpack() does a clone from the repo at DL_DIR to the work directory, implicitly verifying the integrity of the commit history.
Comment 1 Richard Purdie 2015-09-24 14:37:34 UTC 
To be honest we don't use the hg fetcher much. Would you be able to send some patches to make it behave more like the git fetcher?
Comment 2 Richard Tollerton 2015-10-12 17:48:32 UTC 
I'll chuck it onto my team's backlog, but we're pretty hammered too and I don't think we'll get to it before end of dev for 2.1.
Comment 3 Randy MacLeod 2020-01-30 16:14:59 UTC 
Richard,
Are you able to send a patch to the list for this hopefully with a test.
Comment 4 Pavel Zhukov 2022-02-22 17:27:43 UTC 
Currently mercurial is broken in OE. mercurial-native failed to connect to any ssl enabled servers.
Comment 5 Pavel Zhukov 2023-01-17 19:13:16 UTC 
I've created simple testcase for hg/premirrors functionality [1] and with small fix in fetcher code [2] is seems working. fetcher is able to fetch repo, create mirror tarball from it and then use the tarball to download() the project with networking disabled.

[1] https://lists.openembedded.org/g/bitbake-devel/message/14319
[2] https://lists.openembedded.org/g/bitbake-devel/message/14318