Bug 14707

Summary:	do_unpack may access network in FetchMethod.latest_revision
Product:	[Build System, Metadata & Runtime] BitBake	Reporter:	Peter Hoyes <Peter.Hoyes>
Component:	bitbake	Assignee:	Pavel Zhukov <pavel>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	High	CC:	pavel, poky.bs.watcher, poky.watcher, randy.macleod
Version:	5.99
Target Milestone:	4.0 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Peter Hoyes 2022-01-28 15:48:17 UTC

With the latest master (where network access is disabled in do_unpack):

Reproduction steps:
 * Create a recipe whose SRCREV is *not* a SHA (e.g. a Git tag)
 * Build the recipe
 * Make a change to the recipe, such that do_unpack is re-triggered but do_fetch is not
 * Build the recipe again. do_unpack fails as it tries to run `git ls-remote` to resolve the tag.

The offending function call is at https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/fetch2/git.py#n256

I understand that it is best practice to use a SHA in the SRCREV so that no `ls-remote` is necessary, but this does not seem to be mandatory at the moment.

Could do_unpack fetch the latest revision from the clone in the downloads directory instead?

Comment 1 Randy MacLeod 2022-02-03 15:40:08 UTC

Pavel will take this once this he gets a BZ account.

Comment 2 Pavel Zhukov 2022-02-05 09:08:51 UTC

Hello,

Thank you for reporting this issue. 
Bitbake caches srcrev information in PERSISTENT_DIR and it should persist between builds (according to https://www.yoctoproject.org/docs/2.0.3/bitbake-user-manual/bitbake-user-manual.html ). In fact this behavior is controlled by BB_SRCREV_POLICY and the default policy is "clean" that means the cache is being cleaned each time (https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/__init__.py#n511)

To mitigate the issue BB_SRCREV_POLICY can be set to "cache" in local.conf. In that case bitbake uses URI_HEADREVS store to resolve revisions and the issue doesn't occur. 

latest_revisions() for git fetcher can be improved a bit to use rev-parse in case if network is not available but this will not work in all cases (rebasesable repos for example).

Comment 3 Pavel Zhukov 2022-02-22 17:26:18 UTC

This is fixed in https://git.openembedded.org/bitbake/commit/?id=4b5eed1626709ef3dc06b32fd55d40a2a6edd179
Closing