Bug 13233

Summary: fetch2: try_premirror(): improve on updating repo from mirror
Product: [Build System, Metadata & Runtime] BitBake Reporter: Robert Yang <liezhi.yang>
Component: bitbakeAssignee: Pavel Zhukov <pavel>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium+ CC: pavel, poky.bs.watcher, poky.watcher, randy.macleod
Version: 0.0.0   
Target Milestone: 4.1 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Robert Yang 2019-03-25 06:37:37 UTC
Currently, when rework in an old build, e.g, suppose
there is a foo_git.bb:

1) Add foo.git to PREMIRROR

2) In builddir
$ bitbake foo
Then foo.git will be fetched into DL_DIR, which is premirror_foo.git.

3) Upgrade foo_git.bb's SRCREV, and this SRCREV isn't in premirror_foo.git since
   it's very new.

4) Upgrade foo.git in PREMIRROR, now it contains the SRCREV of step 3

5) Go to the old builddir
$ bitbake foo

Then it won't update from PREMIRROR if BB_FETCH_PREMIRRORONLY is not set,
and it would fail to build if BB_NO_NETWORK = "1" since SRCREV can't be found
in DL_DIR. Now the problem is:

- Set BB_FETCH_PREMIRRORONLY = "0" and BB_NO_NETWORK = "1" will make foo fail
  to build.

- Set BB_FETCH_PREMIRRORONLY = "1" will make it work for foo_git.bb, but it
  may not work for another recipe such as foo_X_git.bb even if
  BB_NO_NETWORK = "0".(Imagine the PREMIRROR doesn't contain foo_X_git.bb's
  SRCREV)

- Set BB_FETCH_PREMIRRORONLY = "0" and BB_NO_NETWORK = "0" will make foo work,
  but it won't upgrade from PREMIRROR, though everything is up-to-date there.
  It will update DL_DIR from SRC_URI, this isn't good when network is slow.


But if we make try_mirror always return true would get a little complex, for example if premirrors is a tarball
mirror of a git repo rather than a full repo.

If you change this to always return True, it would always download the
mirror tarball for a git repo rather than trying to do a git pull in
the main repo which would just download the delta.

The correct behaviour would probably be to work out whether we can do a
"git fetch" update from any source, then go back to pulling mirror
tarballs if we can't get the revision by the other means.
Comment 1 Pavel Zhukov 2022-06-09 06:57:30 UTC
This should be fixed in: https://git.openembedded.org/bitbake/commit/?id=b47ecab3e3aad5c5c376ec023aa82a51aa0f3b86 and covered by FetchPremirroronlyLocalTest