Bug 13233 - fetch2: try_premirror(): improve on updating repo from mirror
Summary: fetch2: try_premirror(): improve on updating repo from mirror
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: 0.0.0
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 4.1 M1
Assignee: Pavel Zhukov
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2019-03-25 06:37 UTC by Robert Yang
Modified: 2022-06-09 06:57 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Yang 2019-03-25 06:37:37 UTC
Currently, when rework in an old build, e.g, suppose
there is a foo_git.bb:

1) Add foo.git to PREMIRROR

2) In builddir
$ bitbake foo
Then foo.git will be fetched into DL_DIR, which is premirror_foo.git.

3) Upgrade foo_git.bb's SRCREV, and this SRCREV isn't in premirror_foo.git since
   it's very new.

4) Upgrade foo.git in PREMIRROR, now it contains the SRCREV of step 3

5) Go to the old builddir
$ bitbake foo

Then it won't update from PREMIRROR if BB_FETCH_PREMIRRORONLY is not set,
and it would fail to build if BB_NO_NETWORK = "1" since SRCREV can't be found
in DL_DIR. Now the problem is:

- Set BB_FETCH_PREMIRRORONLY = "0" and BB_NO_NETWORK = "1" will make foo fail
  to build.

- Set BB_FETCH_PREMIRRORONLY = "1" will make it work for foo_git.bb, but it
  may not work for another recipe such as foo_X_git.bb even if
  BB_NO_NETWORK = "0".(Imagine the PREMIRROR doesn't contain foo_X_git.bb's
  SRCREV)

- Set BB_FETCH_PREMIRRORONLY = "0" and BB_NO_NETWORK = "0" will make foo work,
  but it won't upgrade from PREMIRROR, though everything is up-to-date there.
  It will update DL_DIR from SRC_URI, this isn't good when network is slow.


But if we make try_mirror always return true would get a little complex, for example if premirrors is a tarball
mirror of a git repo rather than a full repo.

If you change this to always return True, it would always download the
mirror tarball for a git repo rather than trying to do a git pull in
the main repo which would just download the delta.

The correct behaviour would probably be to work out whether we can do a
"git fetch" update from any source, then go back to pulling mirror
tarballs if we can't get the revision by the other means.
Comment 1 Pavel Zhukov 2022-06-09 06:57:30 UTC
This should be fixed in: https://git.openembedded.org/bitbake/commit/?id=b47ecab3e3aad5c5c376ec023aa82a51aa0f3b86 and covered by FetchPremirroronlyLocalTest