Current git fetcher code has strange behavior with PREMIRRORS. I've noticed that in my test cases, where PREMIRROS are defined, file with repository git://github.com/file/file.git present and has needed version. What happened in log.do_fetch.*: $ grep "Trying" log*fetch*31663 DEBUG: Trying Upstream DEBUG: Trying MIRRORS $ So, it cleanly tries to fetch from upstream, where corporate proxy misbehave, so it can't reach github, then it tries to access MIRROS, but can't find needed revision, and build eventually fails. Expectation that fetcher code should work consistently for both HTTP/FTP sources and sources from version control systems: - get sources from PREMIRRORS, verify that needed sources/revision present there. - if not, try Upstream. - if Upstream not available, try MIRRORS.
I think I might be able to explain this, there were some comments on the bitbake mailing list which shed some light on what is happening. If a git repository has already been fetched and is present in DL_DIR, the git fetcher specifically will try and pull changes from upstream rather than try a PREMIRROR. This is because those premirrors typically don't support git and would require a complete mirror tarball download. Ideally, we should differentiate between complete downloads and git aware mirrors.
I suspect we need to split download() into two passes, and update() and then a full download if update isn't possible or implemented...
The case I'm having trouble with is if BB_ALLOWED_NETWORKS is set, which allows bitbake to access the internal mirror and the internal git repository, but doesn't allow access to gitlab.com for example. It might be possible to add a check if the url git tries to fetch is a trusted url or not and decide based on this if we should download the tarball or fetch from the repo. But this feels like a hack to me, as it would not cover the case where upstream access would be possible but the repository is down. In this case it should also fall back to the mirror as downloading the tarball is still better then to fail.
It has been years since this defect was opened and instead of moving it along to the next release, I'm looking for an indication that this is actually a problem that we should try to fix. We also typically prefer https:// fetching so perhaps this isn't a problem any more. Pascal, could you take a look and comment this week before Thursday 10:30 AM ET?
(In reply to comment #4) > It has been years since this defect was opened and instead of moving it > along to the next release, I'm looking for an indication that this is > actually a problem that we should try to fix. We also typically prefer > https:// fetching so perhaps this isn't a problem any more. > > Pascal, could you take a look and comment this week before Thursday 10:30 AM > ET? This should be fixed as part of https://git.openembedded.org/bitbake/commit/?h=master-next&id=bde98b6c4a23db9571caa28a111d8fdf889ccd94 . Taking this bug to test.
This is fixed by: https://git.openembedded.org/bitbake/commit/?id=b47ecab3e3aad5c5c376ec023aa82a51aa0f3b86