10508 2016-10-27 16:04:38 +0000 bitbake fetcher does not always check for trusted_network 2016-12-06 17:26:58 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 1.4.5 x86 Multiple RESOLVED FIXED Medium+ major 2.3 M1 1 guru markus.lehtonen jose.perez.carranza markus.lehtonen poky.bs.watcher poky.watcher richard.purdie davidx.lopez.barriba --- --- No (bug/feature does not impact docs) oldest_to_newest 67648 0 guru 2016-10-27 16:04:38 +0000 We are using local mirrors on our local network, therefor not disabling global network access via BB_NO_NETWORK, but limiting the network access to our own servers only via BB_ALLOWED_NETWORKS. this does not work in case there is already an extracted git-tarball in the download directory but its content is outdated (e.g. the according tarball on the local mirror was updated). the (in this case git-)fetcher calls "bb.fetch2.check_network_access()" to verify that it is allowed to update the git repo from origin, but the implementation of check_network_access() only checks if BB_NO_NETWORK is not "1", it does not validate the given network address vs. any servers listed in BB_ALLOWED_NETWORKS in case BB_NO_NETWORK is "0"! in my opinion, check_network_access() should be enhanced like this: def check_network_access(d, info = "", url = None): """ log remote network access, and error if BB_NO_NETWORK is set or url is untrusted """ if d.getVar("BB_NO_NETWORK", True) == "1": raise NetworkAccess(url, info) elif not trusted_network(d, url): raise UntrustedUrl(url, info) else: logger.debug(1, "Fetcher accessed the network with the command %s" % info) p.s. this still does not solve the described bug, for this the local git-repo/-tarball should simply be deleted and re-downloaded/-extracted from the local mirror, but at least the build breaks with an "untrusted network" exception and no fetch from an untrusted source is happening! 68025 1 guru 2016-11-07 16:52:05 +0000 I noticed the optional 'url'-parameter in "check_network_access()" too late, but a valid 'url' is needed for 'trusted_network()' to work properly - otherwise a misleading "MalformedURL" exception may be raised! so, here is a slightly improved check_network_access(): def check_network_access(d, info = "", url = None): """ log remote network access, and error if BB_NO_NETWORK is set + or url is untrusted """ if d.getVar("BB_NO_NETWORK", True) == "1": raise NetworkAccess(url, info) + elif url is None and \ + (d.getVarFlag('BB_ALLOWED_NETWORKS', d.expand(d.getVar('PN', False))) or \ + d.getVar('BB_ALLOWED_NETWORKS', True)): + raise FetchError("cannot verify allowed hosts without URL") + elif not trusted_network(d, url): + raise UntrustedUrl(url, info) + elif info: + logger.debug(1, "Fetcher accessed the network with the command '%s'" % info) else: - logger.debug(1, "Fetcher accessed the network with the command %s" % info) + logger.debug(1, "Fetcher accessed the network") p.s. the logger texts (w/ or w/o 'info') in case of a successful check can most probably written shorter with some fancy python oneliner-magic, but I am not such pro yet... 68419 2 markus.lehtonen 2016-11-17 18:05:41 +0000 I wrote a simple patch that is available here: http://git.openembedded.org/openembedded-core-contrib/log/?h=marquiz/bitbake/fixes-10508 The forces all calls of check_network_access() to check that the URI is trusted (i.e. that it is not banned by BB_ALLOWED_NETWORKS). Basically, I think this would be the right thing to do. However, it causes multiple complications, e.g. git ls-remote may stop to work which badly breaks and "AUTOREV" packages. Similar breakages happens for "ls-remote" of other VCSes, too. Regarding Git update in the case that BB_ALLOWED_NETWORKS bans upstream source, I don't know what would be the right thing to do. The problem is that (AFAIU) bitbake only supports git mirrors through mirror tarballs and is not able to handle real git:// mirrors. Thinking to myself: I think the only thing we could reasonably do is delete/re-download/extract as you suggested. I don't know if Richard has any comments/ideas? 68453 3 richard.purdie 2016-11-21 12:09:47 +0000 (In reply to comment #2) > I don't know if Richard has any comments/ideas? FWIW I think the patch is the right thing to do... 68455 4 markus.lehtonen 2016-11-21 12:52:53 +0000 Patch sent for review: https://patchwork.openembedded.org/patch/134042/ 68830 5 markus.lehtonen 2016-12-05 09:07:44 +0000 Fix merged in: http://git.openembedded.org/bitbake/commit/?id=ddd3bc2d64d7240ecb6b6e4a1ae29b1faef6cc22