Bug 9888 - BB_NO_NETWORK prevents uninative fetch from file:// PREMIRROR
Summary: BB_NO_NETWORK prevents uninative fetch from file:// PREMIRROR
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: 2.1
Hardware: x86 x86_64
: Medium normal
Target Milestone: 2.4
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-07-06 19:15 UTC by Shaun
Modified: 2017-08-16 14:43 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
site.conf to set BB_NO_NETWORK and own-mirror (162 bytes, application/octet-stream)
2016-07-06 19:15 UTC, Shaun
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Shaun 2016-07-06 19:15:53 UTC
Created attachment 3272 [details]
site.conf to set BB_NO_NETWORK and own-mirror

Reproduced with poky krogoth commit e8898f018862e7e398ea84732a11630e30fb2ceb on Ubuntu 14.04 LTS (64-bit).  Target is core-image-minimal for beaglebone black machine. Using attached site.conf file to disable network access and use my own-mirror. 

When running bitbake, I get: "WARNING: Disabling uninative as unable to fetch uninative tarball: Network access disabled through BB_NO_NETWORK" (from poky/meta/classes/uninative.bbclass). Besides uninative, the rest of bitbake core-image-minimal builds fine. 

I created the downloads mirror by making a build with the network enabled, then copied the downloads directory to a new downloads_mirror directory.  Then I deleted the downloads and the directory created by the oe init script (which contained tmp), re-ran the oe init script, and added the attached conf/site.conf which disables network access for the build and specifies my own-mirror directory. 

If I manually copy the uninative directory from downloads_mirror to downloads then the warning will go away. There seems to be an issue with the uninative.bbclass when using the premirror/no network build. I'm not sure if this is related to the poky/meta/classes/uninative.bbclass recipe, or the bitbake do fetch.  Could this be related to bug 9175?
Comment 1 Joshua Lock 2017-06-14 13:01:18 UTC
There are related problems when trying to fetch the uninative tarball with own-mirrors enabled.

If I set the following in site.conf:

SOURCE_MIRROR_URL ?= "http://downloads.yoctoproject.org/mirror/sources/"
INHERIT += "own-mirrors"

fetching the uninative tarball fails with checksum mismatch:

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a
WARNING: Checksum mismatch for local file /srv/builds/mirror-test/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2
Cleaning and trying again.
WARNING: Renaming /srv/builds/mirror-test/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2 to /srv/builds/mirror-test/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2_bad-checksum_4b6fb2106c2f18b8f9c8ac67e1cf9e57
ERROR: Checksum failure fetching http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a
WARNING: Disabling uninative as unable to fetch uninative tarball: Fetcher failure for URL: 'http://downloads.yoctoproject.org/releases/uninative/1.6/x86_64-nativesdk-libc.tar.bz2;sha256sum=2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a'. Checksum mismatch!
File: '/srv/builds/mirror-test/downloads/uninative/2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a/x86_64-nativesdk-libc.tar.bz2' has sha256 checksum 26d46c61ad88cc245e31c88549717c0092a838d111b93ec169d88b08cc027581 when 2b4fffa308d9f19e0742a1a404ff42495fb50c165e5ca0458cedca157372691a was expected
If this change is expected (e.g. you have upgraded to a new version without updating the checksums) then you can use these lines within the recipe:
SRC_URI[md5sum] = "4b6fb2106c2f18b8f9c8ac67e1cf9e57"
SRC_URI[sha256sum] = "26d46c61ad88cc245e31c88549717c0092a838d111b93ec169d88b08cc027581"
Otherwise you should retry the download and/or check with upstream to determine if the file has become corrupted or otherwise unexpectedly modified.
Comment 2 Joshua Lock 2017-07-14 09:56:21 UTC
(In reply to comment #1)
> There are related problems when trying to fetch the uninative tarball with
> own-mirrors enabled.

This was two separate issues:
1) that the YP mirror was carrying an invalid copy of the uninative tarball, triggering a correct checksum mismatch failure. This has been removed.
2) that if a file was fetched from a premirror but had an incorrect checksum, mirrors and upstream fetch locations wouldn't be tried. This has been fixed in master: 
http://git.openembedded.org/bitbake/commit/?id=022adb30dbb0df764c9fb515918cb9a88e4f8d6f

These turn out to be unrelated to the original reporter's issue, which we should test whether is still a valid bug.
Comment 3 Joshua Lock 2017-07-14 09:59:01 UTC
I'm surprised that everything other than the uninative fetch succeeds, that suggests that uninative.bbclass is doing something different with file:// URIs than other uses of the fetcher.


Shaun, as own-mirrors sets PREMIRRORS up you may be able to achieve what you want by using BB_FETCH_PREMIRRORONLY rather than BB_NO_NETWORK?
Comment 4 Shaun 2017-07-14 15:13:01 UTC
Joshua, I was able to work around the problem by using BB_FETCH_PREMIRRORONLY using own mirrors. The uninative fetcher would print out 

"NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org..."

Which seemed like the fetch was ignoring my pre-mirror and using the network, but since it worked I was okay with the workaround. I was never sure if the network was being used for uninative or not.  If the build uses BB_FETCH_PREMIRROR_ONLY, I have my own mirror set up, and the NOTE above still prints out, is the network actually being used or is this a misleading NOTE?
Comment 5 Joshua Lock 2017-07-17 12:28:12 UTC
Shaun, the NOTE is misleading. It's reporting the upstream URI (from SRC_URL) for the uninative tarball, but should actually be fetching the version from your PREMIRROR.

For whoever ends up working on this; I can confirm that with own-mirrors enabled and a file:// uri in SOURCE_MIRROR_URL I observe:
* with BB_NO_NETWORK: uninative fetch always fails
* with BB_PREMIRRORS_ONLY: uninative fetch will succeed 

It's also noteworthy to emphasise that with BB_NO_NETWORK all other fetcher uses successfully access the file:// PREMIRRORS