3119 2012-09-14 16:17:28 +0000 Beta - default sanity checking fails if http proxy set but not git / SVN 2012-10-02 22:52:15 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 1.3 x86 Multiple RESOLVED FIXED Medium+ major 1.3 1 david.c.stewart bluelightning andy.ross bluelightning poky.bs.watcher poky.watcher sgw --- --- --- oldest_to_newest 25285 0 david.c.stewart 2012-09-14 16:17:28 +0000 I have my http and https proxies set and do $ bitbake -k core-image-sato If you have your http proxy set correctly, we're supposed to default over to the mirrors. But there are two problems with this now: 1. The sanity test at the beginning fails because git isn't working. 2. When building eglibc, the do_fetch fails because svn is not accessible. So the out-of-box experience with the command line isn't working as expected. believe RP sent mail to the poky list about this behavior and I have not seen a reply. 25287 1 richard.purdie 2012-09-14 16:30:35 +0000 The questions: a) Do we add an svn sanity check? b) Do we remove the git sanity check instead? c) Why didn't the svn tarball come from the mirror? svn shouldn't have been necessary. 25372 2 andy.ross 2012-09-18 17:51:48 +0000 I just hit what I assume is the same issue and was pointed here from #yocto. What's happened is that the downloads.yoctoproject.org/mirrors/sources directory doesn't have a current mirror of the eglibc source code. The most recent svn revision it has is "19383", but yocto commits 91746bab and 5b8e46db have since bumped that revision to "19922" and then "20393". So the build doesn't see a working mirror and tries to get the source from upstream. But www.eglibc.org is unusably slow -- I can't seem to get a transfer rate of more than 1-200kbps ever, and over a day and a half of build attempts I haven't gotten through a working download. It uniformly fails with a timeout after a few hours. 25707 3 bluelightning 2012-09-25 15:47:50 +0000 Michael & Beth have fixed the mirroring so we have up-to-date mirrors for eglibc, and I've verified that the fetch from the mirror now works. 26060 4 bluelightning 2012-10-02 16:56:48 +0000 Richard has now dropped the Git check as it only gets in the way rather than helping the uninitiated user as was originally intended: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=029a74407836390acc29201309130e3860caf072 I think that means we can resolve this bug as the two issues mentioned are now addressed. 26063 5 evadeflow 2012-10-02 17:40:16 +0000 I may have been bitten by this (or Bug 3175, not sure whether they're the same). I'm an inexperienced Yocto Project user trying hard to evolve into an experienced one, but my edit/run/debug cycle is really crippled by the restrictive firewall at work. I have 16 cores at my disposal there, but the machine they're in can only access the Internet via an HTTP proxy. I have this in my .gitconfig: > [http] > proxy=http://user:passwd@usaprox.lightning.com:8080 so you'd think I'd be able to access URLs like this one (from the meta-ti layer's linux-omap4_3.1.0.bb recipe): > SRC_URI = "http://dev.omapzoom.org/pub/scm/integration/kernel-ubuntu.git;protocol=git;branch=ti-ubuntu-3.1-1282 \ > file://0001-Makefile.fwinst-fix-install-breakage-for-FW-images-r.patch \ > file://defconfig \ > " But, from what I can tell, bitbake never tries to fetch this, and the error I wind up getting isn't particularly helpful: > ERROR: Command Error: exit status: 1 Output: > Applying patch 0001-Makefile.fwinst-fix-install-breakage-for-FW-images-r.patch > patching file scripts/Makefile.fwinst > Hunk #1 FAILED at 27. > 1 out of 1 hunk FAILED -- rejects in file scripts/Makefile.fwinst > Patch 0001-Makefile.fwinst-fix-install-breakage-for-FW-images-r.patch does not apply (enforce with -f) > ERROR: Function failed: patch_do_patch What's really frustrating is that I did a fetchall offsite into a mirrors folder on a USB hard drive, and then rsync'ed that folder to the build server at work. Not only did I do 'fetchall core-image-sato' and 'fetchall core-image-minimal', but I also did 'fetchall universe' for good measure. Bitbake still decided, once behind the firewall, "Oh, yeah---I'm gonna need one more thing." (I'm learning that Bitbake can be kind of a jerk that way.) Here's my build configuration: > OE Build Configuration: > BB_VERSION = "1.15.2" > TARGET_ARCH = "arm" > TARGET_OS = "linux-gnueabi" > MACHINE = "pandaboard" > DISTRO = "poky" > DISTRO_VERSION = "1.2.1" > TUNE_FEATURES = "armv7a vfp neon cortexa9" > TARGET_FPU = "vfp-neon" > meta > meta-yocto = "denzil:65ffa7395055f7e012cb973f63f92380828eed0d" > meta-ti = "(nobranch):30fb40ebc13614a74c2e237927c60ac43e01d1bc" Maybe some of this stuff has been fixed already, I'm just relaying my experience as a new user. So far, it seems very, very difficult to either: 1) Get Bitbake to pull everything via HTTP 2) Set up a *complete* mirror in one go I can understand #1, but #2 is vexing. I've made a dozen trips to the local public library to download just "one more package" to my mirror. I've considered writing a python script to go get all of the resources specified in *every single SRC_URI* in my Yocto tree, but then I thought: how silly is that? Bitbake knows about all of these, and it's written in python! Why can't it do this for me? So, I guess that's a feature request for 1.4: add a command to Bitbake that's like 'fetchall', but actually fetches EVERYTHING, and builds NOTHING. (The current 'fetchall' command actually builds quite a lot before downloading anything.) I'm not sure what to call such a command, maybe something like "Hey, I'm gonna be behind a Draconian firewall later today and I need you to download EVERY SINGLE RESOURCE required to build any of the images this build tree can possibly build, right now, while I'm connected to the Internet. Okay, Bitbake?" Here are my mods to conf/local.conf, in case it's helpful: > MACHINE = "pandaboard" > BBMASK = "meta-ti/recipes-misc" > PACKAGE_CLASSES = "package_ipk" > CONNECTIVITY_CHECK_URIS="" > BB_GENERATE_MIRROR_TARBALLS = "1" > SOURCE_MIRROR_URL ?= "file:///home/evadeflow/projects/poky-mirror/" > INHERIT += "own-mirrors" And this is my bblayers.conf: > # LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf > # changes incompatibly > LCONF_VERSION = "4" > > BBFILES ?= "" > BBLAYERS ?= " \ > /home/dwolfe/projects/poky-git/meta \ > /home/dwolfe/projects/poky-git/meta-yocto \ > /home/dwolfe/projects/poky-git/meta-ti \ > " 26085 6 bluelightning 2012-10-02 22:52:15 +0000 So, a few notes in response: 1) The SRC_URI in the linux-omap recipe you quote is at least slightly if not completely wrong. From the suffix it looks like a git URI except bitbake will treat it as a standard http download because of the http:// prefix. Maybe that is the correct thing for fetching the source for this recipe, but I doubt it - if it's intended to fetch from a git repository, it needs to start with git:// even if you later specify protocol=http to use the http transport. If on the other hand the URI uses protocol=git it probably won't be fetchable at all from behind a firewall without a git proxy set up. 2) Unless something very very wrong is going on, the error you are indicating is an error from do_patch, not do_fetch (hence the error mentioning patch application failure). If the URI is broken as I mentioned above, I'm not surprised the patch would fail to apply though as it won't have actually downloaded the kernel source. 3) We can't do a fetchall without building anything because we need to build some of the tools needed to do the fetching. fetchall should fetch everything needed to build whatever target you specify; if it does not that is a bug, but you didn't mention which item actually didn't get fetched assuming it was something other than the situation above...? If there are still bugs by all means open up a new bug report with details and we'll investigate. Rest assured that what you are trying to do - i.e. create a local source mirror and then make that available to a machine without network access - is a fairly common use case and is supported.