Bug 5631

Summary: BB_NO_NETWORK broken for git repositories
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Gary Thomas <gary>
Component: coreAssignee: Richard Purdie <richard.purdie>
Status: VERIFIED FIXED QA Contact:
Severity: critical    
Priority: Medium+ CC: ionutx.chisanovici, meta.mr.watcher, meta.watcher, richard.purdie
Version: unspecified   
Target Milestone: 1.6 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Gary Thomas 2013-12-05 12:17:27 UTC
I try to use a local source mirror, fully populated with packages, so I
can deliver a complete (no network required) bundle to my customers.  This
has recently been broken.

I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit 2354250a95eab484459f41f8715ae112295c2174
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date:   Mon Nov 18 17:17:16 2013 +0000

    bitbake: git: Use merge-base instead of log for testing if a commit is present
    
    The current use of git log to check if a given revision is present can be
    a little fragile.
    
    For example if revision X was on branch A, and then later added to branch
    B, the update checks would not notice this since they just check for X
    being in the repository.
    
    We also had some autobuilder corruption where an older packed-refs file
    was copied over a new repository containing newer pack files. There
    was no update to the refs file since the revision was present but
    not accessible in any branch.
    
    The correct fix is to check that the required revisions are present
    on the specific branches. This patch does this using merge-base.
    
    (Bitbake rev: 89abfbc1953e3711d6c90aff793ee622c22609b1)
    
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

To test this, just add these settings to local.conf
  SOURCE_MIRROR_URL ?= "file:///work/misc/Poky/sources/"
  INHERIT += "own-mirrors"
  BB_NO_NETWORK = "1"
  BB_GENERATE_MIRROR_TARBALLS ?= "1"

Make sure and put the appropriate git tarball into the SOURCE_MIRROR.
I tested this using prelink, e.g.
  $ bitbake prelink -c fetch
the the tarball is git2_git.yoctoproject.org.prelink-cross.git.tar.gz
A copy of this is at
  http://www.mlbassoc.com/poky/git2_git.yoctoproject.org.prelink-cross.git.tar.gz
Comment 1 Richard Purdie 2013-12-05 16:30:52 UTC
The new validation code was catching cases where the incorrect branch was set on git repositories. As of http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=f19546e02d3318ee69fd0c34e21aa97b74c987ec it will catch these even when BB_NO_NETWORK is set. There are patches in OE-Core master to fix the various repositories which had incorrect branches.
Comment 2 Gary Thomas 2013-12-05 16:42:05 UTC
Sorry, this still fails for me.  I updated to the Poky/Yocto master rev 5301c59503e0d3fdc0080d80754ec66db40de46c which includes the mentioned change.

I tried my two test cases - both failed:

ERROR: Function failed: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command git fetch -f --prune git://git.yoctoproject.org/matchbox-keyboard refs/*:refs/* (for url git://git.yoctoproject.org/matchbox-keyboard;branch=matchbox-keyboard-0-1)
ERROR: Logfile of failure stored in: /tmp/p8301_test/tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/matchbox-keyboard/0.0+gitAUTOINC+217f1bfe14-r4/temp/log.do_fetch.30373

ERROR: Function failed: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command git fetch -f --prune git://git.yoctoproject.org/prelink-cross.git refs/*:refs/* (for url git://git.yoctoproject.org/prelink-cross.git;branch=cross_prelink)
ERROR: Logfile of failure stored in: /tmp/p8301_test/tmp/work/cortexa9hf-vfp-neon-amltd-linux-gnueabi/prelink/1.0+gitAUTOINC+6822ec76aa-r0/temp/log.do_fetch.30260

What am I missing?
Comment 3 Gary Thomas 2013-12-05 22:47:24 UTC
Still broken unless I'm missing something.
Comment 4 Richard Purdie 2013-12-06 14:11:13 UTC
If you immediately rerun the command it works. It was a state issue inside the fetcher and is addressed by http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=184038bedb20f74ba4dcad6ef2d92e87a9ba5f9a
Comment 5 Ionut Chisanovici 2013-12-23 11:32:34 UTC
This is verified with master:005af45191ded6185c618c708181b31281e43092.

Cheers,
Ionut C