Bug 11698

Summary: Git shallow fetcher sometimes fails selftest
Product: [Build System, Metadata & Runtime] BitBake Reporter: Ross Burton <ross.burton>
Component: bitbakeAssignee: Christopher Larson <kergoth>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: joshuagloe, poky.bs.watcher, poky.watcher, randy.macleod, richard.purdie
Version: unspecified   
Target Milestone: 2.4 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2017-06-21 14:41:12 UTC 
Sometimes but not always the git shallow fetcher selftest fails:

ERROR: test_bitbake (bb.tests.fetch.GitShallowTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1534, in latest_revision
    return revs[key]
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/persist_data.py", line 83, in __getitem__
    raise KeyError(key)
KeyError: 'git:.tmp.tmpgwkjmxuk.gitsourcemaster-'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/tests/fetch.py", line 1450, in test_bitbake
    self.fetch_shallow()
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/tests/fetch.py", line 1063, in fetch_shallow
    fetcher, ud = self.fetch_and_unpack(uri)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/tests/fetch.py", line 1056, in fetch_and_unpack
    fetcher, ud = self.fetch(uri)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/tests/fetch.py", line 1050, in fetch
    fetcher = bb.fetch2.Fetch(uris, d)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1567, in __init__
    self.ud[url] = FetchData(url, d, localonly)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1246, in __init__
    self.method.urldata_init(self, d)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/git.py", line 239, in urldata_init
    ud.setup_revisions(d)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1276, in setup_revisions
    self.revisions[name] = srcrev_internal_helper(self, d, name)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1141, in srcrev_internal_helper
    srcrev = ud.method.latest_revision(ud, d, name)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/__init__.py", line 1536, in latest_revision
    revs[key] = rev = self._latest_revision(ud, d, name)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/bitbake/lib/bb/fetch2/git.py", line 581, in _latest_revision
    (ud.unresolvedrev[name], ud.host+ud.path))
bb.fetch2.FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /tmp/tmpgwkjmxuk/gitsource

Stdout:
Running git -c core.fsyncobjectfiles=0 ls-remote file:///tmp/tmpgwkjmxuk/gitsource
Comment 1 Joshua Lock 2017-07-17 14:35:18 UTC 
Not sure if the host OS is useful, but this just failed on CentOS 7
Comment 2 Richard Purdie 2017-07-28 16:57:37 UTC 
This only seems to happen on centos 7. Easy/fast way to make it fail:

bitbake-selftest bb.tests.fetch.GitShallowTest.test_bitbake

$ git --version
git version 1.8.3.1

I'm told git 2.11.0 works.
Comment 3 Christopher Larson 2017-07-28 19:48:57 UTC 
Patch sent to the bitbake-devel list.