Bug 8864

Summary: git fetcher does not support non git url format based requests
Product: [Build System, Metadata & Runtime] BitBake Reporter: claude
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher
Version: 2.2   
Target Milestone: 2.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
patch to make the changes as noted above. none

Description claude 2015-12-31 15:49:38 UTC
Created attachment 2914 [details]
patch to make the changes as noted above.

Preconditions/Environment
-------------------------
create a private, secured git repository (e.g. bitbucket private repository)
add ssh key to access repository
create a x.bb file with
SRC_URI = "git://git@bitbucket.org:<username>/<repository>.git;rev=foo"


Triggering Action/Cause
-----------------------
execute bitbake x

Expectation
-----------
code should be checked out

Actual Result
-------------
DEBUG: Fetcher accessed the network with the command git -c core.fsyncobjectfiles=0 ls-remote git://git@bitbucket.org:claudenw/lion.git

ERROR: Function failed: Fetcher failure: Fetch command failed with exit code 128, output: 
 fatal: unable to connect to git@bitbucket.org:
 git@bitbucket.org: Servname not supported for ai_socktype

The error is that the git command does not like the git:// on the url within the git command.  

setting protocol=ssh
yeilds:
DEBUG: Fetcher accessed the network with the command git -c core.fsyncobjectfiles=0 ls-remote ssh://git@bitbucket.org:claudenw/lion.git

ERROR: Function failed: Fetcher failure: Fetch command failed with exit code 128, output:
 conq: invalid repository syntax.
 fatal: Could not read from remote repository.

Reproducibility
---------------
100%

SOLUTION
--------

Need a mechanism to remove the git:// (ssh://) from the url presented to the internal git command.  this can be done by modifying git.py  (patch attached)

line 104 add 'none' as a valid protocol type

change line 333 to be 
        if ud.proto != 'none':  
            return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)
        return "%s%s%s" % (username, ud.host, ud.path)

This adds protocol=none as a valid option to the URL.

SRC_URI = "git://git@bitbucket.org:<username>/<repository>.git;rev=foo;protocol=none"

then works as expected.
Comment 1 Richard Purdie 2016-01-07 11:25:56 UTC
I don't like the idea of a "none" type, not sure that makes sense since it really is using the ssh backend. Two different syntaxes for the same thing is bad.

It appears its actually the bitbucket server which can't cope with this (which is why there is a mention of conq in the error message).

Your patch happens to work by having the ":" in the SRC_URI when combined with the protocol=none parameter which is also a bit ugly.

I'm wondering if a better solution is simple to modify _get_repo_url like this:

             username = ""
+        if ud.proto == "ssh":
+            # Some servers, e.g. bitbucket.org can't cope with ssh://
+            # and removing that means we need a : before path.
+            return "%s%s:%s" % (username, ud.host, ud.path)
         return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)

which then change all ssh urls to use the other syntax. This is more in line with what people use from the commandline.

I would maintain that its the bitbucket server which is broken here mind since git does usually accept this syntax, e.g.:

$ git ls-remote ssh://git@git.yoctoproject.org/psplash 
88343ad23c90fa1dd8d79ac0d784a691aa0c6d2b	HEAD
88343ad23c90fa1dd8d79ac0d784a691aa0c6d2b	refs/heads/master
Comment 3 Richard Purdie 2016-01-15 15:06:21 UTC
Upon further investigation we've reverted the patch. I suspect the original problem was a ":" in the url where there should be a "/" character. If you continue to have issues when using a "/" and protocol=ssh, please reopen the bug.