Bug 7567

Patch sent to the OE mailing list.

patch already sent to the OE mailing list, just waiting for its revision/merge. Moving it to 2.0.

any progress on patch after one week ?

Patch merged on master:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=12cd705b28412c1342c1c022340c13b1ef2c8a71

Is this code ever being tested in real environment where proxy is the only way out to internet ?

I'm observing where it still tries to connect directly, even if proxy variables are present.

$ netstat -tnap | grep SYN
tcp        0      1 10.xx.xx.xxx:56652     130.89.148.12:80        SYN_SENT    20862/python


After a bit more looking inside, it looks like problem is in argument that is passed to 
urllib.urlopen()


it passes dictionary like this:

{'HTTP_PROXY': None, 'ALL_PROXY': None, 'https_proxy': 'http://proxy-chain:911', 'NO_PROXY': None, 'all_proxy': None, 'http_proxy': 'http://proxy-chain:911', 'ftp_proxy': 'http://proxy-chain:911', 'HTTPS_PROXY': None, 'FTP_PROXY': None, 'FTPS_PROXY': None, 'ftps_proxy': None, 'no_proxy': 'localhost, 127.0.0.1'}

Where, in reality, urlopen() expects proxies dictionary in format:
proxies = {'http': 'http://www.someproxy.com:3128'}

https://docs.python.org/2/library/urllib.html#urllib.urlopen

function get_proxies(d) should be looking something like this to work correctly:

def get_proxies(d):
    proxies = {}
    for proto in [ 'http', 'https', 'ftp']:
        for key in [ proto+'_proxy', proto.upper()+'_PROXY']:
            proxy = d.getVar(key, True)
            if proxy:
               proxies[proto] = proxy
               break
    return proxies

(In reply to comment #5)
> Is this code ever being tested in real environment where proxy is the only
> way out to internet ?

I did test it. Let me check again and provide a patch if necessary.

(In reply to comment #5)
> Is this code ever being tested in real environment where proxy is the only
> way out to internet ?
> 
> I'm observing where it still tries to connect directly, even if proxy
> variables are present.
> 
> $ netstat -tnap | grep SYN
> tcp        0      1 10.xx.xx.xxx:56652     130.89.148.12:80        SYN_SENT 
> 20862/python
> 
> 
> After a bit more looking inside, it looks like problem is in argument that
> is passed to 
> urllib.urlopen()
> 
> 
> it passes dictionary like this:
> 
> {'HTTP_PROXY': None, 'ALL_PROXY': None, 'https_proxy':
> 'http://proxy-chain:911', 'NO_PROXY': None, 'all_proxy': None, 'http_proxy':
> 'http://proxy-chain:911', 'ftp_proxy': 'http://proxy-chain:911',
> 'HTTPS_PROXY': None, 'FTP_PROXY': None, 'FTPS_PROXY': None, 'ftps_proxy':
> None, 'no_proxy': 'localhost, 127.0.0.1'}
> 
> Where, in reality, urlopen() expects proxies dictionary in format:
> proxies = {'http': 'http://www.someproxy.com:3128'}
> 
> https://docs.python.org/2/library/urllib.html#urllib.urlopen
> 
> function get_proxies(d) should be looking something like this to work
> correctly:
> 
> def get_proxies(d):
>     proxies = {}
>     for proto in [ 'http', 'https', 'ftp']:
>         for key in [ proto+'_proxy', proto.upper()+'_PROXY']:
>             proxy = d.getVar(key, True)
>             if proxy:
>                proxies[proto] = proxy
>                break
>     return proxies

Alex,

you are right. If proxies are passed through the environment, then the variables end with '_proxies', if these are passed through a python dictionary, the keys do not have this world. Sending a fix today.

Patch sent to the ML:

http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111390.html

Fixed in oe-core 92fb327b08e4f9cfb2400610c2983afe579b3592.

Verified on 8c8c4ede3ff5ac8ca739020d9efed91159acccc1

Tested on a network with proxy, adding 'INHERIT += "distrodata"' on local.conf
with the following commands

    for distrotask in distrodataall distro_checkall checklicenseall; do
        bitbake core-image-minimal -c $distrotask
    done