7567 2015-04-03 18:27:25 +0000 distrodata/distro_check don't take into account proxies 2016-01-22 07:55:36 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.9 x86 Multiple VERIFIED FIXED Medium enhancement 2.0 1 anibal.limon leonardo.sandoval.gonzalez alexander.kanevskiy anibal.limon benjamin.esquivel bogdanx.a.voiculescu meta.mr.watcher meta.watcher ross.burton bogdanx.a.voiculescu --- --- No (bug/feature does not impact docs) oldest_to_newest 50080 0 anibal.limon 2015-04-03 18:27:25 +0000 When try to run distro_check it needs to manual specify proxy exports into local.conf. It needs to do automatically like fetcher. 53643 1 leonardo.sandoval.gonzalez 2015-09-02 17:15:12 +0000 Patch sent to the OE mailing list. 54398 2 leonardo.sandoval.gonzalez 2015-09-21 16:15:47 +0000 patch already sent to the OE mailing list, just waiting for its revision/merge. Moving it to 2.0. 54838 3 alexander.kanevskiy 2015-09-30 15:01:32 +0000 any progress on patch after one week ? 55028 4 leonardo.sandoval.gonzalez 2015-10-07 13:44:40 +0000 Patch merged on master: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=12cd705b28412c1342c1c022340c13b1ef2c8a71 55066 5 alexander.kanevskiy 2015-10-08 13:41:13 +0000 Is this code ever being tested in real environment where proxy is the only way out to internet ? I'm observing where it still tries to connect directly, even if proxy variables are present. $ netstat -tnap | grep SYN tcp 0 1 10.xx.xx.xxx:56652 130.89.148.12:80 SYN_SENT 20862/python After a bit more looking inside, it looks like problem is in argument that is passed to urllib.urlopen() it passes dictionary like this: {'HTTP_PROXY': None, 'ALL_PROXY': None, 'https_proxy': 'http://proxy-chain:911', 'NO_PROXY': None, 'all_proxy': None, 'http_proxy': 'http://proxy-chain:911', 'ftp_proxy': 'http://proxy-chain:911', 'HTTPS_PROXY': None, 'FTP_PROXY': None, 'FTPS_PROXY': None, 'ftps_proxy': None, 'no_proxy': 'localhost, 127.0.0.1'} Where, in reality, urlopen() expects proxies dictionary in format: proxies = {'http': 'http://www.someproxy.com:3128'} https://docs.python.org/2/library/urllib.html#urllib.urlopen function get_proxies(d) should be looking something like this to work correctly: def get_proxies(d): proxies = {} for proto in [ 'http', 'https', 'ftp']: for key in [ proto+'_proxy', proto.upper()+'_PROXY']: proxy = d.getVar(key, True) if proxy: proxies[proto] = proxy break return proxies 55067 6 leonardo.sandoval.gonzalez 2015-10-08 13:50:47 +0000 (In reply to comment #5) > Is this code ever being tested in real environment where proxy is the only > way out to internet ? I did test it. Let me check again and provide a patch if necessary. 55068 7 leonardo.sandoval.gonzalez 2015-10-08 14:06:52 +0000 (In reply to comment #5) > Is this code ever being tested in real environment where proxy is the only > way out to internet ? > > I'm observing where it still tries to connect directly, even if proxy > variables are present. > > $ netstat -tnap | grep SYN > tcp 0 1 10.xx.xx.xxx:56652 130.89.148.12:80 SYN_SENT > 20862/python > > > After a bit more looking inside, it looks like problem is in argument that > is passed to > urllib.urlopen() > > > it passes dictionary like this: > > {'HTTP_PROXY': None, 'ALL_PROXY': None, 'https_proxy': > 'http://proxy-chain:911', 'NO_PROXY': None, 'all_proxy': None, 'http_proxy': > 'http://proxy-chain:911', 'ftp_proxy': 'http://proxy-chain:911', > 'HTTPS_PROXY': None, 'FTP_PROXY': None, 'FTPS_PROXY': None, 'ftps_proxy': > None, 'no_proxy': 'localhost, 127.0.0.1'} > > Where, in reality, urlopen() expects proxies dictionary in format: > proxies = {'http': 'http://www.someproxy.com:3128'} > > https://docs.python.org/2/library/urllib.html#urllib.urlopen > > function get_proxies(d) should be looking something like this to work > correctly: > > def get_proxies(d): > proxies = {} > for proto in [ 'http', 'https', 'ftp']: > for key in [ proto+'_proxy', proto.upper()+'_PROXY']: > proxy = d.getVar(key, True) > if proxy: > proxies[proto] = proxy > break > return proxies Alex, you are right. If proxies are passed through the environment, then the variables end with '_proxies', if these are passed through a python dictionary, the keys do not have this world. Sending a fix today. 55090 8 leonardo.sandoval.gonzalez 2015-10-08 19:44:10 +0000 Patch sent to the ML: http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111390.html 56243 9 ross.burton 2015-11-05 16:27:38 +0000 Fixed in oe-core 92fb327b08e4f9cfb2400610c2983afe579b3592. 58122 10 bogdanx.a.voiculescu 2016-01-22 07:55:36 +0000 Verified on 8c8c4ede3ff5ac8ca739020d9efed91159acccc1 Tested on a network with proxy, adding 'INHERIT += "distrodata"' on local.conf with the following commands for distrotask in distrodataall distro_checkall checklicenseall; do bitbake core-image-minimal -c $distrotask done