Bug 11638 - http_proxy does not work during python build stage
Summary: http_proxy does not work during python build stage
Status: RESOLVED NOTABUG
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Undecided normal
Target Milestone: ---
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-06-09 07:07 UTC by Chanho Park
Modified: 2017-06-16 14:40 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chanho Park 2017-06-09 07:07:53 UTC
All environment variables are empty during build stage. The http_proxy, https_proxy are also empty so some python build does not work inside proxy.

For example, "python3-chardet" is not compiled inside proxy environment.
Below workaround can resolve the issue but I'm not sure whether it is okay.

diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
index 077fddc0eeed..d4e38d5708d3 100644
--- a/bitbake/lib/bb/utils.py
+++ b/bitbake/lib/bb/utils.py
@@ -641,9 +641,10 @@ def empty_environment():
     """
     Remove all variables from the environment.
     """
-    for s in list(os.environ.keys()):
-        os.unsetenv(s)
-        del os.environ[s]
+    if 'BB_PRESERVE_ENV' not in os.environ:
+        for s in list(os.environ.keys()):
+            os.unsetenv(s)
+            del os.environ[s]
 
 def build_environment(d):
     """
Comment 1 Jussi Kukkonen 2017-06-12 08:04:14 UTC
Wouldn't this be a bug in python3-chardet? Why does it need a proxy during build?
Comment 2 Chanho Park 2017-06-12 16:26:07 UTC
Hi, it's not a python3-chardet bug.
For fetch stage, the http_proxy/https_proxy/no_proxy are kept but they have gone in the   build stage.
Many of the companies use the proxy server for caching purpose and I have to use the proxy configuration for in-house network configurations.
Comment 3 Jussi Kukkonen 2017-06-13 07:58:22 UTC
(In reply to comment #2)
> Hi, it's not a python3-chardet bug.
> For fetch stage, the http_proxy/https_proxy/no_proxy are kept but they have
> gone in the   build stage.
> Many of the companies use the proxy server for caching purpose and I have to
> use the proxy configuration for in-house network configurations.

Sure. The question is: why does python3-chardet need network access _after_ fetch? In my opinion it should not -- all required resources should be downloaded during fetch. This still seems like a recipe issue to me.
Comment 4 Richard Purdie 2017-06-15 14:35:52 UTC
I agree with Jussi, network access should not be happening outside of do_fetch. If it does, the manifests and other tools which handle source scanning are incomplete and don't have access to valid data.

This is therefore not a bug. If python3-chardet needs data from the network, it needs to get this in do_fetch.
Comment 5 Chanho Park 2017-06-16 14:40:22 UTC
Hmm. I also agree we have to download the source during the fetch phase.
However, Someone will assume BB_PRESERVE_ENV will keep the environments all of the stage. I found the openobex 1.7.2 version also has same problem. It fetch the xml file during cmake build.