I am using "openembedded-core/scripts/install-buildtools" to install buildtools because GCC is too old the host system. That also installs Python 3. The environment file also sets GIT_SSL_CAINFO, SSL_CERT_FILE, OPENSSL_CONF, but those are not exported into task environment. I tried using BB_ENV_EXTRAWHITE, but that doesn't seem to work in this case. get_waf() function will not have those variables in its build environment: https://github.com/openembedded/meta-openembedded/blob/master/meta-oe/recipes-multimedia/mplayer/mpv_0.32.0.bb#L106 Thus it cannot download WAF due to SSL issue. Note that if I just go into mpv build directory ./bootstrap.py works fine. If I manually add variables (i.e. modify the mpv recipe) it works just fine. Simply put for the Python 3 from build tools to function properly in the recipe build environment it also needs SSL variables.
In addition to that shouldn't mpv depend on python-native to avoid using Python 3 from the build tools?
(In reply to comment #1) > In addition to that shouldn't mpv depend on python-native to avoid using > Python 3 from the build tools? bitbake expect py3 > 3.5 to be on your build host. So using py3 from host is ok here. SSL env not being correct perhaps is the issue we should look into instead of trying to use py3-native here.
Yeah, I tried adding python3-native to DEPENDS, but that didn't help. So after install Yocto build tools: $ type -a python3 python3 is /blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin/python3 python3 is /usr/bin/python3 environment-setup-x86_64-pokysdk-linux file contains: export PATH=/blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux/usr/bin:$PATH #OECORE_NATIVE_SYSROOT="/blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux" export GIT_SSL_CAINFO="/blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux/etc/ssl/certs/ca-certificates.crt" export SSL_CERT_FILE="/blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux/etc/ssl/certs/ca-certificates.crt" export OPENSSL_CONF="/blah/openembedded-core/buildtools/sysroots/x86_64-pokysdk-linux/etc/ssl/openssl.cnf" and bootstrap.py works fine. Adding 'env' to get_waf() function shows that these are not available before bootstrap.py is called and thus SSL errors (even for HTTP download). /blah/build/tmp-glibc/work/riscv64-oe-linux/mpv/0.32.0-r0/git$ ./bootstrap.py Downloading https://waf.io/waf-2.0.9... Checksum verified. /blah/build/tmp-glibc/work/riscv64-oe-linux/mpv/0.32.0-r0/git$ unset OPENSSL_CONF /blah/build/tmp-glibc/work/riscv64-oe-linux/mpv/0.32.0-r0/git$ unset SSL_CERT_FILE /blah/build/tmp-glibc/work/riscv64-oe-linux/mpv/0.32.0-r0/git$ rm waf /blah/build/tmp-glibc/work/riscv64-oe-linux/mpv/0.32.0-r0/git$ ./bootstrap.py Downloading https://waf.io/waf-2.0.9... Download failed. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108) Downloading http://www.freehackers.org/~tnagy/release/waf-2.0.9... Download failed. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108) Could not download waf-2.0.9. Note: I modified bootstrap.py to print exception message. run.get_waf script doesn't incl. GIT_SSL_CAINFO, SSL_CERT_FILE, OPENSSL_CONF. So far I couldn't figure out how to force bitbake to incl. these variables into script environment.
We suspect that we need to update the certs in the build tools or do something else.
Also, the recipe downloading waf that way is *horrible*.
Updating the buildtools doesn't help. Problem can be reproduced outside of bitbake by simply sourcing the buildtools and unsetting SSL_CERT_FILE so that libcrypto doesn't know where the certs are.
There is a problem with buildtools, but the bigger problem is that the mpv recipe is downloading outside of do_fetch.
(In reply to comment #7) > There is a problem with buildtools, but the bigger problem is that the mpv > recipe is downloading outside of do_fetch. mpv is perhaps not doing it right but that is besides the point and perhaps should be a separate issue to track. Here we should only consider buildtools tarball issue.
Patches sent to oe-core/meta-oe to fix the mpv recipe so it isn't doing something horrible, which solves the problem in the mpv recipe. The underlying problem that this exposes is still an issue, we have the same issue for native recipes too.
Trivially unsolvable: we want to prune the environment but buildtools libcrypto needs to be told where the certs are. Same problem with have for native use of libcrypto (typically via urllib2, as with mpv). Maybe making relocating path sections won't be too difficult but I suspect it would be a pain.
Ross: Why can't we leave SSL_CERT_FILE set from the buildtools environment?
I'd note that the bitbake fetcher does have: lib/bb/fetch2/__init__.py: 'GIT_SSL_CAINFO' which is perhaps why bitbake's own fetch tasks are handling this correctly but an individual recipe download isn't working (and is a bug for the reasons discussed?)
My patch to fix mpv to fetch in do_fetch has been merged in oe-core 5af46f89fcef5c436786ed81978de60f26abe054. Fetching outside of do_fetch not recommended, proxies won't be set either, caching won't work, etc. We don't want to pass through from the host every environment variable needed as that destroys host isolation. We can pass it through to the fetcher (as we do with other variables already), but this use-case doesn't use the fetcher. To future readers, if you *really* want to do something like this then you'll need to grab the value from BB_ORIGENV and add it to a local environment. Something like this should work: export SSL_CERT_FILE="${@d.getVar("BB_ORIGENV").getVar("SSL_CERT_FILE")}"