13157 2019-01-29 16:52:16 +0000 Bitbake fails to fetch from a premirror that doesn't support https 2019-05-30 15:24:16 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 2.7 x86 Multiple RESOLVED OBSOLETE Medium normal 2.7 1 maxime.roussinbelanger maxime.roussinbelanger poky.bs.watcher poky.watcher randy.macleod --- --- No (bug/feature does not impact docs) oldest_to_newest 82892 0 maxime.roussinbelanger 2019-01-29 16:52:16 +0000 Preconditions/Environment ------------------------- A PREMIRROR that doesn't support https Triggering Action/Cause ----------------------- bitbake a recipe that uses https protocol and fetches from github. Expectation ----------- Try to redirect to http and download the archive? Skip the PREMIRROR and download from upstream... Actual Result ------------- wget fails when trying to fetch from the premirror because https is reserved for a login page. wget gets redirected to the login page, download the HTML, tries to untar the html page, then obviously fails to untar it. I was trying to build tools inside meta-clang layer. URL transformed to HTTPS due to an HSTS policy is a warning given by wget. Reproducibility --------------- 5/5 82907 1 maxime.roussinbelanger 2019-01-30 18:52:23 +0000 A real-life example of the problem http://errors.yoctoproject.org/Errors/Details/220286/ 82918 2 richard.purdie 2019-01-31 16:22:15 +0000 I'm not sure this is a bitbake problem but a misconfigured server. The logs show a fetch of http://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2 which redirected to https://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2. The fact that this redirected url is invalid is an issue with the hosting of those files. I tried this locally and it worked so perhaps you could retest please? Perhaps the server issue was fixed? 82919 3 maxime.roussinbelanger 2019-01-31 17:03:57 +0000 The redirect URL isn't invalid, you can see that the file was downloaded without problem: 2019-01-30 00:25:44 (3.51 MB/s) - ‘TOPDIR/../downloads/libeigen-3.3.7.tar.bz2’ saved [1665168/1665168] But then for some reason, the fetcher can't find it? How can that happen?! " The fetch command returned success for url http://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2 but TOPDIR/../downloads/libeigen-3.3.7.tar.bz2 doesn't exist?! " I was not able to reproduce the error ever on local. It only did it on the remote build machine that yocto is using. When I sent the patch I made sure that it worked on local first. The only error I was able to create was when I trying to build a recipe from the meta-clang layer that had the protocol set to https. But my company PREMIRROR that had the cache tarball doesn't support HTTPS. But you can download the tarball in HTTP no problem. HTTPS on the PREMIRROR is reserved for a login page. That's probably another issue though. 82924 4 maxime.roussinbelanger 2019-01-31 19:02:53 +0000 To get around the redirection of HTTP -> HTTPS, I had to modify the FETCHCMD_wget with FETCHCMD_wget = "/usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate --no-hsts" I added the "--no-hsts" part. 82925 5 maxime.roussinbelanger 2019-01-31 19:03:34 +0000 To get around the redirection of HTTP -> HTTPS, I had to modify the FETCHCMD_wget with FETCHCMD_wget = "/usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate --no-hsts" I added the "--no-hsts" part. 82948 6 sjolley.yp.pm 2019-02-02 04:03:30 +0000 Richard, It appears that Maxime has answered your question. I am moving it to Accepted unless you have other questions. 84015 7 randy.macleod 2019-05-30 15:24:16 +0000 It appears that his was a problem with the local network configuration. If it is not, please open a new more clear bug describing the issue.