Overview: PyPI has changed the way you download content (Over two years ago, in fact...) so that if you use the current bbclass to download a new version of a recipe or add a new recipe leveraging something from out of PyPI, you will get a 404 error on the download. This is due to a CDN hashing algorithm that is completely different from what is used in the recipe for the URI generation. Steps to Reproduce: Attempt to download a version not in the current global downloads cache or add a new packaging recipe for a new PyPI package. Actual Results: 404 error for fetcher download attempt. Expected Results: Download of the archive file with sources to be set up with setuptools. Builds affected: ANYTHING using pypi.bbclass at this time as the old method will no longer work. Needs to have fixes applied to multiple versions. Happened with me on Warrior and Zeus with a Tegra and a RaspberryPI 4 target. Root Cause: The following block has the issue- def pypi_src_uri(d): package = d.getVar('PYPI_PACKAGE') package_ext = d.getVar('PYPI_PACKAGE_EXT') pv = d.getVar('PV') return 'https://files.pythonhosted.org/packages/source/%s/%s/%s-%s.%s' % (package[0], package, package, pv, package_ext) The URI used in the return is no longer valid. They use a "simple" interface or similar which provides a series of parsable links, checksums, etc. that pip and similar tools use to do their download operations against. A line-item link dump is as follows: <!DOCTYPE html> <html> <head> <title>Links for PyGTK</title> </head> <body> <h1>Links for PyGTK</h1> <a href="https://files.pythonhosted.org/packages/7c/18/fa4f2de77500dd62a314fd845ff6e903ac2ce551164cb421c5750969f799/pygtk-2.24.0.tar.bz2#sha256=cd1c1ea265bd63ff669e92a2d3c2a88eb26bcd9e5363e0f82c896e649f206912">pygtk-2.24.0.tar.bz2</a><br/> <a href="https://files.pythonhosted.org/packages/e3/f0/8b842afbfef3803d01e54951a6b06118a06a4192d0f7ca53151a9d97f25e/pygtk-2.24.0.win32-py2.6.msi#sha256=f7919aa9fe2db45a4f176763fccc4bd7ae1198c6c77698f8469bda106ccdf15d">pygtk-2.24.0.win32-py2.6.msi</a><br/> <a href="https://files.pythonhosted.org/packages/f3/0c/b733dca9201c550935e3e1246ed3657e47bd7e9378ae2154d942d4f6b500/pygtk-2.24.0.win32-py2.7.msi#sha256=b3c6a3f1484acb3dce19e99cf8321fe292846e969fda71289b5bfee0c0cfc30d">pygtk-2.24.0.win32-py2.7.msi</a><br/> <a href="https://files.pythonhosted.org/packages/85/52/3d9bb924bd2c9bdb8afd9b7994cd09160fad5948bb2eca18fd7ffa12cfdc/pygtk-2.24.0.win32-py2.6.exe#sha256=16336e79f9a7913e5b2d1cf50120896495aac8892be2d352f660b905205c48db">pygtk-2.24.0.win32-py2.6.exe</a><br/> </body> </html> <!--SERIAL 265829--> This is fairly straightforward to parse, but I was time constrained so I did the following hack in a new .bbclass: def pypi_src_uri(d): # Bring in requests... We're going to need it to get the actual URI # for the package... import os package = d.getVar('PYPI_PACKAGE') package_ext = d.getVar('PYPI_PACKAGE_EXT') pv = d.getVar('PV') # NASTY, UGLY **HACK** ALERT!! - Use a response fetch from wget to get the official response from # PyPI's simple interface and then crack it apart from there to get the actual URL... cmd = os.popen('wget -q -O - https://pypi.org/simple/%s/ | awk \'BEGIN{FS="[\\"#]"} /%s-%s.%s/ {print $2}\'' % (package, package, pv, package_ext)) actual_uri = cmd.read() cmd.close() # Check for a blind spot. We're grabbing a redirector URI. This means the app server won't hand us anything # useful for a result that we can pick off if version or name is wrong. So flag this and DIE hard... if not actual_uri : bb.fatal("pypi-new : Unable to find %s-%s.%s on PyPI." % (package, pv, package_ext)) return actual_uri
Zeus could use with: https://git.openembedded.org/openembedded-core/commit/meta/classes/pypi.bbclass?h=dunfell&id=0603f6d9f2abfa67b99b1bc39228f6aa16a0370d https://git.openembedded.org/openembedded-core/commit/meta/classes/pypi.bbclass?h=dunfell&id=e5f3f961242d888f3f786af8f793bf1d247fdff0 will have to send patches to list.
patches sent to list https://patchwork.openembedded.org/patch/175156/ https://patchwork.openembedded.org/patch/175157/
in zeus 3.0.4.
closing