Bug 14610

Summary: Stop using python distutils
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Khem Raj <raj.khem>
Component: coreAssignee: Tim Orling <tim.orling>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: meta.mr.watcher, meta.watcher, randy.macleod, TicoTimo, tvgamblin
Version: unspecified   
Target Milestone: 4.0 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Khem Raj 2021-10-22 12:37:59 UTC
distutils is marked deprecated in py 3.10 and will be removed in py 3.12, therefore its time to start reming distutils use in oe-core and bitbake see 

https://www.python.org/dev/peps/pep-0632/
Comment 1 Randy MacLeod 2021-10-28 14:38:53 UTC
Start by moving the distutils class from oe-core to meta-python?
Comment 2 Tim Orling 2021-10-28 14:56:47 UTC
(1) Expand setuptools3 class to do the bits that it inherits from distutils classes as a start.
(2) Build oe-core python modules without distutils and see if there are any problems.
(3) Build meta-python and see the same.
(4) As a stop gap, move distutils classes to meta-python.

Eventually, we will most likely have to figure out a way of using wheels. Some packaging is not even using setuptools anymore and that trend is likely to increase.
Comment 3 Tim Orling 2021-10-28 17:11:05 UTC
waf.bbclass and sanity.bbclass both import distutils, so it is not only the python3 module building infrastructure (distutils*.bbclass) that needs to change. Other upstream things like glib-2.0, gpgme and bind also import or use methods from distutils.
Comment 4 Tim Orling 2021-10-28 17:16:26 UTC
Porting from Distutils:
https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html

PEP 632 -- Deprecate distutils module:
https://www.python.org/dev/peps/pep-0632/

Packaging and distributing projects:
https://packaging.python.org/guides/distributing-packages-using-setuptools/
Comment 5 Tim Orling 2021-10-28 17:30:17 UTC
Things in scripts/ also need to change: wic, buildhistory-diff, recipetool
Comment 6 Tim Orling 2021-11-26 04:40:19 UTC
RFC series sent to oe-core list:

https://lore.kernel.org/openembedded-core/cover.1637900380.git.timothy.t.orling@intel.com/T/#t
Comment 7 Tim Orling 2021-11-26 05:20:23 UTC
Import distutils.version to lib/bb/version

https://lore.kernel.org/bitbake-devel/20211126051800.2364615-1-timothy.t.orling@intel.com/T/#u

This is a standalone with implementations of LooseVersion and StrictVersion which are not replicated elsewhere.
Comment 9 Tim Orling 2022-01-18 20:49:38 UTC
Docs changes are merged as are the fixes to meta-openembedded (and meta-python2). It takes a village :)