Bug 14736 - setuptools3.bbclass: Add check for pyproject.toml and [build-system] to advise users
Summary: setuptools3.bbclass: Add check for pyproject.toml and [build-system] to advis...
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 4.0
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.1
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2022-02-22 23:00 UTC by Tim Orling
Modified: 2024-08-06 15:30 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Orling 2022-02-22 23:00:08 UTC
Add a function to check for the presence of a pyproject.toml file in the source.
If so check if it has a [build-system]
If so check if it has a build-backend= that matches one of the classes in oe-core (currently flit_core and setuptools_build_meta)
If it matches 'poetry.core.masonry.api', recommend (WIP) class from meta-python
If it is some other value, give a warning.

The intent is to identify recipes which _already_ have a proper pyproject.toml and move them to the new PEP-517 compliant tooling.

The presence of a pyproject.toml is not enough and no guarantee that the source is compliant with PEP-517.
Comment 2 Peter Kjellerstedt 2022-02-24 11:09:24 UTC
You can simplify guess_pip_install_package_name() to:

+def guess_pip_install_package_name(d):
+    '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+    return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_'))
Comment 3 Peter Kjellerstedt 2022-02-24 11:10:47 UTC
Correction:

+def guess_pip_install_package_name(d):
+    '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+    return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')
Comment 4 Tim Orling 2022-02-25 21:02:47 UTC
Merged.
https://git.openembedded.org/openembedded-core/commit/?id=9fc8e55892374f1e63b4c995fe1a5539c42d24e0

If there are any further issues, please send an email to oe-core mailing list or file a new bug.
Comment 5 Tim Orling 2022-02-25 21:03:28 UTC
Oops. Wrong patch.
Comment 6 Tim Orling 2022-02-25 23:57:10 UTC
version 3 of the setuptools3.bbclass patch:

https://patchwork.yoctoproject.org/project/oe-core/patch/20220225234317.668778-1-tim.orling@konsulko.com/
Comment 7 Peter Kjellerstedt 2022-02-26 03:06:16 UTC
You can do away with one of the warn*string variables if you use:

warn_string = "The source has a pyproject.toml which declares '%s' as a build backend, please consider 'inherit %s' %sinstead of inheriting setuptools3."

and then either pass '' or 'from meta-python ' as third argument.
Comment 8 Tim Orling 2022-02-28 21:55:24 UTC
The main issue is that we need a TOML parser, e.g. python3-tomli on the host. The reason this worked was because I already had the deb installed.
Comment 9 Peter Kjellerstedt 2022-02-28 23:09:43 UTC
While using something like python3-tomli certainly is more correct, for the purpose of setuptools3.bbclass, reading the file and looking for the first line that matches r'build-backend\s*=\s*"(.*)"' is probably good enough. You can of course also make sure it is inside the "[build-system]" section, but it is most likely not really needed.
Comment 10 Tim Orling 2022-03-10 20:21:20 UTC
New commit from RP fixes Toaster:
https://patchwork.yoctoproject.org/project/bitbake/patch/20220310135344.612089-1-richard.purdie@linuxfoundation.org/

For unknown _reasons_ this is still failing on my local machine (Debian 11 and Python 3.9.2).

However, I can confirm that a web UI build of quilt-native succeeds using master-next and the following toaster-container distro combinations:

ubuntu-18.04
ubutnu-20.04
debian-10
debian-11

That is significant evidence that this issue has been fixed.
Comment 11 Tim Orling 2022-03-10 20:26:22 UTC
and that was the wrong bug again
Comment 12 Tim Orling 2022-10-24 22:45:29 UTC
Needs to be revisited when Python 3.11 (with built-in toml support) lands in "master".
Comment 13 Tim Orling 2023-02-02 16:01:35 UTC
"master" has Python 3.11 now, so time to revisit this.
Comment 14 Tim Orling 2024-02-20 16:59:42 UTC
Bumping to 5.1, since we are too late in the 5.0M3 cycle to be introducing new warnings.

Changing to IN PROGRESS IMPLEMENTATION, since the regex check (not needing tomli nor tomllib on the host python) has not yet been implemented.
Comment 15 Tim Orling 2024-02-20 17:02:36 UTC
WIP branch with a check for tomllib and then falling back to tomli (but not then falling back to regex yet):
https://git.yoctoproject.org/poky-contrib/log/?h=timo/setuptools3-pyproject.toml-check