| Summary: | Can bitbake provide a setup.py or pyproject.toml to package it like gentoo portage | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] BitBake | Reporter: | wuzhenyu <wuzhenyu> |
| Component: | bitbake | Assignee: | Richard Purdie <richard.purdie> |
| Status: | RESOLVED NOTABUG | QA Contact: | |
| Severity: | normal | ||
| Priority: | Undecided | CC: | poky.bs.watcher, poky.watcher, ross.burton |
| Version: | unspecified | ||
| Target Milestone: | 0.0.0 | ||
| Hardware: | All | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Yes (doc changes required) | |
|
Description
wuzhenyu@ustc.edu
2023-06-06 15:28:47 UTC
```diff From d8edeb24a63d467e8084a3b464bccd3271e1f9a5 Mon Sep 17 00:00:00 2001 From: Wu Zhenyu <wuzhenyu@ustc.edu> Date: Wed, 7 Jun 2023 00:21:01 +0800 Subject: [PATCH] support package use `python -m build` to build package set PYPI_API_TOKEN to upload it to PYPI --- .github/workflows/python.yml | 44 ++++++++++++ .gitignore | 128 ++++++++++++++++++++++++++++++++++- lib/bb/server/process.py | 5 +- pyproject.toml | 108 +++++++++++++++++++++++++++++ 4 files changed, 282 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/python.yml create mode 100644 pyproject.toml diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml new file mode 100644 index 0000000..ab3f7ba --- /dev/null +++ b/.github/workflows/python.yml @@ -0,0 +1,44 @@ +--- +"on": + push: + pull_request: + workflow_dispatch: + +# https://github.com/softprops/action-gh-release/issues/236 +permissions: + contents: write + +env: + PYTHONUTF8: "1" + python-version: 3.x + cache: pip + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-python@v4 + with: + python-version: ${{env.python-version}} + cache: ${{env.cache}} + - name: Install dependencies + run: | + pip install build + - name: Build + run: | + python -m build + - uses: pypa/gh-action-pypi-publish@release/v1 + if: startsWith(github.ref, 'refs/tags/') + with: + password: ${{secrets.PYPI_API_TOKEN}} + - uses: actions/upload-artifact@v3 + if: "! startsWith(github.ref, 'refs/tags/')" + with: + path: | + dist/* + - uses: softprops/action-gh-release@v1 + if: startsWith(github.ref, 'refs/tags/') + with: + files: | + dist/* diff --git a/.gitignore b/.gitignore index fbf9eb2..c646b0f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,3 @@ -*.pyc -*.pyo *~ pyshtables.py /doc/manual/html/ @@ -16,3 +14,129 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log + +# Python.gitignore: +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +pip-wheel-metadata/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +.python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# celery beat schedule file +celerybeat-schedule + +# SageMath parsed files +*.sage.py + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 7616ef5..e5d1cbd 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -32,6 +32,7 @@ import stat import bb.server.xmlrpcserver from bb import daemonize from multiprocessing import queues +from shutil import which logger = logging.getLogger('BitBake') @@ -619,7 +620,9 @@ class BitBakeServer(object): os.close(self.readypipe) os.set_inheritable(self.bitbake_lock.fileno(), True) os.set_inheritable(self.readypipein, True) - serverscript = os.path.realpath(os.path.dirname(__file__) + "/../../../bin/bitbake-server") + serverscript = which("bitbake-server") + if serverscript is None: + serverscript = os.path.realpath(os.path.dirname(__file__) + "/../../../bin/bitbake-server") os.execl(sys.executable, "bitbake-server", serverscript, "decafbad", str(self.bitbake_lock.fileno()), str(self.readypipein), self.logfile, self.bitbake_lock.name, self.sockname, str(self.server_timeout or 0), str(int(self.profile)), str(self.xmlrpcinterface[0]), str(self.xmlrpcinterface[1])) def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpcinterface, profile): diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..f189284 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,108 @@ +[build-system] +requires = ["setuptools_scm[toml] >= 6.2"] +build-backend = "setuptools.build_meta" + +# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html +[project] +name = "bitbake" +description = "" +readme = "README.md" +requires-python = ">= 3.7" +keywords = ["yocto"] +version = "2.4.1" +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "Topic :: Software Development :: Build Tools", + "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", + "Operating System :: Microsoft :: Windows", + "Operating System :: POSIX", + "Operating System :: Unix", + "Operating System :: MacOS", + "Programming Language :: Python :: 3 :: Only", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: Implementation :: CPython", + "Programming Language :: Python :: Implementation :: PyPy", +] +dynamic = ["optional-dependencies"] + +[[project.authors]] +name = "Tim Ansell" +email = "mithro@mithis.net" + +[[project.authors]] +name = "Phil Blundell" +email = "pb@handhelds.org" + +[[project.authors]] +name = "Seb Frankengul" +email = "seb@frankengul.org" + +[[project.authors]] +name = "Holger Freyther" +email = "holger@moiji-mobile.com" + +[[project.authors]] +name = "Marcin Juszkiewicz" +email = "marcin@juszkiewicz.com.pl" + +[[project.authors]] +name = "Chris Larson" +email = "kergoth@handhelds.org" + +[[project.authors]] +name = "Ulrich Luckas" +email = "luckas@musoft.de" + +[[project.authors]] +name = "Mickey Lauer" +email = "mickey@Vanille.de" + +[[project.authors]] +name = "Richard Purdie" +email = "rpurdie@rpsys.net" + +[[project.authors]] +name = "Holger Schurig" +email = "holgerschurig@gmx.de" + +[project.license] +text = "GPL v2" + +[project.urls] +Homepage = "https://bitbake.readthedocs.io" +Download = "https://github.com/Freed-Wu/bitbake/releases" +"Bug Report" = "https://github.com/Freed-Wu/bitbake/issues" +Source = "https://github.com/Freed-Wu/bitbake" + +[tool.setuptools] +script-files = [ + "bin/bitbake", + "bin/bitbake-diffsigs", + "bin/bitbake-dumpsig", + "bin/bitbake-getvar", + "bin/bitbake-hashclient", + "bin/bitbake-hashserv", + "bin/bitbake-layers", + "bin/bitbake-prserv", + "bin/bitbake-selftest", + "bin/bitbake-server", + "bin/bitbake-worker", + "bin/git-make-shallow", + "bin/toaster", + "bin/toaster-eventreplay", +] + +[tool.setuptools.packages.find] +where = ["lib"] + +[tool.setuptools.data-files] +"share/man/man1" = ["doc/bitbake.1"] + +[tool.setuptools.dynamic.optional-dependencies.toaster] +file = "toaster-requirements.txt" -- 2.40.1 ``` Sorry, this is right: ```diff From 83220c57dd7075e6836d890d6114c2a9b6584c0e Mon Sep 17 00:00:00 2001 From: Wu Zhenyu <wuzhenyu@ustc.edu> Date: Wed, 7 Jun 2023 00:21:01 +0800 Subject: [PATCH] support package use `python -m build` to build package set PYPI_API_TOKEN to upload it to PYPI --- .github/workflows/python.yml | 44 ++++++++++++ .gitignore | 128 ++++++++++++++++++++++++++++++++++- lib/bb/server/process.py | 5 +- pyproject.toml | 108 +++++++++++++++++++++++++++++ 4 files changed, 282 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/python.yml create mode 100644 pyproject.toml diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml new file mode 100644 index 0000000..ab3f7ba --- /dev/null +++ b/.github/workflows/python.yml @@ -0,0 +1,44 @@ +--- +"on": + push: + pull_request: + workflow_dispatch: + +# https://github.com/softprops/action-gh-release/issues/236 +permissions: + contents: write + +env: + PYTHONUTF8: "1" + python-version: 3.x + cache: pip + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - uses: actions/setup-python@v4 + with: + python-version: ${{env.python-version}} + cache: ${{env.cache}} + - name: Install dependencies + run: | + pip install build + - name: Build + run: | + python -m build + - uses: pypa/gh-action-pypi-publish@release/v1 + if: startsWith(github.ref, 'refs/tags/') + with: + password: ${{secrets.PYPI_API_TOKEN}} + - uses: actions/upload-artifact@v3 + if: "! startsWith(github.ref, 'refs/tags/')" + with: + path: | + dist/* + - uses: softprops/action-gh-release@v1 + if: startsWith(github.ref, 'refs/tags/') + with: + files: | + dist/* diff --git a/.gitignore b/.gitignore index fbf9eb2..c646b0f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,3 @@ -*.pyc -*.pyo *~ pyshtables.py /doc/manual/html/ @@ -16,3 +14,129 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log + +# Python.gitignore: +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +pip-wheel-metadata/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +.python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# celery beat schedule file +celerybeat-schedule + +# SageMath parsed files +*.sage.py + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py index 7616ef5..e5d1cbd 100644 --- a/lib/bb/server/process.py +++ b/lib/bb/server/process.py @@ -32,6 +32,7 @@ import stat import bb.server.xmlrpcserver from bb import daemonize from multiprocessing import queues +from shutil import which logger = logging.getLogger('BitBake') @@ -619,7 +620,9 @@ class BitBakeServer(object): os.close(self.readypipe) os.set_inheritable(self.bitbake_lock.fileno(), True) os.set_inheritable(self.readypipein, True) - serverscript = os.path.realpath(os.path.dirname(__file__) + "/../../../bin/bitbake-server") + serverscript = which("bitbake-server") + if serverscript is None: + serverscript = os.path.realpath(os.path.dirname(__file__) + "/../../../bin/bitbake-server") os.execl(sys.executable, "bitbake-server", serverscript, "decafbad", str(self.bitbake_lock.fileno()), str(self.readypipein), self.logfile, self.bitbake_lock.name, self.sockname, str(self.server_timeout or 0), str(int(self.profile)), str(self.xmlrpcinterface[0]), str(self.xmlrpcinterface[1])) def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpcinterface, profile): diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..3d22005 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,108 @@ +[build-system] +requires = ["setuptools_scm[toml] >= 6.2"] +build-backend = "setuptools.build_meta" + +# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html +[project] +name = "bitbake" +description = "" +readme = "README.md" +requires-python = ">= 3.7" +keywords = ["yocto"] +version = "2.4.1" +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "Topic :: Software Development :: Build Tools", + "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", + "Operating System :: Microsoft :: Windows", + "Operating System :: POSIX", + "Operating System :: Unix", + "Operating System :: MacOS", + "Programming Language :: Python :: 3 :: Only", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.7", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: Implementation :: CPython", + "Programming Language :: Python :: Implementation :: PyPy", +] +dynamic = ["optional-dependencies"] + +[[project.authors]] +name = "Tim Ansell" +email = "mithro@mithis.net" + +[[project.authors]] +name = "Phil Blundell" +email = "pb@handhelds.org" + +[[project.authors]] +name = "Seb Frankengul" +email = "seb@frankengul.org" + +[[project.authors]] +name = "Holger Freyther" +email = "holger@moiji-mobile.com" + +[[project.authors]] +name = "Marcin Juszkiewicz" +email = "marcin@juszkiewicz.com.pl" + +[[project.authors]] +name = "Chris Larson" +email = "kergoth@handhelds.org" + +[[project.authors]] +name = "Ulrich Luckas" +email = "luckas@musoft.de" + +[[project.authors]] +name = "Mickey Lauer" +email = "mickey@Vanille.de" + +[[project.authors]] +name = "Richard Purdie" +email = "rpurdie@rpsys.net" + +[[project.authors]] +name = "Holger Schurig" +email = "holgerschurig@gmx.de" + +[project.license] +text = "GPL v2" + +[project.urls] +Homepage = "https://docs.yoctoproject.org/bitbake" +Download = "https://git.openembedded.org/bitbake" +"Bug Report" = "https://bugzilla.yoctoproject.org" +Source = "https://git.openembedded.org/bitbake" + +[tool.setuptools] +script-files = [ + "bin/bitbake", + "bin/bitbake-diffsigs", + "bin/bitbake-dumpsig", + "bin/bitbake-getvar", + "bin/bitbake-hashclient", + "bin/bitbake-hashserv", + "bin/bitbake-layers", + "bin/bitbake-prserv", + "bin/bitbake-selftest", + "bin/bitbake-server", + "bin/bitbake-worker", + "bin/git-make-shallow", + "bin/toaster", + "bin/toaster-eventreplay", +] + +[tool.setuptools.packages.find] +where = ["lib"] + +[tool.setuptools.data-files] +"share/man/man1" = ["doc/bitbake.1"] + +[tool.setuptools.dynamic.optional-dependencies.toaster] +file = "toaster-requirements.txt" -- 2.40.1 ``` We encourage distributions to not package bitbake, but instead expect people using bitbake to bundle it with the layers they're using. This is mainly because the bitbake API changes in each major release so there's a high chance that the distribution-provided bitbake isn't compatible with the layers that you're trying to use, at which point having a distribution-provided bitbake is pointless. There's a related question over having pyproject.toml so that you can build a wheel of bitbake, and whether pypi should have all of the releases, but right now the maintainers stance is that you bundle bitbake with your layers. Also considering bitbake's git repository is hosted at git.openembedded.org I'm not sure how useful a .github/workflows/python.yml would be. |