13217 2019-03-07 19:09:28 +0000 "bitbake -c create_manifest python" automatically overwrites upstream python-manifest.json, breaking python build 2025-03-27 15:00:07 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 2.0.3 x86 Multiple RESOLVED NOTABUG Low minor Future 1 khays unassigned alejandro meta.mr.watcher meta.watcher randy.macleod richard.purdie --- --- No (bug/feature does not impact docs) oldest_to_newest 83214 0 4446 khays 2019-03-07 19:09:28 +0000 Created attachment 4446 Generated manifest for target. In our embedded devices, we require a very small subset of python ("core", "json", "lang") - that's what we use, and, as we are storage constrained, it is manditory for us to trim back the target python build. Using the "bitbake -c create_manifest python" does exactly what we require, with a replacement "python2-manifest.json" from our layer. After the trimmed manifest is generated, we can check it into our vendor meta-layer, and life is happy. Getting there, however, involved stepping on a rake that *might* violate the intent of poky recipes, in general: To wit, the sumo "python2_2.7.15.bb" recipe, in task "do_create_manifest()", at line 255, copies the newly generated manifest back to "${THISDIR}", overwriting the original manifest from upstream, automatically: "cp python2-manifest.json.new ${THISDIR}/python/python2-manifest.json" While the original intent of "only do this when upgrading python version" is laudable, it also causes the python build to break, as it replaces the default manifest with the trimmed manifest, in the current checkout, which is not appropriate for then building native-python. I have marked this as "minor" severity, as, once the replacement manifest is in our meta-layer, this won't happen, unless we try "create_manifest", again. It did take me a day to figure out that creating our target manifest was breaking the native-python build, though. Perhaps someone else will benefit from this being fixed. To recreate: 1. Checkout a clean copy of poky, sumo branch. 2. Create a vendor meta-layer with a replacement "python2-manifest.json" (I've attached ours as an example). 2. Utter "bitbake -c create_manifest python" 3. In the poky checkout, utter "git status". 4. Note the output: On branch sumo Your branch is up-to-date with 'origin/sumo'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: meta/recipes-devtools/python/python/python2-manifest.json no changes added to commit (use "git add" and/or "git commit -a") 5. Utter "bitbake -c cleanall python; bitbake -c create_manifest python; bitbake -c cleanall python; bitbake python", and enjoy the resulting hilarity: Loading cache: 100% |##########################################################################################################################################################################################################| Time: 0:00:00 Loaded 1430 entries from dependency cache. Parsing recipes: 100% |########################################################################################################################################################################################################| Time: 0:00:21 Parsing of 931 .bb files complete (929 cached, 2 parsed). 1432 targets, 277 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "1.38.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "universal" TARGET_SYS = "arm-poky-linux-gnueabi" MACHINE = "biamp-zynq" DISTRO = "poky" DISTRO_VERSION = "2.0" TUNE_FEATURES = "arm armv7a vfp thumb neon callconvention-hard cortexa9" TARGET_FPU = "hard" meta meta-poky meta-yocto-bsp = "sumo:d3ad2438222050faf33e83598c1f6ecd25ff65b6" meta-gplv2 = "sumo:d7687d404bbc9ba3f44ec43ea8828d9071033513" meta-qt4 = "sumo:8e791c40140460825956430ba86b6266fdec0a93" meta-xilinx-bsp = "sumo:5fccc46503e468ed024185ed032891799a31db58" meta-biamp = "sumo:a30e3642680ddf96806658b8bb3f4e963285acf2" Initialising tasks: 100% |#####################################################################################################################################################################################################| Time: 0:00:00 NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks ERROR: python-2.7.15-r1 do_package_qa: QA Issue: non -staticdev package contains static .a library: python-misc path '/work/cortexa9hf-neon-poky-linux-gnueabi/python/2.7.15-r1/packages-split/python-misc/usr/lib/python2.7/config/libpython2.7.a' [staticdev] ERROR: python-2.7.15-r1 do_package_qa: QA Issue: /usr/bin/pydoc contained in package python requires /usr/bin/python2.7, but no providers found in RDEPENDS_python? [file-rdeps] ERROR: python-2.7.15-r1 do_package_qa: QA run found fatal errors. Please consider fixing them. ERROR: python-2.7.15-r1 do_package_qa: Function failed: do_package_qa ERROR: Logfile of failure stored in: /home/khays/source/biamp-linux-build/build-biamp-linux-zynq/tmp/work/cortexa9hf-neon-poky-linux-gnueabi/python/2.7.15-r1/temp/log.do_package_qa.12172 ERROR: Task (/home/khays/source/biamp-linux-build/build-biamp-linux-zynq/../poky/meta/recipes-devtools/python/python_2.7.15.bb:do_package_qa) failed with exit code '1' NOTE: Tasks Summary: Attempted 610 tasks of which 590 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/khays/source/biamp-linux-build/build-biamp-linux-zynq/../poky/meta/recipes-devtools/python/python_2.7.15.bb:do_package_qa Summary: There were 4 ERROR messages shown, returning a non-zero exit code. 83265 1 richard.purdie 2019-03-14 14:37:34 +0000 Looks like this is working as designed. I do understand the problem, I'm wondering if the code could be smarter about figuring out where the original manifest came from. With that in mind we'll keep this open but its low priority 83269 2 khays 2019-03-14 16:08:09 +0000 Agreed, it may work as designed, but I would argue that it violates the "principal of least surprise" (https://en.wikipedia.org/wiki/Principle_of_least_astonishment ), in that "bitbake -c create_manifest" does not update $WORKDIR, or something under $WORKDIR, but instead updates $THISDIR, which, in the typical usage case, is a pristine layer. I've run across no other bitbake action that does this...which may reflect on my level of experience with bitbake, as I've only been using it for five years. Perhaps it is a documentation issue, instead. 101519 3 randy.macleod 2025-03-27 15:00:07 +0000 Old bug cleanup. This is doing what you should expect, if you'd like to a documentation change, it will be reviewed and likely merged. Thanks. 4446 2019-03-07 19:09:28 +0000 2019-03-07 19:09:28 +0000 Generated manifest for target. python2-manifest.json text/plain 3711 khays ewogICAgImNvcmUiOiB7CiAgICAgICAgImZpbGVzIjogWwogICAgICAgICAgICAiJHtiaW5kaXJ9 L3B5dGhvbioiLAogICAgICAgICAgICAiJHtpbmNsdWRlZGlyfS9weXRob24yLjcvcHljb25maWcq LmgiLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9Vc2VyRGljdC5weSIsCiAgICAg ICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L19fZnV0dXJlX18ucHkiLAogICAgICAgICAgICAi JHtsaWJkaXJ9L3B5dGhvbjIuNy9fYWJjb2xsLnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9w eXRob24yLjcvX3N5c2NvbmZpZ2RhdGEucHkiLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhv bjIuNy9fd2Vha3JlZnNldC5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2Fi Yy5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2F0ZXhpdC5weSIsCiAgICAg ICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2NvZGVjcy5weSIsCiAgICAgICAgICAgICIke2xp YmRpcn0vcHl0aG9uMi43L2NvbGxlY3Rpb25zLnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9w eXRob24yLjcvY29weS5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2NvcHlf cmVnLnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvZW5jb2RpbmdzIiwKICAg ICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvZW5jb2RpbmdzL2FsaWFzZXMucHkiLAogICAg ICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9lbmNvZGluZ3MvdXRmXzgucHkiLAogICAgICAg ICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9nZW5lcmljcGF0aC5weSIsCiAgICAgICAgICAgICIk e2xpYmRpcn0vcHl0aG9uMi43L2dldG9wdC5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0 aG9uMi43L2hlYXBxLnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvaW1wb3J0 bGliIiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcva2V5d29yZC5weSIsCiAgICAg ICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2xpYi1keW5sb2FkL19jb2xsZWN0aW9ucy5zbyIs CiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2xpYi1keW5sb2FkL19oZWFwcS5zbyIs CiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2xpYi1keW5sb2FkL19sb2NhbGUuc28i LAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9saWItZHlubG9hZC9fc3RydWN0LnNv IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvbGliLWR5bmxvYWQvYmluYXNjaWku c28iLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9saWItZHlubG9hZC9pdGVydG9v bHMuc28iLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9saWItZHlubG9hZC9vcGVy YXRvci5zbyIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2xpYi1keW5sb2FkL3Jl YWRsaW5lLnNvIiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvbGliLWR5bmxvYWQv c3Ryb3Auc28iLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9saWItZHlubG9hZC90 aW1lLnNvIiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvbGluZWNhY2hlLnB5IiwK ICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvb3MucHkiLAogICAgICAgICAgICAiJHts aWJkaXJ9L3B5dGhvbjIuNy9wbGF0Zm9ybS5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0 aG9uMi43L3Bvc2l4cGF0aC5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L3Jl LnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvcmxjb21wbGV0ZXIucHkiLAog ICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9zaXRlLnB5IiwKICAgICAgICAgICAgIiR7 bGliZGlyfS9weXRob24yLjcvc2l0ZWN1c3RvbWl6ZS5weSIsCiAgICAgICAgICAgICIke2xpYmRp cn0vcHl0aG9uMi43L3NyZV9jb21waWxlLnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRo b24yLjcvc3JlX2NvbnN0YW50cy5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43 L3NyZV9wYXJzZS5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L3N0YXQucHki LAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9zdHJpbmcucHkiLAogICAgICAgICAg ICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9zdHJ1Y3QucHkiLAogICAgICAgICAgICAiJHtsaWJkaXJ9 L3B5dGhvbjIuNy9zeXNjb25maWcucHkiLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIu Ny90cmFjZWJhY2sucHkiLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy90eXBlcy5w eSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L3dhcm5pbmdzLnB5IiwKICAgICAg ICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvd2Vha3JlZi5weSIKICAgICAgICBdLAogICAgICAg ICJyZGVwZW5kcyI6IFtdLAogICAgICAgICJzdW1tYXJ5IjogIlB5dGhvbiBpbnRlcnByZXRlciBh bmQgY29yZSBtb2R1bGVzIgogICAgfSwKICAgICJqc29uIjogewogICAgICAgICJmaWxlcyI6IFsK ICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvanNvbiIsCiAgICAgICAgICAgICIke2xp YmRpcn0vcHl0aG9uMi43L2xpYi1keW5sb2FkL19qc29uLnNvIgogICAgICAgIF0sCiAgICAgICAg InJkZXBlbmRzIjogWwogICAgICAgICAgICAiY29yZSIKICAgICAgICBdLAogICAgICAgICJzdW1t YXJ5IjogIlB5dGhvbiBKU09OIHN1cHBvcnQiCiAgICB9LAogICAgImxhbmciOiB7CiAgICAgICAg ImZpbGVzIjogWwogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9iaXNlY3QucHkiLAog ICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIuNy9kaXMucHkiLAogICAgICAgICAgICAiJHts aWJkaXJ9L3B5dGhvbjIuNy9mdW5jdG9vbHMucHkiLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5 dGhvbjIuNy9pbnNwZWN0LnB5IiwKICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvbGli LWR5bmxvYWQvX2Jpc2VjdC5zbyIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L2xp Yi1keW5sb2FkL19mdW5jdG9vbHMuc28iLAogICAgICAgICAgICAiJHtsaWJkaXJ9L3B5dGhvbjIu Ny9saWItZHlubG9hZC9hcnJheS5zbyIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43 L2xpYi1keW5sb2FkL3BhcnNlci5zbyIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43 L29wY29kZS5weSIsCiAgICAgICAgICAgICIke2xpYmRpcn0vcHl0aG9uMi43L3Rva2VuLnB5IiwK ICAgICAgICAgICAgIiR7bGliZGlyfS9weXRob24yLjcvdG9rZW5pemUucHkiCiAgICAgICAgXSwK ICAgICAgICAicmRlcGVuZHMiOiBbCiAgICAgICAgICAgICJjb3JlIgogICAgICAgIF0sCiAgICAg ICAgInN1bW1hcnkiOiAiUHl0aG9uIGxvdy1sZXZlbCBsYW5ndWFnZSBzdXBwb3J0IgogICAgfSwK ICAgICJtb2R1bGVzIjogewogICAgICAgICJmaWxlcyI6IFtdLAogICAgICAgICJyZGVwZW5kcyI6 IFsKICAgICAgICAgICAgImNvcmUiLAogICAgICAgICAgICAianNvbiIsCiAgICAgICAgICAgICJs YW5nIgogICAgICAgIF0sCiAgICAgICAgInN1bW1hcnkiOiAiQWxsIFB5dGhvbiBtb2R1bGVzIgog ICAgfQp9