12148 2017-09-30 22:26:50 +0000 bmap-tools does not find pkg_resources built by Yocto, and uses host pkg_resources 2018-04-17 21:48:49 +0000 1 1 7 Build System, Metadata & Runtime OE-Core deployment unspecified x86 Multiple RESOLVED FIXED Medium+ critical 2.4.3 1 yocto wenzong.fan alejandro.hernandez gizero randy.macleod richard.purdie stephano tim.orling --- --- No (bug/feature does not impact docs) oldest_to_newest 77249 0 yocto 2017-09-30 22:26:50 +0000 When executing "do_image_wic" wic create "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/../meta-emagii/wic/emagii_beaglebone.wks" --vars "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/sysroots/beaglebone/imgdata/" -e "emagii-image" -o "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/deploy-emagii-image-image-complete/emagii-image-beaglebone-20170929070314/ Note: I mention bmaptool below, but there are several apps in "bmap-tools" which could be the problem. Then bmaptool executes the statement: "from pkg_resources import load_entry_point" pkg_resources used to contain pkg_resources.py with "load_entry_point". Now, "load_entry_point" is within pkg_resources/__init__.py, but apparently this is not found, so bmaptool finds "/usr/lib/python3/dist-packages/pkg_resources.py" amongst the host files. A check of bmaptool version is done, and not host bmaptool is checked. On Ubuntu-16.04, the bmaptool version is 3.2, which does not meet the 3.4 requirement, so the build fails without creating a rootfs. Obviously using the host version is bad, but if people are using the 3.4 bmap-toos they may not have detected the problem. Error Log: | DEBUG: Executing shell function do_image_wic | wic create "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/../meta-emagii/wic/emagii_beaglebone.wks" --vars "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/sysroots/beaglebone/imgdata/" -e "emagii-image" -o "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/deploy-emagii-image-image-complete/emagii-image-beaglebone-20170929070314/" | INFO: Creating image(s)... | | INFO: The new image(s) can be found here: | /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/deploy-emagii-image-image-complete/emagii-image-beaglebone-20170929070314/emagii_beaglebone-201709290903-mmcblk.direct | | The following build artifacts were used to create the image(s): | ROOTFS_DIR: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/deploy-emagii-image-image-complete/emagii-image-beaglebone-20170929070314/tmp.wic.slysr_64/rootfs_copy | BOOTIMG_DIR: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/recipe-sysroot/usr/share | KERNEL_DIR: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/deploy/images/beaglebone | NATIVE_SYSROOT: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/recipe-sysroot-native | | INFO: The image(s) were created using OE kickstart file: | /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/../meta-emagii/wic/emagii_beaglebone.wks | Traceback (most recent call last): | File "/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/recipe-sysroot-native/usr/bin/bmaptool", line 6, in <module> | from pkg_resources import load_entry_point | File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2749, in <module> | working_set = WorkingSet._build_master() | File "/usr/lib/python3/dist-packages/pkg_resources.py", line 444, in _build_master | ws.require(__requires__) | File "/usr/lib/python3/dist-packages/pkg_resources.py", line 725, in require | needed = self.resolve(parse_requirements(requirements)) | File "/usr/lib/python3/dist-packages/pkg_resources.py", line 628, in resolve | raise DistributionNotFound(req) | pkg_resources.DistributionNotFound: bmap-tools==3.4 | WARNING: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/temp/run.do_image_wic.19342:1 exit 1 from 'bmaptool create emagii-image-beaglebone-20170929070314.rootfs.wic -o emagii-image-beaglebone-20170929070314.rootfs.wic.bmap' | ERROR: Function failed: do_image_wic (log file is located at /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/tmp/work/beaglebone-poky-linux-gnueabi/emagii-image/1.0-r0/temp/log.do_image_wic.19342) ERROR: Task (/home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/../meta-emagii/recipes-images/images/emagii-image.bb:do_image_wic) failed with exit code '1' NOTE: Tasks Summary: Attempted 3187 tasks of which 3186 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/ulf/Kunder/eMagii/Disk/yocto-arm/poky-arm/build/../meta-emagii/recipes-images/images/emagii-image.bb:do_image_wic Summary: There was 1 WARNING message shown. Summary: There was 1 ERROR message shown, returning a non-zero exit code. 78828 1 tim.orling 2017-12-23 01:58:46 +0000 The version of python3-setuptools now available on oe-core master packages pkg_resources. So, bmap-tools now RDEPENDS on python3-setuptools. I'll be sending a patch. 78829 2 tim.orling 2017-12-23 02:47:47 +0000 https://patchwork.openembedded.org/patch/146884/ 78903 3 richard.purdie 2018-01-04 16:24:58 +0000 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cfa9c413fb31b64c8592700d01ca4b038ffbf14c http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0cf9964109ae862f93bb1bad64aee70cd054c3c1 78910 4 akuster 2018-01-04 16:57:17 +0000 (In reply to comment #3) > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > ?id=cfa9c413fb31b64c8592700d01ca4b038ffbf14c > http://git.yoctoproject.org/cgit.cgi/poky/commit/ > ?id=0cf9964109ae862f93bb1bad64aee70cd054c3c1 Richard, This has a dependency on: http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/python?id=5b784c37f1be769a2abcc5bef5b03eb3b349138b and http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/python?id=624a6f209248a0c8e0759d43f246c903a8db6a71 this one needs the new class pypi.bbclass http://cgit.openembedded.org/openembedded-core/commit/meta/classes?id=c350812523017f113f63e0b863fd526b4d6331b9 so what part of setuptools do we need? 78911 5 akuster 2018-01-04 17:02:43 +0000 updating to 36.5 seems safe. http://cgit.openembedded.org/openembedded-core/commit/meta/recipes-devtools/python?id=5b784c37f1be769a2abcc5bef5b03eb3b349138b v36.5.0 ------- * #170: When working with Mercurial checkouts, use Windows-friendly syntax for suppressing output. * Inspired by #1134, performed substantial refactoring of ``pkg_resources.find_on_path`` to facilitate an optimization for paths with many non-version entries. v36.4.0 ------- * #1075: Add new ``Description-Content-Type`` metadata field. `See here for documentation on how to use this field. <https://packaging.python.org/specifications/#description-content-type>`_ * #1068: Sort files and directories when building eggs for deterministic order. * #196: Remove caching of easy_install command in fetch_build_egg. Fixes issue where ``pytest-runner-N.N`` would satisfy the installation of ``pytest``. * #1129: Fix working set dependencies handling when replacing conflicting distributions (e.g. when using ``setup_requires`` with a conflicting transitive dependency, fix #1124). * #1133: Improved handling of README files extensions and added Markdown to the list of searched READMES. * #1135: Improve performance of pkg_resources import by not invoking ``access`` or ``stat`` and using ``os.listdir`` instead. v36.3.0 ------- * #1131: Make possible using several files within ``file:`` directive in metadata.long_description in ``setup.cfg``. 78912 6 akuster 2018-01-04 17:05:38 +0000 updating to 38.2.4 adds new functions. We change update the package and exclude the pypi part. v38.2.4 ------- * #1220: Fix `data_files` handling when installing from wheel. v38.2.3 ------- * fix Travis' Python 3.3 job. v38.2.2 ------- * #1214: fix handling of namespace packages when installing from a wheel. v38.2.1 ------- * #1212: fix encoding handling of metadata when installing from a wheel. v38.2.0 ------- * #1200: easy_install now support installing from wheels: they will be installed as standalone unzipped eggs. v38.1.0 ------- * #1208: Improve error message when failing to locate scripts in egg-info metadata. v38.0.0 ------- * #458: In order to support deterministic builds, Setuptools no longer allows packages to declare ``install_requires`` as unordered sequences (sets or dicts). v37.0.0 ------- * #878: Drop support for Python 2.6. Python 2.6 users should rely on 'setuptools < 37dev'. v36.8.0 ------- * #1190: In SSL support for package index operations, use SNI where available. v36.7.3 ------- * #1175: Bug fixes to ``build_meta`` module. v36.7.2 ------- * #701: Fixed duplicate test discovery on Python 3. v36.7.1 ------- * #1193: Avoid test failures in bdist_egg when PYTHONDONTWRITEBYTECODE is set. v36.7.0 ------- * #1054: Support ``setup_requires`` in ``setup.cfg`` files. v36.6.1 ------- * #1132: Removed redundant and costly serialization/parsing step in ``EntryPoint.__init__``. * #844: ``bdist_egg --exclude-source-files`` now tested and works on Python 3. v36.6.0 ------- * #1143: Added ``setuptools.build_meta`` module, an implementation of PEP-517 for Setuptools-defined packages. * #1143: Added ``dist_info`` command for producing dist_info metadata. 78913 7 tim.orling 2018-01-04 17:11:52 +0000 pkg_resources was available in 36.2.7, apparently it just wasn't being packaged with the state of the recipe. https://github.com/pypa/setuptools/tree/d548901f9911e784892e99e933d6c60016521e34/pkg_resources 78914 8 akuster 2018-01-04 17:33:37 +0000 (In reply to comment #7) > pkg_resources was available in 36.2.7, apparently it just wasn't being > packaged with the state of the recipe. > > https://github.com/pypa/setuptools/tree/ > d548901f9911e784892e99e933d6c60016521e34/pkg_resources ok, so all I need to do is fix the recipe? 79036 9 akuster 2018-01-11 16:02:19 +0000 there is a patch for rocko I need to backport. 80285 10 wenzong.fan 2018-04-17 19:31:16 +0000 (In reply to comment #9) > there is a patch for rocko I need to backport. I checked the 2.4.2 branch and I found this commit for bmap-tools: --------------------------------------- $ git log bmap-tools_3.4.bb commit 8fedae4a8444394a9c00d40e41aa302d2507030b Author: Tim Orling <timothy.t.orling@linux.intel.com> Date: Fri Dec 22 18:46:17 2017 -0800 bmap-tools: fix RDEPENDS and update SRC_URI * RDEPENDS are now all the python3 versions * Upstream SRC_URI has moved to github.com/intel/bmap-tools * Missing RDEPENDS on python3-setuptools added - Provides import for pkg_resources Fixes [YOCTO #12148] (From OE-Core rev: 6cc7da6477c6995e936f241b0f6f6ec147814538) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3d63452d8d8d8dc61cd26b7759b5c0d235d70451) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> ---------------------- And this one for python3-setuptools: --------------------- $ git log python3-setuptools_36.2.7.bb commit 64390f742bea1137ef56a55dbea81e99adf35b64 Author: Tim Orling <timothy.t.orling@linux.intel.com> Date: Thu Jan 4 14:56:44 2018 -0800 python[3]-setuptools: inherit setuptools * distutils[3].bbclass was not including pkg_resources module * Drop uneeded DISTUTILS_INSTALL_ARGS as setuptools[3].bbclass provides it Fixes [YOCTO #12148] (From OE-Core rev: 4e09525c73348c2b47b07654ac31ffc6cbd3d5fe) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> ------------------------ I think it's done for the backport. 80286 11 wenzong.fan 2018-04-17 21:48:49 +0000 This defect has been fixed both on master and rocko.