11724 2017-06-27 12:59:27 +0000 bitbake core-image-minimal -c populate_sdk: -dev packages missing in SDK 2020-05-28 07:28:07 +0000 1 1 7 Build System, Metadata & Runtime OE-Core deployment 2.3.2 x86 Multiple RESOLVED FIXED Medium+ normal 4.99 1 pokylinux richard.purdie anders.darander bluelightning net147 niko.mauno o.mandel Qi.Chen randy.macleod richard.purdie star --- --- Yes (doc changes required) oldest_to_newest 74571 0 pokylinux 2017-06-27 12:59:27 +0000 I thought by bitbake core-image-minimal -c populate_sdk you automatically get all .h and .so files into the SDK. So you have an SDK which can build against a predefined software stack. But [1] is not very clear about it when it says:"When you build a standard SDK using the bitbake -c populate_sdk, a default set of packages is included in the resulting SDK." It looks like those lib packages made it into the rootfs: libblkid1 armv7a-neon 2.29.1-r0.0 libc6 armv7a-neon 2.25-r0.0 libkmod2 armv7a-neon 23+git0+65a885df5f-r0.0 libuuid1 armv7a-neon 2.29.1-r0.0 libz1 armv7a-neon 1.2.11-r0.0 but in the SDK I see those -dev packages: libatomic-dev armv7a-neon 6.3.0-r0.0 libc6-dev armv7a-neon 2.25-r0.0 libgcc-s-dev armv7a-neon 6.3.0-r0.0 libssp-dev armv7a-neon 6.3.0-r0.0 libstdc++-dev armv7a-neon 6.3.0-r0.0 linux-libc-headers-dev armv7a-neon 4.9-r0.0 meaning I don't see e.g. a -dev package for libz automatically installed into the SDK. I can add it with TOOLCHAIN_TARGET_TASK_append = " zlib-dev" but is this the desired behavior? How would I get all the -dev packages into the SDK but not onto the rootfs? [1] http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#sdk-adding-individual-packages 74644 1 randy.macleod 2017-06-29 15:13:37 +0000 The intended work-flow here it to set 'dev-pkgs' in the IMAGE_FEATURES then they will be included in the SDK. We'll need to confirm this so I'll do it unless someone gets to it before me. Thereafter, this defect will require a documentation update. There's some concern about automatically adding all dev packages since for some users, just building say core-image-minimal, only a few dev-pkgs would be added and that is all that is needed and wanted. There was some discussion about teaching bitbake that if an image ends in -dev then the IMAGE_FEATURES variable would be automatically updated but people were not happy with such magic. In addition, we discussed manually defining a FOO-dev for all FOO images, but users already have too many image choices so a documentation solution is preferred. 74786 2 pokylinux 2017-07-05 16:13:33 +0000 Just to clarify that I understood what you are saying. You are saying that the proposed workflow is to to include dev-pkgs in the image features. My understanding is, that like this all dev packages will be installed in the image as well (like with -sdk images when you want to compile on the target). The docu says:"dev-pkgs: Installs development packages (headers and extra library links) for all packages installed in a given image." Is this the case? I am not sure that is what I would want;) And I am pretty sure most developers will agree with me;) I would like some image without headers and if I create an SDK e.g. via do_populate_sdk I would like to find everything needed to build against what's in the image (including headers and libs) in the SDK. "The do_populate_sdk task helps create the standard SDK and handles two parts: a target part and a host part. The target part is the part built for the target hardware and includes libraries and headers. The host part is the part of the SDK that runs on the SDKMACHINE." 76025 3 star 2017-08-21 09:56:32 +0000 Is there any news to this? I would agree to Robert. The expectation of a "-c populate_sdk" is the inclusion of the headers to uses it for development. Why else I do need an SDK? Otherwise (which would be pity) I would like to know, what to do to just add the dev packages to the SDK and not to the image ? Do I have do add all dev packages I need manually or is there a general include? 76035 4 bluelightning 2017-08-21 20:43:10 +0000 Contrary to what Randy said above, if you build an SDK you should get all -dev packages in the SDK corresponding to the normal packages in the image, that's been the design for a very long time. I can't immediately explain what you are seeing, but it does need to be investigated. 76041 5 randy.macleod 2017-08-22 01:11:49 +0000 Thanks for clearing that up Paul. I don't often use the SDK and I was trying to document what I had heard from someone else on the bugboard call. Sorry for causing any confusion. 76413 6 randy.macleod 2017-08-31 15:42:46 +0000 Chen Qi is going to check on this for Mark. 76434 7 Qi.Chen 2017-09-01 06:12:52 +0000 I've tested `bitbake core-image-minimal -c populate_sdk', and both *-dev packages and *-dbg packages are installed correctly, including libz-dev and libz-dbg. Some related codes are as below. meta/classes/populate_sdk_base.bbclass:SDKIMAGE_FEATURES ??= "dev-pkgs dbg-pkgs ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'doc-pkgs', '', d)}" meta/classes/populate_sdk_base.bbclass:SDKIMAGE_INSTALL_COMPLEMENTARY = '${@complementary_globs("SDKIMAGE_FEATURES", d)}' meta/lib/oe/sdk.py: self.target_pm.install_complementary(self.d.getVar('SDKIMAGE_INSTALL_COMPLEMENTARY')) So as long as the user doesn't override SDKIMAGE_FEATURES, the dev and dbg packages would be installed automatically. Robert, could you please help check if the setting for your SDKIMAGE_FEATURES is correct. Also, you could check if SDKIMAGE_INSTALL_COMPLEMENTARY is set correctly by `bitbake <image_recipe> -c populate_sdk -e > env' and checking the file. If they are both set correctly, and you still have this problem, you could follow the install_complementary function and debug a little. Hope to hear some feedback from you. Regards, Chen Qi 76781 8 o.mandel 2017-09-13 11:55:36 +0000 Here is a minimal test-layer consisting of only 7 files that shows the problem. After running "bitbake -c populate_sdk test-image" one expectes the file usr/lib/test-lib to be present in the SDK (which it isn't): # meta-test/conf/bblayers.conf.sample LCONF_VERSION = "7" BBLAYERS ?= " \ ##OEROOT##/meta \ ##OEROOT##/meta-test \ " # meta-test/conf/layer.conf BBPATH .= ":${LAYERDIR}" BBFILES += "${LAYERDIR}/recipes-core/*/*.bb" BBFILE_COLLECTIONS += "test" BBFILE_PATTERN_test = "^${LAYERDIR}/" BBFILE_PRIORITY_test = "7" LAYERDEPENDS_test = "core" # meta-test/conf/local.conf.sample CONF_VERSION = "1" MACHINE = "qemux86-64" # meta-test/recipes-core/images/test-image.bb inherit core-image IMAGE_INSTALL = "test-app" # recipes-core/test-app/files/configure #!/bin/sh if [ -f "${PKG_CONFIG_SYSROOT_DIR:-}${libdir:-/usr/lib}/test-lib" ]; then echo "test-lib found." else echo >&2 "test-lib not found!" exit 1 fi # meta-test/recipes-core/test-app/test-app.bb LICENSE = "PD" LIC_FILES_CHKSUM = "file://${COREBASE}/meta/files/common-licenses/PD;md5=b3597d12946881e13cb3b548d1173851" DEPENDS = "test-lib" SRC_URI = "file://configure;subdir=${BPN}-${PV}" do_configure () { sh configure } # meta-test/recipes-core/test-lib/test-lib.bb LICENSE = "PD" do_install () { install -d "${D}${libdir}" touch "${D}${libdir}/test-lib" } FILES_${PN} = "${libdir}/test-lib" 76782 9 o.mandel 2017-09-13 12:03:30 +0000 (In reply to comment #8) > # recipes-core/test-app/files/configure Oops: forgot the directory for the layer in there. The file is under: # meta-test/recipes-core/test-app/files/configure 77521 10 o.mandel 2017-10-11 16:07:14 +0000 Found a problem with my test-layer: test-app.bb installs no files and thus does not produce a package. This is the reason why the SDK does not contain test-lib. So adding a minimal do_install() to test-app.bb makes the SDK for the test-layer work. But this shows the path to the original problem of this ticket: the SDK population works package-based, not recipe-based. So if an image contains an empty application-package, then the SDK will not contain any library packages mentioned in the application recipe. Likewise, if any of the libraries mentioned in a recipes DEPENDS variable generates an empty package (think: header-only libraries), then the matching lib-dev package will be missing from the SDK. This can be shown in the test-layer by modifying it a bit (see below). Without the ALLOW_EMPTY line, the SDK will be missing test-lib.h, with the line the file will be present. Finally, the original problem of libz-dev missing is also related: the applications in the image depend on the libz1 package, but there is no libz1-dev package; it is called libz-dev, instead. Similar problems seem to exist for all library packages that get their major version number added to their package name. As of this moment, I have no idea how to solve these problems in general. One can go ahead and add PACKAGES += "${PN}" and ALLOW_EMPTY_${PN} = "1" to all header-only library recipes (e.g. cppzmq_git). But that seems hacky, and it does not help with the package-name discrepancy. Any idea where the renaming happens? # meta-test/recipes-core/test-app/files/configure #!/bin/sh if [ -f "${PKG_CONFIG_SYSROOT_DIR:-}${includedir:-/usr/include}/test-lib.h" ]; then echo "test-lib found." else echo >&2 "test-lib not found!" exit 1 fi # meta-test/recipes-core/test-lib/test-lib.bb LICENSE = "PD" do_install () { install -d "${D}${includedir}" touch "${D}${includedir}/test-lib.h" } #ALLOW_EMPTY_${PN} = "1" 77948 11 pokylinux 2017-11-03 16:45:37 +0000 I built with my pre 2.4 release and it shows a similar behaviour as I reported before. Maybe I need to clarify things a bit more and give some more examples: libz is part of core-image-sato-sdk since it's a xxx-sdk I can see on the rootfs: ./core-image-sato-sdk-multi-v7-ml/usr/include/zlib.h and also I can see on the sysroot: ./sysroots/armv7a-neon-poky-linux-gnueabi/usr/include/zlib.h I believe if I build a core-image-minimal (without the sdk) which contains the zlib and then core-image-minimal -c populate_sdk ./sysroots/armv7a-neon-poky-linux-gnueabi/usr/include/zlib.h is missing. (Need to re-test this to be 100% sure). 79628 12 niko.mauno 2018-02-28 16:28:42 +0000 Hi, just in case somebody might find this helpful - I believe I encountered somewhat similar issue, and submitted a patch to make the underlying failure more explicit to bitbake user: http://lists.openembedded.org/pipermail/openembedded-core/2018-February/147966.html 80089 13 pokylinux 2018-04-04 06:22:11 +0000 Just for completeness the oe-patchwork link of Niko Mauno's patch: https://patches.openembedded.org/patch/149321/ 80846 14 pokylinux 2018-06-20 11:54:58 +0000 looks like the patch, or something similar made it into Sumo 87347 15 richard.purdie 2020-05-28 07:28:07 +0000 As above, fixes were merged for this.