Bug 11724

The intended work-flow here it to set 'dev-pkgs' in the IMAGE_FEATURES then they will be included in the SDK. We'll need to confirm this so I'll do it unless someone gets to it before me. Thereafter, this defect will require a documentation update.

There's some concern about automatically adding all dev packages since for some users, just building say core-image-minimal, only a few dev-pkgs would be added and that is all that is needed and wanted.

There was some discussion about teaching bitbake that if an image ends in -dev then the IMAGE_FEATURES variable would be automatically updated but people were not happy with such magic. In addition, we discussed manually defining a FOO-dev for all FOO images, but users already have too many image choices so a documentation solution is preferred.

Just to clarify that I understood what you are saying.

You are saying that the proposed workflow is to to include dev-pkgs in the image features.

My understanding is, that like this all dev packages will be installed in the image as well (like with -sdk images when you want to compile on the target). The docu says:"dev-pkgs: Installs development packages (headers and extra library links) for all packages installed in a given image." 

Is this the case?

I am not sure that is what I would want;) And I am pretty sure most developers will agree with me;)

I would like some image without headers and if I create an SDK e.g. via do_populate_sdk I would like to find everything needed to build against what's in the image (including headers and libs) in the SDK.

"The do_populate_sdk task helps create the standard SDK and handles two parts: a target part and a host part. The target part is the part built for the target hardware and includes libraries and headers. The host part is the part of the SDK that runs on the SDKMACHINE."

Is there any news to this? I would agree to Robert.
The expectation of a "-c populate_sdk" is the inclusion of the headers to uses it for development. Why else I do need an SDK? 

Otherwise (which would be pity) I would like to know, what to do to just add the dev packages to the SDK and not to the image ?

Do I have do add all dev packages I need manually or is there a general include?

Contrary to what Randy said above, if you build an SDK you should get all -dev packages in the SDK corresponding to the normal packages in the image, that's been the design for a very long time. I can't immediately explain what you are seeing, but it does need to be investigated.

Thanks for clearing that up Paul. I don't often use the SDK and I was trying to document what I had heard from someone else on the bugboard call. Sorry for causing any confusion.

Chen Qi is going to check on this for Mark.

I've tested `bitbake core-image-minimal -c populate_sdk', and both *-dev packages and *-dbg packages are installed correctly, including libz-dev and libz-dbg.

Some related codes are as below.

  meta/classes/populate_sdk_base.bbclass:SDKIMAGE_FEATURES ??= "dev-pkgs dbg-pkgs ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'doc-pkgs', '', d)}"

  meta/classes/populate_sdk_base.bbclass:SDKIMAGE_INSTALL_COMPLEMENTARY = '${@complementary_globs("SDKIMAGE_FEATURES", d)}'

  meta/lib/oe/sdk.py:        self.target_pm.install_complementary(self.d.getVar('SDKIMAGE_INSTALL_COMPLEMENTARY'))


So as long as the user doesn't override SDKIMAGE_FEATURES, the dev and dbg packages would be installed automatically.

Robert, could you please help check if the setting for your SDKIMAGE_FEATURES is correct. Also, you could check if SDKIMAGE_INSTALL_COMPLEMENTARY is set correctly by `bitbake <image_recipe> -c populate_sdk -e > env' and checking the file.
If they are both set correctly, and you still have this problem, you could follow the install_complementary function and debug a little.
Hope to hear some feedback from you.

Regards,
Chen Qi

Here is a minimal test-layer consisting of only 7 files that shows the problem. After running "bitbake -c populate_sdk test-image" one expectes the file usr/lib/test-lib to be present in the SDK (which it isn't):


# meta-test/conf/bblayers.conf.sample
LCONF_VERSION = "7"
BBLAYERS ?= " \
  ##OEROOT##/meta \
  ##OEROOT##/meta-test \
  "

# meta-test/conf/layer.conf
BBPATH .= ":${LAYERDIR}"
BBFILES += "${LAYERDIR}/recipes-core/*/*.bb"
BBFILE_COLLECTIONS += "test"
BBFILE_PATTERN_test = "^${LAYERDIR}/"
BBFILE_PRIORITY_test = "7"
LAYERDEPENDS_test = "core"

# meta-test/conf/local.conf.sample
CONF_VERSION = "1"
MACHINE = "qemux86-64"

# meta-test/recipes-core/images/test-image.bb
inherit core-image
IMAGE_INSTALL = "test-app"

# recipes-core/test-app/files/configure
#!/bin/sh
if [ -f "${PKG_CONFIG_SYSROOT_DIR:-}${libdir:-/usr/lib}/test-lib" ]; then
  echo "test-lib found."
else
  echo >&2 "test-lib not found!"
  exit 1
fi

# meta-test/recipes-core/test-app/test-app.bb
LICENSE = "PD"
LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/files/common-licenses/PD;md5=b3597d12946881e13cb3b548d1173851"
DEPENDS = "test-lib"
SRC_URI = "file://configure;subdir=${BPN}-${PV}"
do_configure () {
    sh configure
}

# meta-test/recipes-core/test-lib/test-lib.bb
LICENSE = "PD"
do_install () {
    install -d "${D}${libdir}"
    touch "${D}${libdir}/test-lib"
}
FILES_${PN} = "${libdir}/test-lib"

(In reply to comment #8)
> # recipes-core/test-app/files/configure

Oops: forgot the directory for the layer in there. The file is under:

# meta-test/recipes-core/test-app/files/configure

Found a problem with my test-layer: test-app.bb installs no files and thus does not produce a package. This is the reason why the SDK does not contain test-lib. So adding a minimal do_install() to test-app.bb makes the SDK for the test-layer work.

But this shows the path to the original problem of this ticket: the SDK population works package-based, not recipe-based. So if an image contains an empty application-package, then the SDK will not contain any library packages mentioned in the application recipe. Likewise, if any of the libraries mentioned in a recipes DEPENDS variable generates an empty package (think: header-only libraries), then the matching lib-dev package will be missing from the SDK. This can be shown in the test-layer by modifying it a bit (see below). Without the ALLOW_EMPTY line, the SDK will be missing test-lib.h, with the line the file will be present.

Finally, the original problem of libz-dev missing is also related: the applications in the image depend on the libz1 package, but there is no libz1-dev package; it is called libz-dev, instead. Similar problems seem to exist for all library packages that get their major version number added to their package name.

As of this moment, I have no idea how to solve these problems in general. One can go ahead and add PACKAGES += "${PN}" and ALLOW_EMPTY_${PN} = "1" to all header-only library recipes (e.g. cppzmq_git). But that seems hacky, and it does not help with the package-name discrepancy. Any idea where the renaming happens?



# meta-test/recipes-core/test-app/files/configure
#!/bin/sh
if [ -f "${PKG_CONFIG_SYSROOT_DIR:-}${includedir:-/usr/include}/test-lib.h" ];
then
  echo "test-lib found."
else
  echo >&2 "test-lib not found!"
  exit 1
fi

# meta-test/recipes-core/test-lib/test-lib.bb
LICENSE = "PD"
do_install () {
    install -d "${D}${includedir}"
    touch "${D}${includedir}/test-lib.h"
}
#ALLOW_EMPTY_${PN} = "1"

I built with my pre 2.4 release and it shows a similar behaviour as I reported before. 

Maybe I need to clarify things a bit more and give some more examples:

libz is part of core-image-sato-sdk

since it's a xxx-sdk I can see on the rootfs:

./core-image-sato-sdk-multi-v7-ml/usr/include/zlib.h

and also I can see on the sysroot:

./sysroots/armv7a-neon-poky-linux-gnueabi/usr/include/zlib.h

I believe if I build a core-image-minimal (without the sdk) which contains the zlib and then 

core-image-minimal -c populate_sdk

./sysroots/armv7a-neon-poky-linux-gnueabi/usr/include/zlib.h

is missing. (Need to re-test this to be 100% sure).

Hi, just in case somebody might find this helpful - I believe I encountered somewhat similar issue, and submitted a patch to make the underlying failure more explicit to bitbake user: http://lists.openembedded.org/pipermail/openembedded-core/2018-February/147966.html

Just for completeness the oe-patchwork link of Niko Mauno's patch:

https://patches.openembedded.org/patch/149321/

looks like the patch, or something similar made it into Sumo

As above, fixes were merged for this.