Bug 10054 - [OE-core] multilib support: populate_sdk help
Summary: [OE-core] multilib support: populate_sdk help
Status: RESOLVED DUPLICATE of bug 8642
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.2
Assignee: Juro Bystricky
QA Contact:
URL:
Whiteboard:
: 8616 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-07-29 20:11 UTC by ncrypticus
Modified: 2017-02-15 08:18 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ncrypticus 2016-07-29 20:11:37 UTC
I can succesfully run the command to generate the SDK:
$ bitbake <recipe name> -c populate_sdk

But the output seems to be incorrect. While the aarch sysroots look good, the
sysroots for arm are completely missing. The environment setup script ends
up pointing to non-existent compilers. Please let me know if you need more
information!

Steps to reproduce:
$ bitbake <recipe name> -c populate_sdk
$ cp tmp/deploy/sdk/<sdk-script>.sh /test/directory
$ cd /test/directory
$ bash <sdk-script>.sh
$ ls sysroots/
aarch64-poky-linux/   x86_64-pokysdk-linux/

Note the missing ARM directories.

For example, the setup script for arm sets this as the command for compiler:
 12 export CC="arm-pokymllib32-linux-gnueabi-gcc

But that compiler doesn't exist anywhere in the folder where I installed
everything.

So it appears that it's not completely or correctly packaging everything
for ARMv7.

The documentation does mention some environment variables
<http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#sdk-dev-environment>,
but it's not clear how those related to multilib.

For instance, none of our recipes have "SDKIMAGE_FEATURES" defined, but
clearly the aarch64 sysroots are getting populated correctly (we
succesfully compiled and ran a simple hello world).

*Can anyone point me to an example, or discuss a successful use of
"multilib" on the Jethro branch (given desparate brainstorming below),
share some more verbose documentation, or ask follow-up questions?*


_____________________________________________________________
Desperate brainstorming below:
I did find this years-old patch:
https://patchwork.openembedded.org/patch/30941/
Notably, the do_populate_sdk method in
poky/meta/classes/populate_sdk_base.bbclass iterates over the values listed
in the MULTILIB_VARIANTS variable:

+	variants = d.getVar("MULTILIB_VARIANTS", True) or ""+	for item in
variants.split():+		# Load overrides from 'd' to avoid having to reset
the value...+		overrides = d.getVar("OVERRIDES", False) +
":virtclass-multilib-" + item+		localdata.setVar("OVERRIDES",
overrides)+		bb.data.update_data(localdata)+		bb.build.exec_func("create_sdk_files",
localdata)

The latest version in Jethro does not have anything equivalent to that, but
I did not dig deeply into the code or the commit history.

______________________________________________________________
More data:

local.conf relevant section:
#
# Multilib configuration
#
# This sets any packages preprended with lib32- to be built with
# the armv7a tuning (32 bit) instead of 64 bit aarch.
#
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "armv7at-neon"
Comment 1 ncrypticus 2016-08-01 23:10:12 UTC
Bitbake version is 1.28.0
Comment 2 Juro Bystricky 2016-08-04 20:57:27 UTC
This is what I did:
(Downloaded "jethro")
In the local.conf added:

MACHINE = "qemuarm64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "armv7a"

$ bitbake core-image-minmal -c populate-sdk
( I got two warnings:
WARNING: gcc multilib setup is not supported for TARGET_ARCH=aarch64
WARNING: gcc multilib setup is not supported for TARGET_ARCH=aarch64
these may or may not be benign, I will look into that)

This created the sdk in the "deploy" folder:

"poky-glibc-x86_64-core-image-minimal-aarch64-toolchain-2.0.2.sh"

At first glance it look like this SDK does contain all toolchains.
However, the environment setup file and the installed files do not see eye to eye (one of them has uses wrong naming convention).

Please try to edit (global search/replace in the file) the file "environment-setup-armv7a-vfp-pokymllib32-linux-gnueabi", removing all references to "-gnueabi", i.e.:

export CPP="arm-pokymllib32-linux-gnueabi-gcc -E  -march=armv7-a -mfpu=vfp  -mfloat-abi=softfp --sysroot=$SDKTARGETSYSROOT" 

becomes 

export CPP="arm-pokymllib32-linux-gcc -E  -march=armv7-a -mfpu=vfp  -mfloat-abi=softfp --sysroot=$SDKTARGETSYSROOT"

etc.

Source this file and run:
$ $CC -dumpspecs


Please let me know if it worked.
Just FYI, the same problem seem to exist in master.
Comment 3 ncrypticus 2016-08-08 23:01:10 UTC
Thanks for the quick response.

I did as you described, and replaced all references to "-gnueabi" from the environment setup script for armv7a.

The output of $CC -dumpspecs is here:
http://pastebin.com/C2Gn4fwt

I was also expecting the sysroot to be named something like "armv7a", but instead we only have aarch64 sysroot:

$ ls sysroots/
aarch64-poky-linux  x86_64-pokysdk-linux

 Is that correct? And some of our lib32-bit recipes are ending up in aarch64 sysroot:

./sysroots/aarch64-poky-linux/usr/src/debug/lib32-<recipe-name>/0.1-r0/git/<recipe-name>

I also don't see the kernel headers. I'm expecting them in sysroots/aarch64/usr/src/kernel, as described here:

http://stackoverflow.com/questions/31256770/using-populate-sdk-to-include-kernel-headers
(I tried adding TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc" but the do_install step fails. I'm looking into that now).

So to sum up, three questions:
1. Does the dumpspecs output look ok?
2. Shouldn't there exist an arm sysroot? Why are lib32 recipes ending up inside aarch64, is that correct?
3. Where are the kernel headers?
Comment 4 ncrypticus 2016-08-09 16:56:11 UTC
Ok, so I tried the version you did:

$ bitbake core-image-minimal -c populate_sdk

Build Configuration:
BB_VERSION        = "1.28.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Ubuntu-14.04"
TARGET_SYS        = "aarch64-poky-linux"
MACHINE           = "qcom-bsp"
DISTRO            = "poky"
DISTRO_VERSION    = "2.0.1"
TUNE_FEATURES     = "aarch64"

Note that the TUNE_FEATURES are still set to aarch64. I do NOT get the same warnings that you do, about "WARNING: gcc multilib setup is not supported for TARGET_ARCH=aarch64", but googling it looks like this might fix it: https://patchwork.openembedded.org/patch/106801/

Also, note that the sysroots still only has one machine architecture in it, but shouldn't it have two sysroots? One for arm and one for arch? Is my understanding correct there?


I can change the TUNE_FEATURES by editing the machine .conf file we're using, from having this line:
require conf/machine/include/arm/arch-armv8.inc

To this:
require conf/machine/include/arm/arch-armv7a.inc


But then I get compile errors with gcc (looking into this now):
ERROR: Task 796 (/local/mnt/workspace2/self/platform/1aug/poky/meta/recipes-devtools/gcc/gcc-runtime_4.9.bb, do_compile) failed with exit code '1'
ERROR: S is not set to the linux source directory. Check 
ERROR: the recipe and set S to the proper extracted subdirectory
ERROR: Function failed: do_kernel_checkout (log file is located at /local/mnt/workspace2/self/platform/1aug/poky/build/tmp/work/qcom_bsp-poky-linux-gnueabi/linux-msm/1.0-r0/temp/log.do_kernel_checkout.3188)
ERROR: Logfile of failure stored in: /local/mnt/workspace2/self/platform/1aug/poky/build/tmp/work/qcom_bsp-poky-linux-gnueabi/linux-msm/1.0-r0/temp/log.do_kernel_checkout.3188


Is my big-picture understanding of this correct? That there should be two folders in the sysroots directory, one for each architecture in multilib configuration?
Comment 5 Juro Bystricky 2016-08-11 15:21:26 UTC
two toolchains are built:
x86_64-pokysdk-linux
    usr/bin/aarch64-poky-linux
    usr/bin/arm-poky-linux

so there are, for example, two compilers:
arm-poky-linux-gnueabi-gcc
aarch64-poky-linux-gcc
Comment 6 ncrypticus 2016-08-12 15:20:56 UTC
Yes, the two compilers makes sense.

Why aren't there two sysroot folders, for appropriate headers to match those toolchains?
Comment 7 Saul Wold 2016-08-25 14:57:11 UTC
*** Bug 8616 has been marked as a duplicate of this bug. ***
Comment 8 leimaohui 2016-09-27 08:59:18 UTC
I met the same Issue. Is there improvement now?
Comment 9 Richard Purdie 2016-09-28 15:09:43 UTC
This is actually the same issue as bug #8642 which I've just sent a patch for.
Comment 10 Richard Purdie 2016-10-01 21:05:19 UTC
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=b6253350fb86d8e54792905342a5892af570d526

*** This bug has been marked as a duplicate of bug 8642 ***
Comment 11 leimaohui 2016-12-19 07:13:39 UTC
(In reply to comment #9)
> This is actually the same issue as bug #8642 which I've just sent a patch
> for.

With these patches, there is still another issue. When I build with aarch64 toolchain, there is an error message as following:

$ . /opt/poky/v16/environment-setup-aarch64_be-linux
/opt/poky/v16/sysroots/aarch64_be-linux/usr/include/gnu/stubs.h:7:29: fatal error: gnu/stubs-soft.h: No such file or directory

I found that stubs.h and lib-names.h can't support multilib. Is there any patch to resolve this issue?
Comment 12 Richard Schmitt 2016-12-29 16:48:28 UTC
I have run into similar issues.  To resolve the issue about stubs-soft.h you need to add -mfloat-abi=hard to your cc flags.  This should be in the environment setup file.  I had found though that the toochain in the sysroot that populate_sdk exports doesn't support this option.  There is a sysroot in the build directory that has a different toolchain that does support it.  So basically I've been copying the build directory's sysroot to my /opt/poky directory and using the toolchain from that directory.

I am using the krogoth branch.
Comment 13 leimaohui 2017-01-24 03:25:22 UTC
The issue is that if you enable multilib, the stubs.h should be like as:
(In reply to comment #12)
> I have run into similar issues.  To resolve the issue about stubs-soft.h you
> need to add -mfloat-abi=hard to your cc flags.  This should be in the
> environment setup file.  I had found though that the toochain in the sysroot
> that populate_sdk exports doesn't support this option.  There is a sysroot
> in the build directory that has a different toolchain that does support it. 
> So basically I've been copying the build directory's sysroot to my /opt/poky
> directory and using the toolchain from that directory.
> 
> I am using the krogoth branch.

The issue is I enabled multilib to build aarch64, and add -mfloat-abi=hard in 32bit cc flag, but in toolchain, the stubs.h file still only support 64bit.

I think, when the multilib is enabled, the stubs.h should like as (x86-64 for example):

#if !defined __x86_64__
# include <gnu/stubs-32.h>
#endif
#if defined __x86_64__ && defined __LP64__
# include <gnu/stubs-64.h>
#endif
#if defined __x86_64__ && defined __ILP32__
# include <gnu/stubs-x32.h>
#endif

But actually,if you enable the multilib when you build aarch 64, the stubs.h can't support multilib. The same as lib-names.h.

Notes: I am using krogoth branch
Comment 14 leimaohui 2017-02-15 08:18:26 UTC
Who has any suggestions?