Bug 4908

Summary: External toolchain does not work with poky dylan
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Laszlo Papp <lpapp>
Component: devtools / tool chainAssignee: Paul Eggleton <bluelightning>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: kergoth, laurentiu.palcu, meta.mr.watcher, meta.watcher, sgw
Version: unspecified   
Target Milestone: 1.4.2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
conf/local.conf
none
meta-sourcery-bblayers-conf
none
meta-sourcery-local-conf
none
meta-sourcery-bblayers-2-conf
none
meta-sourcery-local-2-conf
none
meta-sourcery-bblayers-3-conf
none
meta-sourcery-local-3-conf none

Description Laszlo Papp 2013-07-25 08:21:08 UTC
Created attachment 1356 [details]
conf/local.conf

bblayers.conf is the default generated with ". oe-init-build-env"

Tried using both csl and sourcery. Here you can find the error which is 100% reproducable:

ERROR: Multiple .bb files are due to be built which each provide virtual/arm-none-linux-gnueabi-libc-for-gcc (/home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/eglibc/eglibc_2.17.bb /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/meta/external-sourcery-toolchain.bb).
 This usually means one provides something the other doesn't and should.
ERROR: Multiple .bb files are due to be built which each provide virtual/libc (/home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/eglibc/eglibc_2.17.bb /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/meta/external-sourcery-toolchain.bb).
 This usually means one provides something the other doesn't and should.
ERROR: Multiple .bb files are due to be built which each provide virtual/libiconv (/home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/eglibc/eglibc_2.17.bb /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta/recipes-core/meta/external-sourcery-toolchain.bb).
 This usually means one provides something the other doesn't and should.

Here you can find the toolchain I am using for this test:

https://developer.ridgerun.com/wiki/index.php/Code_Sourcery_ARM_toolchain_2009q1-203

Note that, it is "gnueabi", so you have to obtain the right toolchain.

Using dylan 9.0.0, so not git. It would be awesome to have a fix for this in the upcoming bugfix release.

The command probably does not matter, but it was "bitbake core-image-minimal".

Same issue with 9.0.1 and master HEAD as with 9.0.0.

Perhaps, users should use the meta-sourcery layer instead of the oe-core import?

Any workaround? Otherwise, it is a full blocker. :'(
Comment 1 Laszlo Papp 2013-07-25 08:21:51 UTC
*** Bug 4899 has been marked as a duplicate of this bug. ***
Comment 2 Laurentiu Palcu 2013-07-25 09:13:20 UTC
I'm looking into it. I just reproduced it and, apparently, even though PREFERRED_PROVIDER is set correctly for virtual/arm-none-linux-gnueabi-libc-for-gcc, virtual/libc and virtual/libiconv to point to external-sourcery-toolchain, the error still appears.

I have no workaround or fix yet... I'll keep you posted if I've got any news.
Comment 3 Laszlo Papp 2013-07-25 09:35:44 UTC
I opened up an issue for meta-sourcery as well fwiw, but it would be nice to get the oe-core variant working:

https://github.com/MentorEmbedded/meta-sourcery/issues/9
Comment 4 Laszlo Papp 2013-07-25 14:33:48 UTC
Can anyone reproduce the error below with the following settings in conf/local.conf?

TCMODE = "external-csl"
EXTERNAL_TOOLCHAIN = "/usr/local/arm-2009q1"
TARGET_PREFIX = "arm-none-linux-gnueabi-"

INSANE_SKIP_external-sourcery-toolchain-dev += "ldflags"

PREFERRED_PROVIDER_linux-libc-headers = "external-sourcery-toolchain"
PREFERRED_PROVIDER_linux-libc-headers-dev = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc-initial = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc-intermediate = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}g++ = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}binutils = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}libc-initial = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}libc-for-gcc = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}compilerlibs = "external-sourcery-toolchain"
PREFERRED_PROVIDER_libgcc = "external-sourcery-toolchain"
PREFERRED_PROVIDER_eglibc = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/libc = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/libintl = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/libiconv = "external-sourcery-toolchain"
PREFERRED_PROVIDER_glibc-thread-db = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/linux-libc-headers = "external-sourcery-toolchain"
PREFERRED_PROVIDER_virtual/linux-libc-headers-dev = "external-sourcery-toolchain"
PREFERRED_PROVIDER_gdbserver = "external-sourcery-toolchain"

For the setup above, I am getting the error below:

bitbake core-image-minimal
Pseudo is not present but is required, building this first before the main build
Parsing recipes: 100% |######################################################################################################################################################################################################| Time: 00:00:27
Parsing of 831 .bb files complete (0 cached, 831 parsed). 1140 targets, 51 skipped, 0 masked, 0 errors.

Build Configuration:
BB_VERSION        = "1.18.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "arch-rolling"
TARGET_SYS        = "arm-foo-linux-gnueabi"
MACHINE           = "foo"
DISTRO            = "foo"
DISTRO_VERSION    = "0.1"
TUNE_FEATURES     = "armv5 thumb dsp"
TARGET_FPU        = "soft"
meta              
meta-yocto        
meta-yocto-bsp    = "master:3d4147a43126a7b031ac583221012442dde7beab"
meta-sourcery     = "master:832ce1937780bbb508050f02ac8a8bf309045fef"
meta-foo      = "master:3d4147a43126a7b031ac583221012442dde7beab"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 63 tasks of which 0 didn't need to be rerun and all succeeded.
Loading cache: 100% |########################################################################################################################################################################################################| ETA:  00:00:00
Loaded 1141 entries from dependency cache.

Build Configuration:
BB_VERSION        = "1.18.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "arch-rolling"
TARGET_SYS        = "arm-foo-linux-gnueabi"
MACHINE           = "foo"
DISTRO            = "foo"
DISTRO_VERSION    = "0.1"
TUNE_FEATURES     = "armv5 thumb dsp"
TARGET_FPU        = "soft"
meta              
meta-yocto        
meta-yocto-bsp    = "master:3d4147a43126a7b031ac583221012442dde7beab"
meta-sourcery     = "master:832ce1937780bbb508050f02ac8a8bf309045fef"
meta-foo      = "master:3d4147a43126a7b031ac583221012442dde7beab"

NOTE: Resolving any missing task queue dependencies
NOTE: preferred version 141 of udev not available (for item udev)
NOTE: versions of udev available: 182
NOTE: preferred version 141 of udev not available (for item udev-dev)
NOTE: versions of udev available: 182
NOTE: preferred version 141 of udev not available (for item udev-utils)
NOTE: versions of udev available: 182
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: Failed to fetch URL http://www.zlib.net/zlib-1.2.7.tar.bz2, attempting MIRRORS if available
No currently running tasks (83 of 1366)
ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored.
ERROR: Function failed: do_install (see /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/foo-builds/tmp/work/armv5te-foo-linux-gnueabi/external-sourcery-toolchain/2009q1-203-r22/temp/log.do_install.25593 for further information)
ERROR: Logfile of failure stored in: /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/foo-builds/tmp/work/armv5te-foo-linux-gnueabi/external-sourcery-toolchain/2009q1-203-r22/temp/log.do_install.25593
Log data follows:
| ERROR: ld.so: object 'libpseudo.so' from LD_PRELOAD cannot be preloaded: ignored.
| DEBUG: Executing shell function do_install
| cp: target '/home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/foo-builds/tmp/work/armv5te-foo-linux-gnueabi/external-sourcery-toolchain/2009q1-203-r22/image/lib' is not a directory
| ERROR: Function failed: do_install (see /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/foo-builds/tmp/work/armv5te-foo-linux-gnueabi/external-sourcery-toolchain/2009q1-203-r22/temp/log.do_install.25593 for further information)
ERROR: Task 31 (/home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta-sourcery/recipes/meta/external-sourcery-toolchain.bb, do_install) failed with exit code '1'
NOTE: Tasks Summary: Attempted 83 tasks of which 63 didn't need to be rerun and 1 failed.
No currently running tasks (83 of 1366)

Summary: 1 task failed:
  /home/lpapp/Projects/foo/Yocto/poky-dylan-9.0.0/meta-sourcery/recipes/meta/external-sourcery-toolchain.bb, do_install
Summary: There was 1 WARNING message shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Comment 5 Laszlo Papp 2013-07-25 14:34:29 UTC
Oh yeah, that error comes when just cloning meta-sourcery into my project, and trying to use based on the README from the meta-xilinx layer.
Comment 6 Christopher Larson 2013-07-25 15:05:01 UTC
My first question is why your path includes 'dylan' while all your branches are master, but that's immaterial. I haven't seen the error you have when using meta-sourcery, but I likely haven't tested a toolchain that old in some time. Would you be able to link me to the sourcery toolchain you're using? Thanks.
Comment 7 Laszlo Papp 2013-07-25 15:06:52 UTC
Repaste from my first post:

Here you can find the toolchain I am using for this test:

https://developer.ridgerun.com/wiki/index.php/Code_Sourcery_ARM_toolchain_2009q1-203
Comment 8 Christopher Larson 2013-07-25 15:09:16 UTC
Thanks, must have missed that. I'll attempt to reproduce.
Comment 9 Christopher Larson 2013-07-25 15:16:38 UTC
The first potential issue I see is the layer order listed in your build summary. Layer order in BBLAYERS has importance, I'm afraid. The order is highest priority to lowest. When bitbake goes to locate a relative path, like 'conf/distro/include/tcmode-external-sourcery.inc', it will search the layers (technically, BBPATH, but the layers add themselves to it) in priority order, and use the first it finds.

To summarize: if the main meta layer is before meta-sourcery in bblayers, its tcmode will be used rather than meta-sourcery's. I'll look into renaming the tcmode for the sourcery layer to avoid this usability problem in the future. Try moving meta-sourcery before meta in BBLAYERS in your conf/bblayers.conf, wipe tmp, and see how that goes.
Comment 10 Laszlo Papp 2013-07-25 15:42:54 UTC
This is the bblayers config by the way:

# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta-sourcery \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta-yocto \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta-yocto-bsp \
  "
BBLAYERS_NON_REMOVABLE ?= " \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta \
  /home/lpapp/Projects/polatis/Yocto/poky-dylan-9.0.0/meta-yocto \
  "
Comment 11 Laszlo Papp 2013-07-25 16:41:26 UTC
Created attachment 1360 [details]
meta-sourcery-bblayers-conf
Comment 12 Laszlo Papp 2013-07-25 16:41:58 UTC
Created attachment 1361 [details]
meta-sourcery-local-conf
Comment 13 Christopher Larson 2013-07-25 16:45:14 UTC
Your TCMODE is wrong for meta-sourcery, use external-sourcery, not external-csl. Also all the preferences are unnecessary.
Comment 14 Laszlo Papp 2013-07-25 16:46:29 UTC
Interesting because those are coming from the direct copy/paste of the meta-xilinx README.

Could they please file a bug against them?
Comment 15 Laszlo Papp 2013-07-25 16:52:26 UTC
Created attachment 1362 [details]
meta-sourcery-bblayers-2-conf
Comment 16 Laszlo Papp 2013-07-25 16:52:47 UTC
Created attachment 1363 [details]
meta-sourcery-local-2-conf
Comment 17 Christopher Larson 2013-07-25 16:55:10 UTC
Those configs look relatively sane. The INSANE_SKIP shouldn't be needed, as the recipe handles that already, but it won't harm anything.
Comment 18 Laszlo Papp 2013-07-25 17:02:38 UTC
Created attachment 1364 [details]
meta-sourcery-bblayers-3-conf
Comment 19 Laszlo Papp 2013-07-25 17:02:53 UTC
Created attachment 1365 [details]
meta-sourcery-local-3-conf
Comment 20 Saul Wold 2013-07-30 15:50:30 UTC
Merged to Master 1.5
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d1b189f4769e215419fbfad40d9da52f59a820d0

Needs to be merged into Dylan also
Comment 21 Laszlo Papp 2013-07-30 15:51:55 UTC
Unfortunately, I cannot verify it because I switched to meta-sourcery.

I have been told that is the official Code Sourcery support, and the one in oe-core is just a reference, and should not be widely adopted.
Comment 22 Paul Eggleton 2013-08-13 12:20:41 UTC
FWIW, the changes from master are now in dylan for 1.4.2:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=dylan&id=b4ba3269102424b614d91ca8be10501176042f62
Comment 23 Laszlo Papp 2013-08-13 12:26:17 UTC
OK, thanks. Feel free to ping me on IRC any evening to give it a try, but it is probably easy enough to verify, and I assume you did, so all good then.