Bug 9020 - populate_sdk fails when I have another version of gcc in my own layer
Summary: populate_sdk fails when I have another version of gcc in my own layer
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: configuration (show other bugs)
Version: unspecified
Hardware: x86 x86_64
: Medium+ normal
Target Milestone: 2.1 M4
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-02-01 13:53 UTC by Maksim E. Kozlov
Modified: 2016-03-29 15:11 UTC (History)
1 user (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Maksim E. Kozlov 2016-02-01 13:53:06 UTC
Error occurs when I try to do '-c populate_sdk'

I have my own layer which contains additional gcc version (4.8.4) which was just copied from older version of OE core with only modifications of paths of included files to get inc files from current dirrectory, not from meta/recipes-devtools/gcc

Layer has BBFILE_PRIORITY_my-layer = "6" and my distro has GCCVERSION ?= "4.8%". With this configuration all build tasks are working fine (include -c populate_sdk).
Bat when I change GCCVERSION ?= "5.%" I get error.

At first I faced with strange error during the nativesdk-libgcc-initial building:

...

| /work/tmp/ram/proteus/tmp-uclibc/work-shared/gcc-5.3.0-r0/gcc-5.3.0/libgcc/config/i386/cpuinfo.c: In function 'get_available_features':
| /work/tmp/ram/proteus/tmp-uclibc/work-shared/gcc-5.3.0-r0/gcc-5.3.0/libgcc/config/i386/cpuinfo.c:303:17: error: 'bit_AVX512F' undeclared (first use in this function)
|        if (ebx & bit_AVX512F)
|                  ^

...

| make: *** [cpuinfo.o] Error 1
| make: *** Waiting for unfinished jobs....

...

| WARNING: exit code 1 from a shell command.
| ERROR: oe_runmake failed
| ERROR: Function failed: do_compile (log file is located at /work/tmp/ram/proteus/tmp-uclibc/work/x86_64-nativesdk-oesdk-linux/nativesdk-libgcc-initial/5.3.0-r0/temp/log.do_compile.12613)
ERROR: Task 5 (virtual:nativesdk:/work/openembedded/meta/recipes-devtools/gcc/libgcc-initial_5.3.bb, do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 176 tasks of which 0 didn't need to be rerun and 1 failed.
No currently running tasks (176 of 181)

Summary: 1 task failed:
  virtual:nativesdk:/work/openembedded/meta/recipes-devtools/gcc/libgcc-initial_5.3.bb, do_compile
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

After some investigation I found that it is building not only gcc 5.3.0 which defined by GCCVERSION but also 4.8.4 version from my layer. And after that I print current and preferred versions of all recipes and saw strange thing:

$ bitbake nativesdk-libgcc-initial -s | grep gcc
gcc                                                 :4.8.4-r0                 :5.3.0-r0
gcc-cross-canadian-mipsel                           :4.8.4-r0                 :5.3.0-r0
gcc-cross-initial-mipsel                            :4.8.4-r0                 :5.3.0-r0
gcc-cross-mipsel                                    :4.8.4-r0                 :5.3.0-r0
gcc-crosssdk-initial-x86_64                         :4.8.4-r0                          
gcc-crosssdk-x86_64                                 :4.8.4-r0                          
gcc-runtime                                         :4.8.4-r0                 :5.3.0-r0
gcc-source-4.8.4                                    :4.8.4-r0                          
gcc-source-4.9.3                                    :4.9.3-r0                          
gcc-source-5.3.0                                    :5.3.0-r0                          
gccmakedep                                         1:1.0.3-r3                          
libgcc                                              :4.8.4-r0                 :5.3.0-r0
libgcc-initial                                      :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc                                       :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc-runtime                               :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc-sanitizers                            :4.8.4-r0                 :5.3.0-r0
nativesdk-libgcc                                    :4.8.4-r0                 :5.3.0-r0
nativesdk-libgcc-initial                            :4.8.4-r0                 :5.3.0-r0

Here we can see that not for all gcc packages preferred version defined correctly - for gcc-crosssdk-initial-x86_64 and gcc-crosssdk-x86_64 preferred version udefined.
I think this is bug because 

At first, when I define SDKMACHINE ?= "i686" I see

$ bitbake nativesdk-libgcc-initial -s | grep gcc
gcc                                                 :4.8.4-r0                 :5.3.0-r0
gcc-cross-canadian-mipsel                           :4.8.4-r0                 :5.3.0-r0
gcc-cross-initial-mipsel                            :4.8.4-r0                 :5.3.0-r0
gcc-cross-mipsel                                    :4.8.4-r0                 :5.3.0-r0
gcc-crosssdk-i686                                   :4.8.4-r0                 :5.3.0-r0
gcc-crosssdk-initial-i686                           :4.8.4-r0                 :5.3.0-r0
gcc-runtime                                         :4.8.4-r0                 :5.3.0-r0
gcc-source-4.8.4                                    :4.8.4-r0                          
gcc-source-4.9.3                                    :4.9.3-r0                          
gcc-source-5.3.0                                    :5.3.0-r0                          
gccmakedep                                         1:1.0.3-r3                          
libgcc                                              :4.8.4-r0                 :5.3.0-r0
libgcc-initial                                      :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc                                       :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc-runtime                               :4.8.4-r0                 :5.3.0-r0
nativesdk-gcc-sanitizers                            :4.8.4-r0                 :5.3.0-r0
nativesdk-libgcc                                    :4.8.4-r0                 :5.3.0-r0
nativesdk-libgcc-initial                            :4.8.4-r0                 :5.3.0-r0

and '-c populate_sdk' work fine

At second, when I decrease my layer priority to "5", I see

$ bitbake nativesdk-libgcc-initial -s | grep gcc
gcc                                                 :5.3.0-r0                          
gcc-cross-canadian-mipsel                           :5.3.0-r0                          
gcc-cross-initial-mipsel                            :5.3.0-r0                          
gcc-cross-mipsel                                    :5.3.0-r0                          
gcc-crosssdk-initial-x86_64                         :5.3.0-r0                          
gcc-crosssdk-x86_64                                 :5.3.0-r0                          
gcc-runtime                                         :5.3.0-r0                          
gcc-source-4.8.4                                    :4.8.4-r0                          
gcc-source-4.9.3                                    :4.9.3-r0                          
gcc-source-5.3.0                                    :5.3.0-r0                          
gccmakedep                                         1:1.0.3-r3                          
libgcc                                              :5.3.0-r0                          
libgcc-initial                                      :5.3.0-r0                          
nativesdk-gcc                                       :5.3.0-r0                          
nativesdk-gcc-runtime                               :5.3.0-r0                          
nativesdk-gcc-sanitizers                            :5.3.0-r0                          
nativesdk-libgcc                                    :5.3.0-r0                          
nativesdk-libgcc-initial                            :5.3.0-r0                          

and '-c populate_sdk' work fine as well

So I think that it happens something wrong with GCCVERSION when SDKMACHINE is defined as "x84_64"
Comment 1 Richard Purdie 2016-02-01 21:59:27 UTC
Which version of the base metadata are you using?

We did used to see problems with the PREFERRED_VERSION settings for gcc-crosssdk not working. This was one of the issues which the datastore reimplementation allowed us to fix but that wouldn't fix older branches.

The settings come from:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/conf/distro/include/tcmode-default.inc

PREFERRED_VERSION_gcc-crosssdk-${SDK_ARCH} ?= "${SDKGCCVERSION}"
PREFERRED_VERSION_gcc-crosssdk-initial-${SDK_ARCH} ?= "${SDKGCCVERSION}"

however you could try setting these manually and seeing if that helps you work around the issue?

PREFERRED_VERSION_gcc-crosssdk-i686 ?= "${SDKGCCVERSION}"
PREFERRED_VERSION_gcc-crosssdk-initial-i686 ?= "${SDKGCCVERSION}"
PREFERRED_VERSION_gcc-crosssdk-x86_64 ?= "${SDKGCCVERSION}"
PREFERRED_VERSION_gcc-crosssdk-initial-x86_64 ?= "${SDKGCCVERSION}"

should do it.
Comment 2 Maksim E. Kozlov 2016-02-02 09:03:52 UTC
(In reply to comment #1)
> Which version of the base metadata are you using?

I'm using master branch from 
git://git.openembedded.org/openembedded-core repository, but I did this on the jethro branch as well... the same effect

> 
> We did used to see problems with the PREFERRED_VERSION settings for
> gcc-crosssdk not working. This was one of the issues which the datastore
> reimplementation allowed us to fix but that wouldn't fix older branches.
> 
> The settings come from:
> 
> http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/conf/distro/include/
> tcmode-default.inc
> 
> PREFERRED_VERSION_gcc-crosssdk-${SDK_ARCH} ?= "${SDKGCCVERSION}"
> PREFERRED_VERSION_gcc-crosssdk-initial-${SDK_ARCH} ?= "${SDKGCCVERSION}"
> 
> however you could try setting these manually and seeing if that helps you
> work around the issue?
> 
> PREFERRED_VERSION_gcc-crosssdk-i686 ?= "${SDKGCCVERSION}"
> PREFERRED_VERSION_gcc-crosssdk-initial-i686 ?= "${SDKGCCVERSION}"
> PREFERRED_VERSION_gcc-crosssdk-x86_64 ?= "${SDKGCCVERSION}"
> PREFERRED_VERSION_gcc-crosssdk-initial-x86_64 ?= "${SDKGCCVERSION}"
> 
> should do it.
I tried to set these variables explicitly but with no effect

Since I'm using git://git.openembedded.org/openembedded-core repository, I tried to check how it works on the master branch from git://git.yoctoproject.org/poky.git repo - the same wrong behavior

But when I switched to the jethro branch and tried again, I found that it works correctly

So this bug appears in 'master' branches of both repositories and in the jethro branch from git://git.openembedded.org/openembedded-core repo, but doesn't appear in the jethro branch from git://git.yoctoproject.org/poky.git
Comment 3 Richard Purdie 2016-02-24 17:43:05 UTC
I think I tracked down the issue illustrated by the following patch:

diff --git a/bitbake/lib/bb/providers.py b/bitbake/lib/bb/providers.py
index 68c8d59..ffa532c 100644
--- a/bitbake/lib/bb/providers.py
+++ b/bitbake/lib/bb/providers.py
@@ -121,11 +121,14 @@ def findPreferredProvider(pn, cfgData, dataCache, pkg_pn = None, item = None):
     preferred_file = None
     preferred_ver = None
 
-    localdata = data.createCopy(cfgData)
-    localdata.setVar('OVERRIDES', "%s:pn-%s:%s" % (data.getVar('OVERRIDES', localdata), pn, pn))
-    bb.data.update_data(localdata)
+    # pn can contain '_', e.g. gcc-cross-x86_64 and an override cannot
+    # hence we do this manually rather than use OVERRIDES
+    preferred_v = cfgData.getVar("PREFERRED_VERSION_pn-%s" % pn, True)
+    if not preferred_v:
+        preferred_v = cfgData.getVar("PREFERRED_VERSION_%s" % pn, True)
+    if not preferred_v:
+        preferred_v = cfgData.getVar("PREFERRED_VERSION", True)
 
-    preferred_v = localdata.getVar('PREFERRED_VERSION', True)
     if preferred_v:
         m = re.match('(\d+:)*(.*)(_.*)*', preferred_v)
         if m:
Comment 4 Maksim E. Kozlov 2016-03-04 14:17:38 UTC
It seems working. Thanks