Bug 11166

Summary: multiconfig: builds fails due to quilt-native errors when targeting arduino-101 and arduino-101-sss
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Henry Bruce <henry.bruce>
Component: devtools / tool chainAssignee: Juro Bystricky <juro.bystricky>
Status: VERIFIED FIXED QA Contact: oscar lopez arandas <oscarx.lopez.arandas>
Severity: normal    
Priority: Medium+ CC: jose.perez.carranza, juro.bystricky, meta.mr.watcher, meta.watcher, richard.purdie
Version: 2.2   
Target Milestone: 2.3 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
Compressed temp dir (multithreaded build)
none
Compressed temp dir (singleithreaded build)
none
cooker log, see duplicates none

Description Henry Bruce 2017-03-15 20:19:51 UTC
Steps to re-create
===================
$ git clone -b morty git://git.yoctoproject.org/poky
$ git clone -b morty git://git.yoctoproject.org/meta-zephyr
$ source poky/oe-init-build-env
$ bitbake-layers add-layer ../meta-zephyr
$ echo 'DISTRO = "zephyr"' >> conf/local.conf
$ echo 'MACHINE = "arduino-101"' >> conf/local.conf
$ echo 'BBMULTICONFIG = "arduino-101-sss"' >> conf/local.conf
$ mkdir conf/multiconfig
$ echo 'MACHINE = "arduino-101-sss"' >> conf/multiconfig/arduino-101-sss.conf
$ bitbake zephyr-ipm

Expected behavior
=================
Successful build

Observed behavior
=================
Build fails, either due to missing quilt-native 
ERROR: mc:arduino-101-sss:gettext-minimal-native-0.19.8.1-r0 do_patch: Command Error: 'quilt --quiltrc /media/md126p1/yocto/hbruce/morty/zephyr/build/tmp/sysroots/x86_64-linux/etc/quiltrc push' exited with 0  Output:
/bin/sh: quilt: command not found

or error in do_configure
ERROR: mc:arduino-101-sss:quilt-native-0.64-r0 do_configure: configure failed
ERROR: mc:arduino-101-sss:quilt-native-0.64-r0 do_configure: Function failed: do_configure (log file is located at /media/md126p1/yocto/hbruce/morty/zephyr/build/tmp/work/x86_64-linux/quilt-native/0.64-r0/temp/log.do_configure.2226)
ERROR: Task (/media/md126p1/yocto/hbruce/morty/zephyr/poky/meta/recipes-devtools/quilt/quilt-native_0.64.bb:do_configure) failed with exit code '1'

| configure: creating ./config.status
| config.status: creating Makefile
| ./config.status: line 1145: _ACAWK: command not found
| ./config.status: line 1194: s/
| /\\r/g
| else
|   cat
| fi < ./confiWoWOW/subs1.awk > ./confiWoWOW/subs.awk   || as_fn_error 0 could: No such file or directory
| sed: -e expression #1, char 232: unterminated `s' command
| awk: ./confiWoWOW/subs.awk:79: cat >>"$ac_tmp/subs1.awk" <<\_ACAWK &&
| awk: ./confiWoWOW/subs.awk:79:     ^ syntax error
| awk: ./confiWoWOW/subs.awk:79: cat >>"$ac_tmp/subs1.awk" <<\_ACAWK &&
| awk: ./confiWoWOW/subs.awk:79:                             ^ backslash not last character on line
| config.status: error: could not create Makefile

Workaround
==========
Set different TMPDIR in conf/multiconfig/arduino-101-sss.conf
Comment 1 Juro Bystricky 2017-03-15 22:42:21 UTC
Also observed with master 2.3. Seems "configure" creates a busted config.status.
The errors can be different, but seem to be always caused by a corrupted configure.status.

Probably a race condition while creating the file.
Comment 2 Juro Bystricky 2017-03-16 21:44:51 UTC
Created attachment 3650 [details]
Compressed temp dir (multithreaded build)
Comment 3 Juro Bystricky 2017-03-16 21:46:59 UTC
Created attachment 3651 [details]
Compressed temp dir (singleithreaded build)
Comment 4 Juro Bystricky 2017-03-16 21:51:29 UTC
I attached a snapshot of "temp" folder. The snapshot was saved after the very first time trying to build a recipe. Note there are two occurrences of several files: notably run.do_configure. 
I would expect a single occurrence. I suspect the two occurrences correspond to two different MACHINEs in the multiconfig setup (I think the "native" builds in x86_64-linux folder should be MACHINE agnostics. The two occurrences of run.XXX are logically the same, so one of them is redundant and removing one of them would most likely fix the problem.)

Some observations:
1. no problems: when multiconfig set up for two but identical MACHINEs
2. no problems: when running the original 2 MACHINE setup:
   $ BB_NUMBER_THREADS=1 bitbake multiconfig:*:zephyr-ipm
   (although there are double occurrences of log/run files as well)
   I also attached a snapshot of quilt "temp" folder with BB_NUMTHREADS=1.
3. I don't think this is "quilt" specific, it just happens to be the first task
   that breaks.
Comment 5 Juro Bystricky 2017-03-16 21:56:13 UTC
Created attachment 3652 [details]
cooker log, see duplicates

Cooker log, see the duplicate tasks, such as
recipe quilt-native-0.64-r0: task do_configure: Started
Comment 6 Richard Purdie 2017-03-16 22:06:32 UTC
Right, this is why I asked for the logs. The question is why bitbake is running two identical tasks...
Comment 7 Juro Bystricky 2017-04-03 18:00:02 UTC
Well, in the end it turned out to be a problem caused by meta-zephyr.
The root of the errors was the fact that various SITEINFO_EXTRA_DATAFUNCS and PACKAGEQA_EXTRA_MACHDEFFUNCS definitions were not located in a bbclass file but in various tune-xxx include files. This (I speculate) somehow silently corrupted tables in siteinfo.bbclass resulting in the reported problem.

Actually, the problem has nothing to do with SITEINFO_EXTRA_DATAFUNCS at all.
 
The problem can be reproduced easily if in "siteinfo_data" (from siteinfo.bbclass) we simply try to read any variable located in any of the include files, i.e.

var = d.getVar("DUMMY_XXX")

(The variable DUMMY_XXX must exist in order to reproduce the problem)

Fixed in:
http://git.yoctoproject.org/cgit/cgit.cgi/meta-zephyr/commit/?h=morty
Comment 8 oscar lopez arandas 2017-04-18 17:18:44 UTC
multiconfig: builds fails 

Build Configuration:
BB_VERSION        = "1.33.3"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i686-poky-linux"
MACHINE           = "core2-32"
DISTRO            = "poky - tiny"
DISTRO_VERSION    = "2.2+snapshot-20170417"
TUNE_FEATURES     = "m32 core2"
COMMIT: eff56e4f0d59b1d965a68e4f009b7f07717b7edd

Steps to re-create

===================
$ git clone -b morty git://git.yoctoproject.org/poky
$ git clone -b morty git://git.yoctoproject.org/meta-zephyr
$ source poky/oe-init-build-env
$ bitbake-layers add-layer ../meta-zephyr
$ echo 'DISTRO = "zephyr"' >> conf/local.conf
$ echo 'MACHINE = "arduino-101"' >> conf/local.conf
$ echo 'BBMULTICONFIG = "arduino-101-sss"' >> conf/local.conf
$ mkdir conf/multiconfig
$ echo 'MACHINE = "arduino-101-sss"' >> conf/multiconfig/arduino-101-sss.conf
$ bitbake zephyr-ipm

EXPECTED OUTCOME:

Successful build

ACTUAL OUTCOME:

Build successfully

NOTES:

----snip----

oscar@oscar-ThinkPad-T540p:~/build/conf/multiconfig$ ls
arduino-101-sss.conf