Bug 10918

Summary:	devtool fails to build linux-yocto with yocto-bsp generated configuration
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Andrea Galbusera <gizero>
Component:	Scripts and Tools	Assignee:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	bluelightning, bruce.ashfield, randy.macleod, ross.burton
Version:	unspecified
Target Milestone:	2.4 M1
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Andrea Galbusera 2017-01-12 10:55:26 UTC

Steps to Repeat
================
$ git clone git://git.yoctoproject.org/poky
$ cd poky
$ git checkout master
$ source oe-init-build-env
$ yocto-bsp create -c my-qemux86 qemu
Answers: 1/y/n/2/n/n/y

* add meta-my-qemux86 to bblayers.conf
* Comment AUTOREVs in meta-my-qemux86/recipes-kernel/linux/linux-yocto_4.8.bbappend (to stick with current srcrevs)

$ MACHINE="my-qemux86" devtool modify linux-yocto (no complaints)
$ MACHINE="my-qemux86" devtool build linux-yocto

Observed Behavior
==================
Build Configuration:                                                                                                                                                                               [90/2513]
BB_VERSION        = "1.32.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "universal"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "my-qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta
meta-poky
meta-yocto-bsp    = "master:81021bc0aa0f64e67535f6a9551e921a64fe4395"
meta-my-qemux86
workspace         = "<unknown>:<unknown>"

Initialising tasks: 100% |############################################################| Time: 0:00:00
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
ERROR: linux-yocto-4.8.12+git999-r0.1 do_kernel_metadata: Could not generate configuration queue for my-qemux86.
ERROR: linux-yocto-4.8.12+git999-r0.1 do_kernel_metadata: Function failed: do_kernel_metadata (log file is located at /home/gizero/work/upstreaming/build2/tmp/work/my_qemux86-poky-linux/linux-yocto/4.8.12
+git999-r0.1/temp/log.do_kernel_metadata.7604)
ERROR: Logfile of failure stored in: /home/gizero/work/upstreaming/build2/tmp/work/my_qemux86-poky-linux/linux-yocto/4.8.12+git999-r0.1/temp/log.do_kernel_metadata.7604
Log data follows:
| DEBUG: Executing shell function do_kernel_metadata
| ERROR: Could not generate configuration queue for my-qemux86.
| ERROR: could not find file my-qemux86.scc, included from /home/gizero/work/upstreaming/build2/workspace/sources/linux-yocto/oe-local-files/my-qemux86-standard.scc
| ERROR: could not process input files: /home/gizero/work/upstreaming/build2/workspace/sources/linux-yocto/oe-local-files/my-qemux86-standard.scc /home/gizero/work/upstreaming/build2/workspace/sources/lin
ux-yocto/oe-local-files/my-qemux86-standard.scc /home/gizero/work/upstreaming/build2/workspace/sources/linux-yocto/oe-local-files/my-qemux86-user-config.cfg /home/gizero/work/upstreaming/build2/workspace/
sources/linux-yocto/oe-local-files/my-qemux86-user-features.scc features/nfsd/nfsd-enable.scc features/debug/printk.scc features/netfilter/netfilter.scc cfg/virtio.scc
|        See /tmp/tmp.CIOS1kojOq for details
| ERROR: Function failed: do_kernel_metadata (log file is located at /home/gizero/work/upstreaming/build2/tmp/work/my_qemux86-poky-linux/linux-yocto/4.8.12+git999-r0.1/temp/log.do_kernel_metadata.7604)
ERROR: Task (/home/gizero/work/upstreaming/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_kernel_metadata) failed with exit code '1'
NOTE: Tasks Summary: Attempted 397 tasks of which 393 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /home/gizero/work/upstreaming/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_kernel_metadata
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.


Expected Behavior
==================
Build completes successfully

Comment 1 Andrea Galbusera 2017-01-12 11:00:02 UTC

Seems more a yocto-bsp's issue than a devtool's one. The missing files fails to be copied to workspace/sources/linux-yocto/oe-local-files/ by "devtool modify" because they are not included in the meta-my-qemux86/recipes-kernel/linux/linux-yocto_4.8.bbappend's SRC_URI as generated by yocto-bsp. Manually adding them seems to fix the thing.

Comment 2 Leonardo Sandoval Gonzalez 2017-04-07 17:35:00 UTC

I got the same Andrea's conclusion and I will rephrase it with my comments:

Paul,

this is the problem: the yocto-bsp creates a bbappend and includes some scc files and inside these, there are references to other scc not tracked at SRC. When devtool modify is executed, the latter does not move all scc to the workspace/sources/linux-yocto/oe-local-files folder, leading to the original problem.

Another workaround is to flip priorities between meta-my-* and workspace, the latter having lower priority than the later. In this case, scc are found.

I will work on a fix at devtool, but I just wanted to let the latest findings.

Comment 3 Leonardo Sandoval Gonzalez 2017-04-13 18:41:33 UTC

Moving to 2.4M1, not a blocker for 2.3. In parallel, investigation is being done by Bruce Ashfield

Comment 4 Leonardo Sandoval Gonzalez 2017-06-09 20:38:53 UTC

Sent series to review: https://lists.yoctoproject.org/pipermail/poky/2017-June/011020.html

Comment 5 Leonardo Sandoval Gonzalez 2017-06-14 17:06:49 UTC

Patch at poky:

commit 2db8f3265e3b253f9e21301661f9fbe102aa595a
Refs: yocto-2.3-516-g2db8f3265e3
Author:     Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
AuthorDate: Fri Jun 9 13:32:28 2017 -0700
Commit:     Richard Purdie <richard.purdie@linuxfoundation.org>
CommitDate: Tue Jun 13 10:47:09 2017 +0100

    linux-yocto: include missing description files and conf fragments

    Include missing description files and configuration fragments into the SRC_URI.

    [YOCTO #10918]