Bug 843 - kernel: support standalone kernel builds
Summary: kernel: support standalone kernel builds
Status: VERIFIED FIXED
Alias: None
Product: Kernel
Classification: Yocto Project Subprojects
Component: kernel-tooling (show other bugs)
Version: unspecified
Hardware: All Multiple
: Low enhancement
Target Milestone: 1.3 M3
Assignee: Bruce Ashfield
QA Contact:
URL:
Whiteboard: (P3)(QA Testing Completed)(Done)
Depends on:
Blocks:
 
Reported: 2011-03-10 12:12 UTC by Bruce Ashfield
Modified: 2012-09-29 09:00 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments
base.cfg.sanitized (6.12 KB, application/octet-stream)
2012-07-23 09:07 UTC, Yi Zhao
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Bruce Ashfield 2011-03-10 12:12:58 UTC
The kernel tools are already captured in the kernel
itself and they are there to support a standalone kernel build.

But having the separate kern-tools repo tends to blur that. We
should merge the tools into yocto/base and use them as part
of every kernel build. This means that they always stay in
sync.
Comment 1 Darren Hart 2012-05-01 15:57:44 UTC
Bruce, where are we with this. Can we wrap it up for 1.3?
Comment 2 Darren Hart 2012-05-01 15:57:52 UTC
.
Comment 3 Bruce Ashfield 2012-05-01 22:15:21 UTC
This will complete for 1.3.
Comment 4 Darren Hart 2012-05-02 16:24:29 UTC
Bruce, marking as P3, can you provide a sizing?
Comment 5 Bruce Ashfield 2012-05-02 23:48:19 UTC
sized. 3 days work. mainly testing.
Comment 6 Bruce Ashfield 2012-07-18 01:40:18 UTC
The tree can now be built without the build system.

See: 

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/buildall

and

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/sample-buildallrc

For the details.
Comment 7 Yi Zhao 2012-07-19 05:48:22 UTC
I tried to verify this feature but encountered some issues.

Here are my steps:
1. $ git clone git://git.yoctoproject.org/yocto-kernel-tools
2. $ cp sample-buildallrc ~/.buildallrc
3. $ /home/builder/yocto-kernel-tools/tools/buildall 


I got the following failed messages:
#############
Sourcing custom settings in /home/builder/.buildallrc
Using git://git.yoctoproject.org/linux-yocto-3.4 as upstream source
You can use "export TREE=git://myhost.com/mypath" to change.
Doing initial setup....
Configuration of arm-versatile-926ejs-preempt-rt-meta failed
Configuration of arm-versatile-926ejs-standard-meta failed
Configuration of atom-pc-preempt-rt-meta failed
Configuration of atom-pc-standard-meta failed
Configuration of beagleboard-preempt-rt-meta failed
Configuration of beagleboard-standard-meta failed
Configuration of cedartrail-preempt-rt-meta failed
Configuration of cedartrail-standard-meta failed
Configuration of chiefriver-preempt-rt-meta failed
Configuration of chiefriver-standard-meta failed
Configuration of common-pc-64-preempt-rt-meta failed
Configuration of common-pc-64-standard-meta failed
Configuration of common-pc-preempt-rt-meta failed
Configuration of common-pc-standard-meta failed
Building standard/tiny/common-pc for common-pc-tiny-meta, status:
##############

$ cat kernel_builds/beagleboard-standard/cfg.log
############### 
[INFO] Configuring target/machine combo: "standard/beagleboard"
[INFO] collecting configs in ./meta/meta-series 
ERROR: could not sanitize configuration fragments
   errors are logged in /home/builder/yocto-kernel-tools/tools/kernel_builds/linux/meta/cfg/standard/beagleboard/config.log
#################

$ cat /home/builder/yocto-kernel-tools/tools/kernel_builds/linux/meta/cfg
/standard/beagleboard/config.log
#############
[INFO] Sanitizing meta/cfg/kernel-cache/ktypes/base/base.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/kgdb/kgdb.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/lttng/lttng.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/blktrace/blktrace.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/systemtap/systemtap.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/utrace/utrace.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/arch/arm/arm.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/hrt/hrt.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/ftrace/ftrace.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/unionfs/unionfs.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/cgroups/cgroups.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/namespaces/namespaces.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/namespaces/namespaces-experimental.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/features/fuse/fuse.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/ktypes/standard/standard.cfg
[INFO] Sanitizing meta/cfg/kernel-cache/cfg/devtmpfs.cfg
[ERROR] Kern frag meta/cfg/kernel-cache/cfg/devtmpfs.cfg does not exist
##############

Is there anything that I do wrong? Thanks.
Comment 8 Bruce Ashfield 2012-07-19 15:32:34 UTC
Nothing is wrong with your use case, but you somehow don't have the
latest linux-yocto-3.4 kernel.

That error is good, in the sense that the tools are telling you about
an issue with the cfg data.

  [ERROR] Kern frag meta/cfg/kernel-cache/cfg/devtmpfs.cfg does not exist

But if you look at the latest meta branch, there shouldn't be any
references to devtmpfs in that location anymore.

http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto-3.4/tree/meta/cfg/kernel-cache/ktypes/standard/standard.scc?h=meta

I only see the following references:

./cfg/fs/devtmpfs.scc:define KFEATURE_DESCRIPTION "Enable devtmpfs for tmpfs/ramfs support at early bootup"
./cfg/fs/devtmpfs.scc:kconf non-hardware devtmpfs.cfg
./ktypes/standard/standard.scc:include cfg/fs/devtmpfs.scc
./ktypes/preempt-rt/preempt-rt.scc:include cfg/fs/devtmpfs.scc

What are you seeing ?
Comment 9 Yi Zhao 2012-07-23 09:06:29 UTC
(In reply to comment #8)
> Nothing is wrong with your use case, but you somehow don't have the
> latest linux-yocto-3.4 kernel.
> 
> That error is good, in the sense that the tools are telling you about
> an issue with the cfg data.
> 
>   [ERROR] Kern frag meta/cfg/kernel-cache/cfg/devtmpfs.cfg does not exist
> 
> But if you look at the latest meta branch, there shouldn't be any
> references to devtmpfs in that location anymore.
> 
> http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto-3.4/tree/meta/cfg/
> kernel-cache/ktypes/standard/standard.scc?h=meta
> 
> I only see the following references:
> 
> ./cfg/fs/devtmpfs.scc:define KFEATURE_DESCRIPTION "Enable devtmpfs for
> tmpfs/ramfs support at early bootup"
> ./cfg/fs/devtmpfs.scc:kconf non-hardware devtmpfs.cfg
> ./ktypes/standard/standard.scc:include cfg/fs/devtmpfs.scc
> ./ktypes/preempt-rt/preempt-rt.scc:include cfg/fs/devtmpfs.scc
> 
> What are you seeing ?

Here are my seeing for beagleboard-standard building failed:
###########
$ pwd
/home/builder/yocto-kernel-tools/tools/kernel_builds/linux
$ git branch
  eg20t
  emgd
  emgd-1.10
  emgd-1.14
  master
  meta
  standard/arm-versatile-926ejs
  standard/base
* standard/beagleboard
  standard/cedartrail
  standard/common-pc-64/base
  standard/common-pc-64/chiefriver
  standard/common-pc-64/jasperforest
  standard/common-pc-64/romley
  standard/common-pc-64/sugarbay
  standard/common-pc/atom-pc
  standard/common-pc/base
  standard/crownbay
  standard/emenlow
  standard/fishriver
  standard/fri2
  standard/fsl-mpc8315e-rdb
  standard/mti-malta32
  standard/mti-malta64
  standard/preempt-rt/base
  standard/preempt-rt/fri2
  standard/preempt-rt/qemuppc
  standard/preempt-rt/routerstationpro
  standard/qemuppc
  standard/routerstationpro
  standard/sys940x
  standard/tiny/base
  standard/tiny/common-pc
$ pwd
/home/builder/yocto-kernel-tools/tools/kernel_builds/linux/meta/cfg/kernel-cache
$ grep "devtmpfs" ./* -rnI
./cfg/fs/devtmpfs.scc:1:define KFEATURE_DESCRIPTION "Enable devtmpfs for tmpfs/ramfs support at early bootup"
./cfg/fs/devtmpfs.scc:4:kconf non-hardware devtmpfs.cfg
./ktypes/base/base.cfg.sanitized:20:# /home/builder/yocto-kernel-tools/tools/kernel_builds/linux/meta/cfg/kernel-cache/cfg/devtmpfs.cfg
./ktypes/standard/standard.scc:9:include cfg/fs/devtmpfs.scc
./ktypes/preempt-rt/preempt-rt.scc:5:include cfg/fs/devtmpfs.scc

############
Comment 10 Yi Zhao 2012-07-23 09:07:59 UTC
Created attachment 649 [details]
base.cfg.sanitized

I attached the base.cfg.sanitized file
Comment 11 Bruce Ashfield 2012-07-23 20:33:24 UTC
you are still seeing different configuration fragments than I am, I'll have a closer
look at this soon, but this still looks like something is out of sync.
Comment 12 Bruce Ashfield 2012-07-25 18:00:33 UTC
Update on this: the meta branch of the tree had some out of date
meta-series files. They are used by buildall, and that was where the old
cfg was coming from. I just pushed an update to the branch, so a fresh clone
of the 3.4 kernel will work (and only do this against the 3.4 kernel). 

Some tools update will follow to ensure that the files are in sync, but that is
on top of this original feature.
Comment 13 Yi Zhao 2012-09-29 09:00:50 UTC
Steps:
1. $ git clone git://git.yoctoproject.org/yocto-kernel-tools
2. $ cp sample-buildallrc ~/.buildallrc
3. edit the file ~/.buildallrc to build a beagleboard-standard-meta kernel.
4. $ /home/builder/yocto-kernel-tools/tools/buildall 

The kernel build succeeded:

========================= Build Summary ============================
Kernel config: Total: 1, Pass: 1, Fail: 0
Kernel build: Total: 1, Pass: 1, Fail: 0
====================================================================

I can find the kernel image in yocto-kernel-tools/tools/kernel_builds/beagleboard-standard-meta/ directory. And no error message in build.log.

Verified it.

yocto-kernel-tools: master/a04138a02644eada0d012196e5ac3db4f516114d