Bug 11581 - Performance enhancement for kernel recipe kernel_configcheck
Summary: Performance enhancement for kernel recipe kernel_configcheck
Status: RESOLVED WORKSFORME
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: kernel (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 2.4 M4
Assignee: Bruce Ashfield
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-05-25 08:12 UTC by jonathan.yong
Modified: 2017-09-08 14:33 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jonathan.yong 2017-05-25 08:12:49 UTC
Right now, kernel config check is taking in excess of 4000 seconds before spending merely 200s to compile.

The kernel config is as such:

SRC_URI += "file://defconfig \
file://bsp/${BSP_SUBTYPE}/intel-dev.scc \
"
SRC_URI += "${@bb.utils.contains('DISTRO_FEATURES', 'hardened', 'file://bsp/${BSP_SUBTYPE}/security.scc', '', d)}"


defconfig is a basic kernel config file with defaults
intel-dev contains intel specific hardware, which includes about 8 cfg files.
security.scc pulls in a single cfg file for kernel security features.
Comment 1 Bruce Ashfield 2017-05-25 12:39:29 UTC
defconfgs are a pathological case, since they contain thousands of options which all need to be checked.

That being said, there are improvements that can be done here, and I've identified a way to shortcut the process.
Comment 2 Bruce Ashfield 2017-08-16 01:18:22 UTC
While working on another bug in master, I was retrying the optimizations that have already been done and I'm seeing about 14seconds to process a defconfig:

---------------
yow-bashfiel-d4 [/home/bruc...trib/build]> time MACHINE=qemunios2 bitbake -f -c kernel_configcheck virtual/kernel

NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 296 tasks of which 295 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages shown.

real    0m14.499s
user    0m6.080s
sys     0m1.828s
-------------

The size of the defconfig is this:

cat defconfig  | wc -l
1575

And the config.queue looks like this:

yow-bashfiel-d4 [/home/bruc...ernel-meta]> pwd
/home/bruce/poky-contrib/build/tmp/work-shared/qemunios2/kernel-source/.kernel-meta
yow-bashfiel-d4 [/home/bruc...ernel-meta]> cat config.queue 
configs//./defconfig # non-hardware
configs//features/nfsd/nfsd.cfg # non-hardware
configs//features/debug/printk.cfg # non-hardware
configs//features/netfilter/netfilter.cfg # non-hardware
configs//cfg/virtio.cfg # non-hardware

------------------------

So the optimizations/streamlining do seem to be holding.

Can you see if this is still happening and send a layer / machine configuration for me to use ?
Comment 3 jonathan.yong 2017-08-16 01:28:19 UTC
Hi Bruce,

How/where do I get the new code to test out the changes?
Comment 4 Bruce Ashfield 2017-08-16 01:33:32 UTC
I just pushed all of my staged commits to poky-config zedd/wip

It is poky master + a few of my changes.

I didn't do much in particular to fix this specific issue, which is why I'm interested to see if you are still seeing the slow processing, and getting that config. 

With defconfigs the processing is supposed to short-circuit and save a lot of time, so it shouldn't be hard to fix if it is still happening.
Comment 5 Bruce Ashfield 2017-08-16 01:33:52 UTC
urk. that last comment should be "poky-contrib".
Comment 6 Stephen K Jolley 2017-08-18 16:39:54 UTC
It appears this is no longer a NEEDINFO bug.  If it is, please restate the question.
Comment 7 Bruce Ashfield 2017-08-18 16:43:14 UTC
Nope. Definitely still a needinfo. I asked a question if the performance issue was still present.

Until I see a yes/no, answer, and steps to reproduce it is needsinfo.
Comment 8 jonathan.yong 2017-09-07 23:09:30 UTC
I don't see the problem happening anymore, so you can go ahead and close this.
Comment 9 Stephen K Jolley 2017-09-08 14:22:36 UTC
It appears this is no longer a NEEDINFO bug.  If it is, please restate the question.
Comment 10 Bruce Ashfield 2017-09-08 14:26:51 UTC
(In reply to comment #9)
> It appears this is no longer a NEEDINFO bug.  If it is, please restate the
> question.

Again. 

This is a yes/no answer required to "does this still happen".
Comment 11 Stephen K Jolley 2017-09-08 14:28:59 UTC
Bruce, he answered no, he is no longer seeing it.  I would close the bug works for me unless you have something else you want from him than that.
Comment 12 Bruce Ashfield 2017-09-08 14:33:40 UTC
ok. I can do that.

We can re-open this if it happens again.