Bug 1677

Summary:	[Performance] Improve speed of packaging for all-arch recipes
Product:	[Build System, Metadata & Runtime] Meta-yocto	Reporter:	Paul Eggleton <bluelightning>
Component:	meta-yocto	Assignee:	Paul Eggleton <bluelightning>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	Low	CC:	jessica.zhang, poky.bs.watcher, poky.watcher, sgw
Version:	unspecified
Target Milestone:	Future
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	---

Description Paul Eggleton 2011-10-14 09:06:09 UTC

Web applications (e.g. Webmin) don't usually require any compilation but can contain a lot of files. The packaging code in package.bbclass does a lot of walking of file trees and processing of files, some of which is extraneous for non-binaries (e.g. debug splitting / stripping). There must be some optimisations we can make to speed this up - at the moment building webmin takes 21 minutes on my system and most of that is spent within the do_package task.

Most of these types of recipes should inherit allarch.bbclass, so where we can't do any generic optimisations, some other extraneous processing may be able to be disabled from there - I have a patch ready to do this for the shared library dependencies at least.

Comment 1 Paul Eggleton 2011-11-16 10:01:45 UTC

I made some improvements to allarch.bbclass which have been merged in commit b96390473d786924d6081319db8cbf31dd3e7d9a; this brings the time to build webmin down to about 12 minutes compared to 21 minutes previously. I still think we can do better however so this bug should remain open.

Comment 2 Richard Purdie 2013-02-14 11:33:02 UTC

How is this after the recent generic do_package improvements? We should be better again...

Comment 3 Richard Purdie 2013-02-14 11:34:23 UTC

Specifically, I think we really need to turn this into specific action items to target. Reassigning to Paul to either close afer checking the packaging is faster after recent improvements, or to point at specific areas to improve in do_package.

Comment 4 Paul Eggleton 2013-03-07 15:08:10 UTC

So for the webmin recipe with current master there appears to be a significant improvement - even on my laptop it's around 4 minutes to build it now (although it was quite a while since my previous tests so add salt to taste).

With all of the improvements in allarch.bbclass since this bug was filed, the webmin recipe only has one workaround left which is to override package_do_pkgconfig to do nothing; however when I removed this it didn't increase the build time measurably, so I suspect that could be dropped.

Thus I'm going to mark this one as resolved; webmin is a fairly extreme case in terms of the number of files it contains and I think it's building in a reasonable time now.