Bug 1677 - [Performance] Improve speed of packaging for all-arch recipes
Summary: [Performance] Improve speed of packaging for all-arch recipes
Status: RESOLVED FIXED
Alias: None
Product: Meta-yocto
Classification: Build System, Metadata & Runtime
Component: meta-yocto (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Low enhancement
Target Milestone: Future
Assignee: Paul Eggleton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2011-10-14 09:06 UTC by Paul Eggleton
Modified: 2013-03-07 15:08 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Eggleton 2011-10-14 09:06:09 UTC
Web applications (e.g. Webmin) don't usually require any compilation but can contain a lot of files. The packaging code in package.bbclass does a lot of walking of file trees and processing of files, some of which is extraneous for non-binaries (e.g. debug splitting / stripping). There must be some optimisations we can make to speed this up - at the moment building webmin takes 21 minutes on my system and most of that is spent within the do_package task.

Most of these types of recipes should inherit allarch.bbclass, so where we can't do any generic optimisations, some other extraneous processing may be able to be disabled from there - I have a patch ready to do this for the shared library dependencies at least.
Comment 1 Paul Eggleton 2011-11-16 10:01:45 UTC
I made some improvements to allarch.bbclass which have been merged in commit b96390473d786924d6081319db8cbf31dd3e7d9a; this brings the time to build webmin down to about 12 minutes compared to 21 minutes previously. I still think we can do better however so this bug should remain open.
Comment 2 Richard Purdie 2013-02-14 11:33:02 UTC
How is this after the recent generic do_package improvements? We should be better again...
Comment 3 Richard Purdie 2013-02-14 11:34:23 UTC
Specifically, I think we really need to turn this into specific action items to target. Reassigning to Paul to either close afer checking the packaging is faster after recent improvements, or to point at specific areas to improve in do_package.
Comment 4 Paul Eggleton 2013-03-07 15:08:10 UTC
So for the webmin recipe with current master there appears to be a significant improvement - even on my laptop it's around 4 minutes to build it now (although it was quite a while since my previous tests so add salt to taste).

With all of the improvements in allarch.bbclass since this bug was filed, the webmin recipe only has one workaround left which is to override package_do_pkgconfig to do nothing; however when I removed this it didn't increase the build time measurably, so I suspect that could be dropped.

Thus I'm going to mark this one as resolved; webmin is a fairly extreme case in terms of the number of files it contains and I think it's building in a reasonable time now.