Bug 3221 - Save more space in WORKDIR with rm_work
Summary: Save more space in WORKDIR with rm_work
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 1.4
Hardware: x86 Multiple
: Low enhancement
Target Milestone: 1.4
Assignee: Saul Wold
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2012-10-03 16:33 UTC by Martin Jansa
Modified: 2013-05-08 16:22 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Jansa 2012-10-03 16:33:53 UTC
Current rm_work.bbclass implementation keeps some directories
'temp' 'package' 'packages-split'
in workdir even when they can be recreated from sstate package if we find out where to execute _setscene tasks correctly.

Another way (I'm using right now) is to remove ${S} between do_install and do_package, this way we don't get source files packaged in PN-dbg, but saves a lot of space before disk space hungry do_package is executed.

Also because rm_work task was usually executed as almost last one, bitbake scheduler was executing it near the end of runqueue, with rm_sourcedir task before do_package we force ${S} removal much sooner.
Comment 1 Richard Purdie 2012-10-19 14:44:18 UTC
'temp' 'package' 'packages-split' need to stick around. We've dicussed this before and I am not supporting any workflow which involves removing them as it complicates core code without good reason. There are better ways to save space.

For disabling the large -dbg packages, we should make including the source code a configuration option if it isn't already. 

I'd also suggest we hardlink the source files to save space, this is a cheap and easy way to improve things that benefits everyone. Hardllinks are going to be a way of gaining a lot of space and hance performace back going forward.

So I suggest this enhacement be about using hardlinks for the -dbg source code and ensuring the source inclusion in -dbg packages is configurable *and* documented in the manual.

Creation of some new rm_work equivalent which removes ${S} before do_package is a separate issue. I don't really like the idea at all, I think there are better things to focus on than trying to fix random deletion of parts of the build tree.
Comment 2 Martin Jansa 2012-10-19 14:47:43 UTC
(In reply to comment #1)
> 'temp' 'package' 'packages-split' need to stick around. We've dicussed this
> before and I am not supporting any workflow which involves removing them as
> it complicates core code without good reason. There are better ways to save
> space.

IIRC you said that it's hard to know when exactly call those setscene tasks, not that it's impossible.
Comment 3 Richard Purdie 2012-10-19 15:20:31 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > 'temp' 'package' 'packages-split' need to stick around. We've dicussed this
> > before and I am not supporting any workflow which involves removing them as
> > it complicates core code without good reason. There are better ways to save
> > space.
> 
> IIRC you said that it's hard to know when exactly call those setscene tasks,
> not that it's impossible.

Nothing is impossible. It doesn't make it a good idea or means its a sane design.
Comment 4 Richard Purdie 2013-01-24 23:26:50 UTC
The do_package do_packagedata split that is proposed offers some potential to rm_work to allow removal of the package and package-split directories in rm_work. See the commit message for that change for more hints.
Comment 6 Stoicescu Cornel 2013-05-08 16:22:53 UTC
as advertised, the output of do_package is removed from the work directory. The "temp" directory is  saved as it only contains logs.

For a core-image-minimal it saved almost 1 GB. Verified.