Bug 3702

Summary: FEATURE: in images allow easy selection of files within packages
Product: [Build System, Metadata & Runtime] BitBake Reporter: Frans Meulenbroeks <fransmeulenbroeks>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher, Qi.Chen, sgw
Version: unspecified   
Target Milestone: Future   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Frans Meulenbroeks 2013-01-11 08:50:53 UTC
This may not apply to bitbake on its own, but I could not find a better place. Feel free to move.

For resource constrained systems with limited flash it may be desired to prune the image as much as possible (so more fine-grain control of what goes into an image is desired).

It would help if there was an easy way to select only specific files from a package to be included in the rootfs.

E.g. instead of including mtd-utils one could say mtd-utils:usr/sbin/flash_erase and only get the flash_erase executable.

Of course this also can be achieved by doing things in a post process step, but having a mechanism as outlined above would make things simpler and also a lot easier to understand/explain

(and of course the user must take care to include all libs etc)

Ideally one should not only be able to specify a single file, but also allow directories (e.g. mtd-utils:usr/sbin) and regular expressions (e.g. mtd-utils:usr/sbin/*ubi*) 
I can also imagine that we allow partial matches, not sure about the syntax but e.g. mtd-utils:flash could match all files in a package that contain the string flash (so no need to give a path spec).
Comment 1 Chen Qi 2013-01-11 09:26:18 UTC

(In reply to comment #0)
> This may not apply to bitbake on its own, but I could not find a better
> place. Feel free to move.
> 
> For resource constrained systems with limited flash it may be desired to
> prune the image as much as possible (so more fine-grain control of what goes
> into an image is desired).
> 
> It would help if there was an easy way to select only specific files from a
> package to be included in the rootfs.
> 
> E.g. instead of including mtd-utils one could say
> mtd-utils:usr/sbin/flash_erase and only get the flash_erase executable.
> 

Can't we just use FILES_${PN} = "xxx xxx" ?

> Of course this also can be achieved by doing things in a post process step,
> but having a mechanism as outlined above would make things simpler and also
> a lot easier to understand/explain
> 
> (and of course the user must take care to include all libs etc)
> 
> Ideally one should not only be able to specify a single file, but also allow
> directories (e.g. mtd-utils:usr/sbin) and regular expressions (e.g.
> mtd-utils:usr/sbin/*ubi*) 
> I can also imagine that we allow partial matches, not sure about the syntax
> but e.g. mtd-utils:flash could match all files in a package that contain the
> string flash (so no need to give a path spec).
Comment 2 Frans Meulenbroeks 2013-01-11 14:11:46 UTC
Of course one can use FILES_${PN} = "xxx xxx" in a recipe.
However that is not what I want. 

When defining the image I would like to be able to select parts of packages (when defining the image, and without modifying the underlying package recipe.

Rationale is that if I can do it in the image I do not have to modify all kind of recipes (and maintain that), but instead it would all be centralized in the image recipe.

And of course we can split every package into very fine grained (sub) packages but that is a lot of work and not really convenient.

Thinking of it a little bit further part of what I want could probably also be realised using distro features. E.g. my distro does not support ubifs, and by introducing a distro feature for that I also could get rid of all ubi stuff.
Then again that could give rise to an unwieldy large set of distro features.

By providing a mechanism to specify sub-package parts in images we offer embedded developers (like myself ;-) ) with tools to makee things as compact as possible without introducing complexity for those who do not need it.
Comment 3 Chen Qi 2013-01-11 15:12:18 UTC
(In reply to comment #2)
> Of course one can use FILES_${PN} = "xxx xxx" in a recipe.
> However that is not what I want. 
> 
> When defining the image I would like to be able to select parts of packages
> (when defining the image, and without modifying the underlying package
> recipe.
> 
> Rationale is that if I can do it in the image I do not have to modify all
> kind of recipes (and maintain that), but instead it would all be centralized
> in the image recipe.
> 

Hi,
Sorry if I misunderstand what you mean, but what about a customized layer which contains all .bbappend files for those recipes that you want to customize the generated package? 

> And of course we can split every package into very fine grained (sub)
> packages but that is a lot of work and not really convenient.
> 
> Thinking of it a little bit further part of what I want could probably also
> be realised using distro features. E.g. my distro does not support ubifs,
> and by introducing a distro feature for that I also could get rid of all ubi
> stuff.
> Then again that could give rise to an unwieldy large set of distro features.
> 

The distro conf file could also reside in the customized layer.

> By providing a mechanism to specify sub-package parts in images we offer
> embedded developers (like myself ;-) ) with tools to makee things as compact
> as possible without introducing complexity for those who do not need it.
Comment 4 Frans Meulenbroeks 2013-01-14 09:43:26 UTC
local bbappends for recipes for which one wants a subset of definitely will work but is somewhat more work.

keeping a local distro.conf file is no problem but if a new distrofeature is added, relevant recipes need to be made aware of this feature.

Ofc this can be done locally with bbappend, but then probably the distrofeature is not really adding much).
Comment 5 Richard Purdie 2013-01-24 23:32:53 UTC
To be honest I don't see what the benefit of this would be, over and above adding an image post processing function which removes pieces of the filesystem. Adding such a function is trivial with today's codebase, e.g.:

zap_bindir () {
        rm ${IMAGE_ROOTFS}/bin/*
} 

ROOTFS_POSTPROCESS_COMMAND += 'zap_bindir'

I'm therefore marking this as WONTFIX since I think there are enough other ways this can be done.
Comment 6 Frans Meulenbroeks 2013-01-25 07:47:13 UTC
Agree. Actually I was unaware that I also could invoke a function; I always put all shell commands directly in the postprocess command.

Having a selector would be a little bit more handy if you only need one file or so from a dir with many files. Then again this is a fairly rare case and the postprocess would suffice.