Bug 4146 - Allow a layer to include another layer
Summary: Allow a layer to include another layer
Status: RESOLVED WORKSFORME
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: configuration (show other bugs)
Version: unspecified
Hardware: All Multiple
: Medium enhancement
Target Milestone: 1.9 M3
Assignee: Jason Wessel
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-03-28 22:09 UTC by Jason Wessel
Modified: 2015-10-01 15:03 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Wessel 2013-03-28 22:09:50 UTC
There is no documented and defined way to dynamically add to BBLAYERS.

It would be ideal if a layer could add additional sub layers automatically.  When making large features it is nice to be able to just include a single layer which then includes additional layers automatically.
Comment 1 Paul Eggleton 2013-05-24 08:46:39 UTC
Assuming this is a way of satisfying layer dependencies automatically, how would the system know where to find layers? If you mean by downloading them, how do we handle when the user already has the layer stored somewhere else locally?
Comment 2 Paul Eggleton 2014-02-27 16:33:55 UTC
We still need more definition around this; setting to NEEDINFO accordingly.
Comment 3 Jason Wessel 2014-04-10 21:33:46 UTC
Let us use an example, and I appologize this has to do more with a commercial product, but the idea is that I could put some kind of reference in the layer to automatically expand the bblayers list.

The example would be if I want to build a layer which has my own java application. It would be ideal if I could specify to automatically add meta-java, oe-core/meta, meta-networking, and meta-webserver.

The automated downloading of an entire layer with a fetch method is a bit of a separate problem. Perhaps it should have its own feature request.
Comment 4 Paul Eggleton 2014-04-11 09:16:54 UTC
I certainly understand the use case and the motivation; as above the part I'm less clear on is how it would be implemented - how does it find layers? Would the system have a pre-defined map somewhere of available layers so that each layer would list its dependencies by name i.e. "meta-java" and then the system would de-reference that name to /home/paul/oe/layers/meta-java/ using the map? Or would it just be a nominated directory/path where layers should be found? Or, would it rely on an online service like the layer index? How do you deal with two versions of the same layer that you have fetched locally, which might have the same name (e.g. forks)?
Comment 5 Jason Wessel 2014-04-11 14:24:39 UTC
(In reply to comment #4)
> I certainly understand the use case and the motivation; as above the part
> I'm less clear on is how it would be implemented - how does it find layers?
> Would the system have a pre-defined map somewhere of available layers so
> that each layer would list its dependencies by name i.e. "meta-java" and
> then the system would de-reference that name to
> /home/paul/oe/layers/meta-java/ using the map? Or would it just be a
> nominated directory/path where layers should be found? Or, would it rely on
> an online service like the layer index? 


The way we typically use bitbake and oe today is to create a layers directory.  The layers directory is flat and looks like what you would find in layers.openembedded.org.  We could always elect to add a new type of file which is a layer.hints or layer.requires file, or perhaps add the needed data to layer.conf in the custom layer.



How do you deal with two versions of
> the same layer that you have fetched locally, which might have the same name
> (e.g. forks)?

My opinion is that the fetch model is the right one to go with since it also has the concept of premirrors.  This would allow you to use git, a tarball or a fully exploded layer (which you could just symlink).

In the commercial products based on oe, most use some kind of workspace setup script which will clone, link or copy all the pieces you need into the workspace and then populate the bblayers.  The idea here is to simplify that process, as well as to allow layer designers to have multiple layers even in the layers.openembedded.org.  Conceptually it is similar to using git modules where the master git tree tracks all the other submodules.
Comment 6 Chen Qi 2015-04-03 03:30:39 UTC
Can `bitbake-layers layerindex-fetch meta-xxx' satisfy the requirements of this feature?

//Chen Qi
Comment 7 Chen Qi 2015-04-07 02:40:31 UTC
Below is an example.

chenqi@pek-hostel-deb01:~/poky/build-sysvinit [0] $ bitbake-layers layerindex-fetch meta-networking
Layer                Required by          Git repository                                          Subdirectory
===================================================================================================================
meta-networking      -                    git://git.openembedded.org/meta-openembedded            meta-networking
openembedded-core    meta-networking      git://git.openembedded.org/openembedded-core            meta
meta-python          meta-networking      git://git.openembedded.org/meta-openembedded            meta-python
meta-oe              meta-python          git://git.openembedded.org/meta-openembedded            meta-oe
Adding layer "meta-networking" to conf/bblayers.conf
Adding layer "meta-oe" to conf/bblayers.conf
Adding layer "meta-python" to conf/bblayers.conf
chenqi@pek-hostel-deb01:~/poky/build-sysvinit [0] $ cat conf/bblayers.conf 
# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /buildarea2/chenqi/poky/meta \
  /buildarea2/chenqi/poky/meta-yocto \
  /buildarea2/chenqi/poky/meta-yocto-bsp \
  /buildarea2/chenqi/poky/meta-openembedded/meta-networking \
  /buildarea2/chenqi/poky/meta-openembedded/meta-oe \
  /buildarea2/chenqi/poky/meta-openembedded/meta-python \
  "
BBLAYERS_NON_REMOVABLE ?= " \
  /buildarea2/chenqi/poky/meta \
  /buildarea2/chenqi/poky/meta-yocto \
  "
chenqi@pek-hostel-deb01:~/poky/build-sysvinit [0] $
Comment 8 Chen Qi 2015-08-04 01:53:48 UTC
As I stated above, I believe the current utility could satisfy the original requirements.

Change the status to "NEEDINFO" to get more feed back.
Comment 9 Paul Eggleton 2015-10-01 15:03:59 UTC
This has been sitting around for some time without much comment; we've implemented the capability to bring in layers from the OE layer index (or an internal layer index); if we need to extend that or do something different, let's open a bug with a specific request for the desired functionality.