Bug 1654

Mark, if I understand your description fully this tool already exists in the form of the "flatten" subcommand of bitbake-layers - could you take a look at that and let me know if it meets your requirements?

I wasn't aware of the bitbake-layers tool had a flatten option before.  After exploring it, let me clarify the request:

The current flatten takes the bblayers.conf and simply flattens that into a single layer.  That certainly should be one of the functions.  However, I would like the ability to specify an arbitrary list of layers (preferably on the command line) and have it only flatten the components within those layers.

i.e.

bitbake-layers flatten meta-intel meta-x32 meta-intelx32

(last argument being the destination)

Also I noticed that if the directory structure of the layers was not the same, then there was no combination of the .bb and .bbappend files.  Such as:

meta-bar/recipes-devtools/baroptimizer/baroptimizer_1.0.bb

meta-foo/baroptimizer/baroptimizer_1.0.bbappend

This currently results in both of them being available.  My assumption, based on the command flatten A B C DEST, is that we are merging B and C into A, placing the result into DEST.

Mark, I just attempted to reproduce the latter issue and it worked as expected - there were no bbappends in the resulting directory. Are you sure you had the layer configuration for the second layer set up such that the different path structure was recognised? Most of our layers have the appends part of BBFILES set to point to "${LAYERDIR}/recipes-*/*/*.bbappend" which would not pick up the example bbappend you gave.

I've sent an implementation of the feature request to be able to specify the layers to flatten, as well as a patch which warns the user if any recipes/appends in the flattened layer won't be referenced by the BBFILES paths in the flattened layer's layer.conf file.

Talking to Mark on IRC he can't shed any light on me not being able to reproduce the issue he described. Looking at the code it shouldn't have been possible to have bbappends in the flattened output layer because it explicitly skipped copying them. In any case I'll let him test the updated code with the aforementioned patches and will address any issues he finds.

Mark also thinks we should be able to automatically move recipes and their associated patches/config files to the appropriate place in the flattened layer in the case where some of the layers have a different structure. I can see this getting messy when the structure is wildly different (particularly where recipes share a directory in one layer and are split in another, which is perfectly legal as far as bitbake is concerned), but I will have a go at implementing this for M3, so this bug should be left open (or a new one split off.)

I think currently QA already has cases(bitbake-layers testing) to cover the feature.

M2 target for this has been completed. But still work left to automatically move recipes and their associated patches/config files to the appropriate place in the flattened layer in the case where some of the layers have a different structure, which is targeted for M3.

Moved this to M3

So I think at this point we've implemented all that we plan to on this. In any case I'm not totally convinced that these days munging layers together is something we want to encourage.

If there truly is demand for this to handle all structural differences then we can open up new bugs covering the issues, but for now I'm marking this one resolved.

I agree.. what is implemented meets all of my 'modern' needs.