KERNEL_FEATURES is documented here:

https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-KERNEL_FEATURES

The .inc file needs some tidying up. I see a few different mechanisms mixing in there, += and _append, and even += AND _append. We can just use _append and it should be ok.

The elements in that file have migrated from different places in the layer and hence had different formatting standards/techniques and even were done certain ways for evaluation time issues.

but the difference between KERNEL_FEATURES append and SRC_URI is on purpose.

KERNEL_FEATURES come from the main kernel-cache, and are sanity checked for failure. They are also, by design, the last element applied to the merging process, since they signify a strict dependency.

SRC_URI fragments are merged and applied using the normal fragment rules.

Everything is working in all the meta-virt sanity tests that I have, what is failing in your broken case ?

I just needed to make some adjustments to be able to work with dunfell compared to zeus.

e.g. since I use my own custom-yocto-kernel recipe I don't have this:

KERNEL_FEATURES_append = " cfg/virtio.scc"

but I have that

SRC_URI += "file://features/virtio/virtio.scc"

This is at the moment the only thing which troubles me a bit, since I needed to patch meta-virtualization, which I usually try to avoid.

Here[1] is some part of the BSP and I guess you are after this[2]

[1] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/tree/dunfell

[2] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/tree/dunfell/recipes-kernel/linux

For a "virtualization" kernel (which is able to run docker) I have this kernel type[3]

It pull in my standard kernel config[4] and the virtualization config[5] on top.

Now it also adds some extra thingy ;)[6]

[3] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-kernel/linux/config/multi-v7-ml-base/ktypes/virt/virt.scc

[4] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-kernel/linux/config/multi-v7-ml-base/features-collection/std-collection.scc

[5] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-kernel/linux/config/multi-v7-ml-base/features-collection/virt-addon-collection.scc

[6] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-kernel/linux/linux-yocto-custom-common_5.4.inc

(In reply to comment #3)
> I just needed to make some adjustments to be able to work with dunfell
> compared to zeus.
> 
> e.g. since I use my own custom-yocto-kernel recipe I don't have this:
> 
> KERNEL_FEATURES_append = " cfg/virtio.scc"
> 
> but I have that
> 
> SRC_URI += "file://features/virtio/virtio.scc"

I'll have a look at your links in a bit more detail shortly, but I do have a question about this.

With the current .inc, are you seeing your virtio.scc not applied at all ? or some other side effect ?

> 
> This is at the moment the only thing which troubles me a bit, since I needed
> to patch meta-virtualization, which I usually try to avoid.
> 

We can come up with a dunfell solution that doesn't require you to patch the layer. If the .inc's aren't flexible enough, we'll fix them.

> Here[1] is some part of the BSP and I guess you are after this[2]
> 
> [1] https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/tree/dunfell
> 
> [2]
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/tree/dunfell/recipes-
> kernel/linux
> 
> For a "virtualization" kernel (which is able to run docker) I have this
> kernel type[3]
> 
> It pull in my standard kernel config[4] and the virtualization config[5] on
> top.
> 
> Now it also adds some extra thingy ;)[6]

If possible, can you share a local.conf and bblayers.conf to setup your build ? I can spin something up here and see for myself.

Bruce

> 
> [3]
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-
> kernel/linux/config/multi-v7-ml-base/ktypes/virt/virt.scc
> 
> [4]
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-
> kernel/linux/config/multi-v7-ml-base/features-collection/std-collection.scc
> 
> [5]
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-
> kernel/linux/config/multi-v7-ml-base/features-collection/virt-addon-
> collection.scc
> 
> [6]
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp/-/blob/dunfell/recipes-
> kernel/linux/linux-yocto-custom-common_5.4.inc

if you can give me ssh access somewhere I can set it up for you, or I can set up something and give you ssh access, otherwise you need these layers:

git://github.com/RobertBerger/poky branch: 2020-05-26-dunfell-3.1+

https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp.git branch: dunfell

https://gitlab.com/meta-layers/meta-u-boot-wic-bsp.git branch: dunfell

https://gitlab.com/meta-layers/meta-resy.git branch: dunfell

git://github.com/RobertBerger/meta-openembedded branch: 2020-04-30-dunfell-3.1

git://github.com/RobertBerger/meta-virtualization branch: 2020-04-30-dunfell-3.1

git://github.com/RobertBerger/meta-wifi-credentials branch: dunfell

--------

I have them all under /workdir/sources

--------

In one of the layers is the templateconf I use:
meta-u-boot-wic-bsp/template-imx6q-phytec-mira-rdk-nand-virt

bblayer.conf.sample, local.conf.sample, conf-notes.txt

--------

In another layer is the siteconf I use:

meta-resy/template-common/site.conf.sample

--------

to see the problem you can bitbake:

    core-image-minimal
    core-image-minimal-virt-docker-ce

--------

In case I forgot something please let me know.

(In reply to comment #5)
> if you can give me ssh access somewhere I can set it up for you, or I can
> set up something and give you ssh access, otherwise you need these layers:
> 
> git://github.com/RobertBerger/poky branch: 2020-05-26-dunfell-3.1+
> 
> https://gitlab.com/meta-layers/meta-multi-v7-ml-bsp.git branch: dunfell
> 
> https://gitlab.com/meta-layers/meta-u-boot-wic-bsp.git branch: dunfell
> 
> https://gitlab.com/meta-layers/meta-resy.git branch: dunfell
> 
> git://github.com/RobertBerger/meta-openembedded branch:
> 2020-04-30-dunfell-3.1
> 
> git://github.com/RobertBerger/meta-virtualization branch:
> 2020-04-30-dunfell-3.1
> 
> git://github.com/RobertBerger/meta-wifi-credentials branch: dunfell
> 
> --------
> 
> I have them all under /workdir/sources
> 
> --------
> 
> In one of the layers is the templateconf I use:
> meta-u-boot-wic-bsp/template-imx6q-phytec-mira-rdk-nand-virt
> 
> bblayer.conf.sample, local.conf.sample, conf-notes.txt
> 
> --------
> 
> In another layer is the siteconf I use:
> 
> meta-resy/template-common/site.conf.sample
> 
> --------
> 
> to see the problem you can bitbake:
> 
>     core-image-minimal
>     core-image-minimal-virt-docker-ce
> 
> --------
> 
> In case I forgot something please let me know.

That looks like it should be enough. I'll try that build locally, and if not, I may take you up on the ssh access offer.

I've cloned the layers and will have a look on Friday.

Since we are talking about a kernel configuration issue, I'll streamline the build and just look at virtual/kernel.

And so I'm clear, it is the lack of virtio configuration fragments being applied to the kernel that is the core of the issue. Correct ?

In this case, if you revert my commit f87eb93151e22cb80bcf56389bb439789f9191f2 the problem is, that cfg/virtio.scc is not being found and the build breaks.

(In reply to comment #8)
> In this case, if you revert my commit
> f87eb93151e22cb80bcf56389bb439789f9191f2 the problem is, that cfg/virtio.scc
> is not being found and the build breaks.

Aha. Yes, that's what I thought and was surprised when my build worked overnight. I'll do the revert and start looking.

After having poked at the build, the issue is clear. The KERNEL_FEATURES variable isn't just a plain .scc file, it contains a subdir, since it has to target some meta-data to construct a not too broad a search (otherwise, you'll start picking up filesystem order configurations, and that's bad).

So yes, it targets the layout of the kernel-cache meta data, since that's what we ensure through validation.

There are two ways to fix it in your bbappend.

a) clone the kernel-cache as part of the SRC_URI and the meta-data will be found a the right place, and all is well.

SRCREV_meta = "aafb8f095e97013d6e55b09ed150369cbe0c6476"
            file://multi-v7-ml-user-patches.scc \
            "
SRC_URI_append += " \
                file://${PATCHPATH}/patches;type=kmeta;destsuffix=patches \
                git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.4;destsuffix=kernel-meta \
                "
b) if you don't want the extra meta-data around, then you do need to follow that subdirectory structure, with the way it is currently expressed.

/home/bruce/poky-virt-issue/meta-multi-v7-ml-bsp/recipes-kernel/linux/config/multi-v7-ml-base
build2 [/home/bruc...v7-ml-base]> find cfg
cfg
cfg/virtio.cfg
cfg/virtio.scc

Both of those fix the configuration issue here.

We can also fix the problem in meta-virt, I can always create a secondary variable that is a weak assignment and use that variable in the KERNEL_FEATURES_append. That offers a way for you to easily override it in your layer.

The ability to have dangling kernel features was added to oe-core.

That addresses the core issue in this bug .. that as long as you know what you are doing, you can making missing kernel feature non-fatal.