see meta-virt

Is it about *just* providing own version of a recipe?
Or is it about setting preferred version to own version?

I understand what yocto-check-layer is trying to prevent, but it should only be checking and failing the second part, not the first one.

I have an idea

First add:

DEFAULT_PREFERENCE = "-1"

to each duplicate recipe

Now depending on how many recipes there are you can do one of two ways.

In the layer.conf add

require ${@bb.utils.contains('DISTRO_FEATURES', 'virtualization', 'meta-virt-default-versions.inc', '', d)

in the "inc" file include
+# Meta-virtuailization PREFERED_VERSION
+
+PREFERRED_VERSION_python-blinker = "1.3"

Or you can just use the "PREFERRED_VERSION_" in the above require

This scheme allowed the script to run clean. 

I sent patches to meta-virt.

The solution Armin added is the correct one, setting DEFAULT_PREFERENCE for new versions and then enabling them is the right way to handle this. Layers shouldn't change behavior just by adding them, they need to be enabled.

should this be an FAQ?

ah, no better if I add it to the wiki or doc about YP test script I need to create.

Taking issue so I remember to include.

is 

require ${@bb.utils.contains('DISTRO_FEATURES', 'virtualization', 'meta-virt-default-versions.inc', '', d)

in layer.conf 

supposed to ever expand to anything?

I did not manage to get this to work with warrior.

The expansion only seems to work from a .bb file and not from a .conf file.

Bruce also pointed me to this thread on the list[1]

Just added it here for reference.

[1] https://www.mail-archive.com/meta-virtualization@yoctoproject.org/msg04063.html

Michael, We should document this somewhere!

Bulk move of 5.1 M4 to 5.2 M2 as approved by AlexB.

This is now documented by following commit:
https://git.yoctoproject.org/yocto-docs/commit/?id=cc3fa1b0e51377f4e03eaa1ca60c2f2ee0cd917e

I took the example from meta-virt to write some documentation about providing confs from layer.conf file, while insisting on the fact that it is better practice when conditioned by a feature.
 
Closing this bug.

I'm going to reopen this, as the use case outlined in Comment #4 https://bugzilla.yoctoproject.org/show_bug.cgi?id=12688#c4 got recently broken.

Denys,

Where are you seeing, the problem?
Can you provide more detail or the steps to reproduce ?

Simplified reproduction steps:

$ cp -r openembedded-core/meta/recipes-bsp/formfactor <your-layer>/recipes-bsp/
$ mv <your-layer>/recipes-bsp/formfactor/formfactor_0.0.bb <your-layer>/recipes-bsp/formfactor/formfactor_1.0.bb
$ echo 'DEFAULT_PREFERENCE = "-1"' >> <your-layer>/recipes-bsp/formfactor/formfactor_1.0.bb
$ yocto-check-layer <your-layer>

Based on the Comment #4 and Richard's confirmation in Comment #5 this was previously sufficient to pass.

Now it fails the signature check:

AssertionError: Adding layer <your-layer> changed signatures.
19 signatures changed, initial differences (first hash before, second after):
   formfactor:do_create_recipe_spdx: 0e6a08af4be7be22d545ce5a60c12bdcdcec2963a02c28b05753fc2a71e63131 -> 621b7d1eac524fddf4741e1b9cac52168ed8e67e8f875fb8b40c853d0164a6e4
      bitbake-diffsigs --task formfactor do_create_recipe_spdx --signature 0e6a08af4be7be22d545ce5a60c12bdcdcec2963a02c28b05753fc2a71e63131 621b7d1eac524fddf4741e1b9cac52168ed8e67e8f875fb8b40c853d0164a6e4
      basehash changed from fe6b03ce7e7037b678b03b060e06bbd1b30c306a3aad86d2e6c0a5373fd407cc to 9219037157aea33fd41a490e1aa74b9ddb1040c573041ef78ff9a83753f8d918
      Variable FILE_LAYERNAME value changed from 'core' to '<your-layer>'
      Variable PV value changed from '0.0' to '1.0'

   formfactor:do_recipe_qa: ac11a1aa4e6514527d69e9700ea8d2a8ae0079218b3e3bc25a49c274f68318a8 -> f05a2943dfab19f3f5367ad48f3ac43909d130c10122796dd916599c8a5bdcc3
      bitbake-diffsigs --task formfactor do_recipe_qa --signature ac11a1aa4e6514527d69e9700ea8d2a8ae0079218b3e3bc25a49c274f68318a8 f05a2943dfab19f3f5367ad48f3ac43909d130c10122796dd916599c8a5bdcc3
      basehash changed from 68d08963da2451870d24c77018566c0ba5032e5ff332d3872ff71847d23744ed to 2d73e41565142099c1f125f29aae0f2a583f11a801d3cf6cef4b04ee7cabea1f
      Variable PV value changed from '0.0' to '1.0'

Hi Denys, could you retry with oe-core rolled back to:
- 13cd668c77d287bc8ae9397bb11ff53d37bae5b0, before do_create_recipe_spdx was added
- 9148a8b3ce6b2d6d192c56956a5fb0682c86b189, after the above, before check-layer fixes

That will help narrow down which change has caused the regression.

Paul,

Rolling back to before SPDX changes removed the signature change in do_create_recipe_spdx, obviously. But do_recipe_qa still remains and fails the test.

So, I've done more testing and tried to manually bisect this using my simplistic reproduction steps (OE-Core + my-layer), instead of doing it on a live setup with over a dozen of layers.

So, it would always fail the signature check for formfactor's PV with DEFAULT_PREFERENCE = "-1" going all the way back to yocto-2.5 (sumo), which was when this thread being discussed in 2018.

Armin's complete fix for meta-virtualization:
https://git.yoctoproject.org/meta-virtualization/commit/?h=sumo&id=032ef5310419563a01baad0b1b94e4587d12f777

Those were some older versions of python modules, compared to what was in meta-python - the duplication was between meta-python and meta-virtualization. Duplicated recipes set DEFAULT_PREFERENCE = "-1" and corresponding PREFERRED_VERSIONs are gated.

In my case the duplication is between openembedded-core and my-layer. I set DEFAULT_PREFERENCE = "-1", but don't do PREFERRED_VERSION for simplicity. I also tried decreasing PV in my-layer, instead of increasing it - doesn't help.

Interestingly, setting BBFILE_PRIORITY_my-layer lower than OE-Core, it passes! Then I compared BBFILE_PRIORITY values of meta-python and meta-virtualization in sumo - virtualization is higher. So. I'm not even sure how it used to work...

We've discussed this at the TSC meeting and I've done a few more experiments - layer priority (BBFILE_PRIORITY) here plays a major role and supersedes the DEFAULT_PREFERENCE (Richard confirmed this)

Here's what happens using my example of duplicated formfactor recipe in upstream oe-core and downstream my-layer:

* oe-core doesn't set explicit PREFERRED_VERSION for formfactor, as there's just a single version
* my-layer is expected to gate its PREFERRED_VERSION by some means, i.e. require explicit activation

In this case the signature check will pass if oe-core layer priority is higher than my-layer and will fail otherwise. Regardless of the DEFAULT_PREFERENCE in the recipe and the actual version of the recipe being higher or lower does not matter.

Now, if I explicitly set PREFERRED_VERSION in upstream oe-core to its version, while my-layer PREFERRED_VERSION is still gated, the signature check starts passing, as it seems to supersede layer priority.

I can also set default PREFERRED_VERSION in my-layer pointing to oe-core recipe version, while PREFERRED_VERSION pointing to my version is still gated - that also passes.

And lowering DEFAULT_PREFERENCE in the recipe is now useless - removed and confirmed.

So, the solution I came up with is for my-layer to have this line in the layer.conf:

PREFERRED_VERSION:formfactor = "<oe-core version>"

And in the gated .inc file that only gets included when explicitly enabled by end user with custom DISTRO_FEATURES, DISTRO or MACHINE that is specific to my-layer:

PREFERRED_VERSION:formfactor = "<my-layer version>"

The only drawback of this is it requires keeping track of oe-core version of the recipe and updating it in my-later layer.conf when it changes upstream...

I've also went back to re-test the original fix for meta-virtualization from Armin by setting all layers to use sumo branch.

Granted that back then meta-openembedded layers were not YP compatible, resulting in a lot of noise, I can still see the failure for python-blinker:

Variable PV value changed from '1.4' to '1.3'

As meta-python provides 1.4 version and no PREFERRED_VERSION, meta-virtualization provides 1.3 version and its PREFERRED_VERSION is gated by Armin's fix, and meta-virtualization has a higher layer priority vs. meta-python.

Nothing in comment 19 surprises me, I think it is all working as was designed/intended, even if it is much less than ideal.

As discussed, I don't think the behaviour is ideal, however changing it is a breaking architectural change and not easy to undertake.

I don't know what happened with the meta-virtualisation change, I think the details of that are lost in time and perhaps not so relevant now. I still maintain the behaviour you're seeing is correct, even if it isn't what we'd ideally like or is the most usable.

After further discussion with Richard (and the YP TSC earlier), this bug can now be closed.

DEFAULT_PREFERENCE behavior is suboptimal for this use case with its current architectural implementation.

The TSC has agreed to re-visit this in the future and consider changing this behavior, for which there's a new "feature request":

https://bugzilla.yoctoproject.org/show_bug.cgi?id=16257

For the reference, this also has been discussed all the way back in 2012 and was closed as NOTABUG with the current design/implementation:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2964