On the dunfell branch I've recently updated meta-sancloud so that LAYERDEPENDS and LAYERRECOMMENDS are set as follows:

LAYERDEPENDS_meta-sancloud = " \
    core \
    meta-ti \
    "
LAYERRECOMMENDS_meta-sancloud = " \
    rtlwifi \
    "

The `rtlwifi` collection is provided by https://github.com/EmbeddedAndroid/meta-rtlwifi which only contains a master branch, it has no dunfell branch. The master branch of meta-rtlwifi indicates compatibility with dunfell so our usual process is to use the dunfell branch of meta-sancloud, oe-core, meta-ti and other layers plus the master branch of meta-rtlwifi. However, the layer index only looks at the branch names and so does not list meta-rtlwifi on the dunfell branch.

If meta-sancloud had a dependency on meta-rtlwifi, it would be dropped from the layerindex for the dunfell branch as dependencies could not be satisfied. This was previously seen before I modified the LAYERDEPENDS and LAYERRECOMMENDS values above in our layer. However, with meta-rtlwifi listed as a recommendation the meta-sancloud layer is accepted and included in the layerindex for the dunfell branch.

This results in invalid dependency information being generated. I'll attach the full json files in follow up comments as I can only add one attachment at a time but here I will just include the relevant lines. In the layerBranches list for dunfell meta-sancloud is present:

{"id":1631501,"collection":"meta-sancloud","version":null,"vcs_subdir":"","vcs_last_fetch":"2020-06-23T11:06:23+0000","vcs_last_rev":"e3117953a0213c896cc9c8559dc91fffdff41baf","vcs_last_commit":"2020-06-23T08:06:22+0000","actual_branch":"","local_path":"","updated":"2020-06-23T11:06:23+0000","layer":469,"branch":18,"yp_compatible_version":null}

In the layerDependencies list for dunfell there are 3 entries which match the ID of the meta-sancloud layer:

{"id":700743,"required":true,"layerbranch":1631501,"dependency":1}
{"id":700744,"required":true,"layerbranch":1631501,"dependency":26}
{"id":700745,"required":false,"layerbranch":1631501,"dependency":325}

These correspond to the layers oe-core, meta-ti and meta-rtlwifi. The first two are present in the layerBranches list for dunfell however meta-rtlwifi (layer 325) is not. meta-rtlwifi is present in the layerBranches list for master but the client does not look there.

This data appears to be invalid as the "dependency" value for the layerDependencies entry with ID 700745 does not refer to a layer that is present in the corresponing layerBranches list.

Bitbake fails to parse this data and prints out a backtrace:

pokyuser@86145e976448:/w/poky/build$ bitbake-layers layerindex-fetch meta-sancloud -b dunfell
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=dunfell...
Traceback (most recent call last):
  File "/w/poky/bitbake/bin/bitbake-layers", line 95, in <module>
    ret = main()
  File "/w/poky/bitbake/bin/bitbake-layers", line 88, in main
    return args.func(args)
  File "/w/poky/bitbake/lib/bblayers/layerindex.py", line 119, in do_layerindex_fetch
    (dependencies, invalidnames) = lIndex.find_dependencies(names=args.layername, ignores=ignore_layers)
  File "/w/poky/bitbake/lib/layerindexlib/__init__.py", line 443, in find_dependencies
    (dependencies, invalid) = _resolve_dependencies(layerbranches, ignores, dependencies, invalid)
  File "/w/poky/bitbake/lib/layerindexlib/__init__.py", line 386, in _resolve_dependencies
    deplayerbranch = layerdependency.dependency_layerBranch
  File "/w/poky/bitbake/lib/layerindexlib/__init__.py", line 915, in __getattr__
    raise AttributeError('%s not in datastore' % name)
AttributeError: dependency_layerBranch not in datastore

I think the backtrace is also an issue - bitbake-layers should reject invalid data from the layerindex with a meaningful error message instead of a backtrace.

I've opened a bug against meta-rtlwifi to request that a dunfell branch is created: https://github.com/EmbeddedAndroid/meta-rtlwifi/issues/16

I'm also looking to work around this by temporarily dropping the LAYERRECOMMENDS setting from meta-sancloud and leaving users to add the meta-rtlwifi layer separately if they need wifi support. However that's not ideal.

For us the ideal fix would be to extend the layerindex to handle repositories which contain only the master branch but actually declare support for multiple Yocto Project release series. That would require parsing LAYERSERIES_COMPAT for the master branch of each layer, possibly with the addition of some way of indicating which branches to check when a layer is added to the index. That may not be the quickest thing to fix though, maybe it would be better to add data validation and handle layer recommendations (allowing them to fail with a warning) separately from layer dependencies.