Bug 15365

Summary: bitbake-layers layerindex-fetch (or layerindex-show-depends) backtrace when layerbranch of dependent layer does not exist
Product: [Build System, Metadata & Runtime] BitBake Reporter: Tim Orling <tim.orling>
Component: bitbakeAssignee: Tim Orling <tim.orling>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher, randy.macleod
Version: 5.0   
Target Milestone: 5.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Bug Depends on:    
Bug Blocks: 13954    

Description Tim Orling 2024-01-23 20:00:38 UTC
Related to issue originally reported in https://bugzilla.yoctoproject.org/show_bug.cgi?id=13954

On dunfell:

$ bitbake-layers layerindex-show-depends meta-intel
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=dunfell...
Traceback (most recent call last):
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/bin/bitbake-layers", line 95, in <module>
    ret = main()
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/bin/bitbake-layers", line 88, in main
    return args.func(args)
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/lib/bblayers/layerindex.py", line 209, in do_layerindex_show_depends
    self.do_layerindex_fetch(args)
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/lib/bblayers/layerindex.py", line 119, in do_layerindex_fetch
    (dependencies, invalidnames) = lIndex.find_dependencies(names=args.layername, ignores=ignore_layers)
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/lib/layerindexlib/__init__.py", line 442, in find_dependencies
    (dependencies, invalid) = _resolve_dependencies(layerbranches, ignores, dependencies, invalid)
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/lib/layerindexlib/__init__.py", line 385, in _resolve_dependencies
    deplayerbranch = layerdependency.dependency_layerBranch
  File "/srv/build/ttorling/workspace-dunfell/poky/bitbake/lib/layerindexlib/__init__.py", line 914, in __getattr__
    raise AttributeError('%s not in datastore' % name)
AttributeError: dependency_layerBranch not in datastore

This is happening in this case because meta-intel-qat LayerBranch does not exist for 'dunfell' Branch.
Comment 1 Tim Orling 2024-01-23 20:03:13 UTC
Also happens on "master":

$ bitbake-layers layerindex-show-depends meta-intel
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=nanbield...
Traceback (most recent call last):
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/layerindexlib/__init__.py", line 389, in _resolve_dependencies
    deplayerbranch = layerdependency.dependency_layerBranch
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/layerindexlib/__init__.py", line 922, in __getattr__
    raise AttributeError('%s not in datastore' % name)
AttributeError: dependency_layerBranch not in datastore

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/bin/bitbake-layers", line 95, in <module>
    ret = main()
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/bin/bitbake-layers", line 88, in main
    return args.func(args)
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/bblayers/layerindex.py", line 243, in do_layerindex_show_depends
    self.do_layerindex_fetch(args)
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/bblayers/layerindex.py", line 146, in do_layerindex_fetch
    (dependencies, invalidnames) = lIndex.find_dependencies(names=args.layername, ignores=ignore_layers)
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/layerindexlib/__init__.py", line 450, in find_dependencies
    (dependencies, invalid) = _resolve_dependencies(layerbranches, ignores, dependencies, invalid)
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/layerindexlib/__init__.py", line 392, in _resolve_dependencies
    logger.warning('layerdependency.dependency_layerBranch %s' % layerdependency.dependency_layerBranch)
  File "/srv/build/ttorling/workspace-upgrades/poky/bitbake/lib/layerindexlib/__init__.py", line 922, in __getattr__
    raise AttributeError('%s not in datastore' % name)
AttributeError: dependency_layerBranch not in datastore

In this case it is because meta-dpdk 'nanbield' LayerBranch does not exist.

$ bitbake-layers layerindex-show-depends meta-dpdk
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=nanbield...
ERROR: Layer "meta-dpdk" not found in layer index

While this exposes gaps in Layerindex Metadata, it is still a bug in bitbake/lib/layerindexlib.
Comment 2 Tim Orling 2024-01-23 21:29:00 UTC
Beginnings of a better error message:

https://git.yoctoproject.org/poky-contrib/log/?h=timo/layerindex-show-depends-debug

$ bitbake-layers layerindex-show-depends meta-acrn
NOTE: Starting bitbake server...
Loading https://layers.openembedded.org/layerindex/api/;branch=nanbield...
WARNING: layerdependency.dependency_layerBranch does not exist for meta-dpdk:nanbield
WARNING: Consider reaching out the to the layer maintainers or the layerindex admins
Layer                                              Git repository (branch)                                 Subdirectory
=============================================================================================================================
local:HEAD:openembedded-core                       https://git.yoctoproject.org/poky-contrib.git (timo/layerindex-show-depends-debug) meta
  required by: meta-acrn meta-intel meta-python meta-oe
  recommended by: meta-intel
layers.openembedded.org:nanbield:meta-intel        git://git.yoctoproject.org/meta-intel (nanbield)
  required by: meta-acrn
local:HEAD:meta-oe                                 https://git.openembedded.org/meta-openembedded-contrib (master) meta-oe
  required by: meta-python
local:HEAD:meta-python                             https://git.openembedded.org/meta-openembedded-contrib (master) meta-python
  required by: meta-acrn
layers.openembedded.org:nanbield:meta-acrn         https://github.com/intel/meta-acrn (nanbield)
Repository git://git.yoctoproject.org/meta-intel needs to be fetched
Repository https://github.com/intel/meta-acrn needs to be fetched
Comment 3 Tim Orling 2024-02-22 16:23:30 UTC
This was not ready in time for 5.0M3 feature freeze, so move it to 5.1