Bug 11149

Summary:	buildinfohelper.py Could not match layer dependency
Product:	[Build System, Metadata & Runtime] Toaster	Reporter:	Ferry <ferry.toth>
Component:	toaster	Assignee:	David Reyna <david.reyna>
Status:	RESOLVED FIXED	QA Contact:	Libertad <libertad.gonzalez.de.la.cruz>
Severity:	normal
Priority:	Medium	CC:	ferry.toth, jessica.zhang, stanciux.mihail
Version:	2.2.1
Target Milestone:	2.4 M1
Hardware:	x86
OS:	x86_64
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Ferry 2017-03-13 21:55:13 UTC

I start a new Morty project with Toaster (on master b1f09df0f664052e39d939b759a63b05d767b8ec  ), add meta-intel, select machine intel-corei7-64 then:

<SNIP>
Build Configuration:
BB_VERSION        = "1.32.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Ubuntu-16.10"
TARGET_SYS        = "x86_64-poky-linux"
MACHINE           = "intel-corei7-64"
DISTRO            = "poky"
DISTRO_VERSION    = "2.2.1"
TUNE_FEATURES     = "m64 corei7"
TARGET_FPU        = ""
meta              
meta-poky         
meta-yocto-bsp    = "master:6a1f33cc40bfac33cf030fe41e1a8efd1e5fad6f"
_git___git.yoctoproject.org_meta-intel_morty = "master:6add41510412ca196efb3e4f949d403a8b6f35d7"

NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/1.4/x86_64-nativesdk-libc.tar.bz2;sha256sum=101ff8f2580c193488db9e76f9646fb6ed38b65fb76f403acb0e2178ce7127ca
WARNING: Could not match layer dependency for path ^/home/ferry/tmp/edison-intel/my/edison-morty/poky/_toaster_clones/_git___git.yoctoproject.org_meta-intel_morty/common : [<Layer_Version: 1 openembedded-core (morty)>, <Layer_Version: 4 meta-poky (morty)>, <Layer_Version: 269 meta-intel (morty)>, <Layer_Version: 7 meta-yocto-bsp (morty)>]
ERROR: 
Traceback (most recent call last):
  File "/home/ferry/tmp/edison-intel/my/edison-morty/poky/bitbake/lib/bb/ui/toasterui.py", line 456, in main
    buildinfohelper.store_dependency_information(event)
  File "/home/ferry/tmp/edison-intel/my/edison-morty/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1399, in store_dependency_information
    assert layer_version_obj is not None
AssertionError

After this the build continues, with building the image at the end

Comment 1 Ferry 2017-03-14 16:24:39 UTC

Eh sorry, I mean withOUT building the image (not all tasks completed due to the error)

Comment 2 Ferry 2017-03-16 08:50:39 UTC

What can I do to help debug this?

Strangely, when I don't add meta-intel and select qemu (32 or 64bits) it build until completion.

Comment 3 David Reyna 2017-04-12 01:55:49 UTC

Ok, I have done some debugging.

The error happens when the layer_version path from the "DepTreeGenerated" event cannot be found in the database's 'orm_layer_version' table. 

Instrumenting 'store_dependency_information' I see:
	NOTE: FOO1:^/opt/dreyna/yp-2.2/poky/meta/,5
	NOTE: FOO2:OK
	NOTE: FOO1:^/opt/dreyna/yp-2.2/poky/meta-poky/,5
	NOTE: FOO2:OK
	NOTE: FOO1:^/opt/dreyna/yp-2.2/poky/meta-yocto-bsp/,5
	NOTE: FOO2:OK
	NOTE: FOO1:^/opt/dreyna/yp-2.2/poky/meta-intel/common/,5
	<assert fails>

When I look at 'orm_layer_version' table directly via the SQLite Database Browser app I see that all instances of the "meta-intel" layer have a local path of "/opt/dreyna/yp-2.2/poky/meta-intel/", which is indeed where it was installed.

The problem is that the "meta-intel" layer people have done a trick of having a layer within a layer, specifically "meta-intel/common", which is not found by the code in "buildinfohelper.py".

New code will need to added to resolve these cases.

Comment 4 David Reyna 2017-04-12 22:49:55 UTC

Here is the patch for Morty and YP-2.3:

diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index 3ddcb2ac6d..f79df9dbc2 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -995,6 +995,7 @@ class BuildInfoHelper(object):
 
         # we don't care if we match the trailing slashes
         p = re.compile(re.sub("/[^/]*?$","",pathRE))
+        path=re.sub(r'[$^]',r'',pathRE)
         # Heuristics: we always match recipe to the deepest layer path in the discovered layers
         for lvo in sorted(self.orm_wrapper.layer_version_objects, reverse=True, key=_sort_longest_path):
             if p.fullmatch(lvo.local_path):
@@ -1002,6 +1003,10 @@ class BuildInfoHelper(object):
             if lvo.layer.local_source_dir:
                 if p.fullmatch(lvo.layer.local_source_dir):
                     return lvo
+            if 0 == path.find(lvo.local_path):
+                # sub-layer path inside existing layer
+                return lvo
+