Bug 6127

Summary: Problem with globs in SRC_URI
Product: [Build System, Metadata & Runtime] BitBake Reporter: Jacob Kroon <jacob.kroon>
Component: bitbakeAssignee: Paul Eggleton <bluelightning>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Undecided CC: bluelightning, poky.bs.watcher, poky.watcher
Version: unspecified   
Target Milestone: ---   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Jacob Kroon 2014-04-08 02:24:44 UTC
I've run into a strange problem on master. I have a recipe:

test-bug.bb
===========
LICENSE = "CLOSED"

SRC_URI = "\
    file://a \
    file://dir/* \
"
============

The accompaning test-bug/ folder looks like:
test-bug/a
test-bug/b
test-bug/dir
test-bug/dir/c

Note the extra file "b", which is not specified in the SRC_URI.

Adding debug prints according to the patch at the bottom of this report gives me the following output:

WARNING: #1 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/.
WARNING: #2 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a
WARNING: #5 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a
WARNING: #2 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/.
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./a
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./b
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./dir/c

and running bitbake-diffsigs on the resulting test-bug:do_fetch signature gives me:

This task depends on the checksums of files: [
('a', 'b026324c6904b2a9cb4b88d6d61c81d1'),
('a', 'b026324c6904b2a9cb4b88d6d61c81d1'),
('b', '26ab0db90d72e28ad0ba1e22ee510510'),
('c', '6d7fce9fee471194aa8b5b6e47267f03')
]

'a' is duplicated, and 'b' is included although I assume it shouldn't be.

The debug prints I've added are:

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 8e5342f..15f4081 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -961,13 +961,16 @@ def get_file_checksums(filelist, pn):
         return checksum
 
     checksums = []
+    bb.warn("#1 " + filelist)
     for pth in filelist.split():
         checksum = None
+        bb.warn("#2 " + pth)
         if '*' in pth:
             # Handle globs
             for f in glob.glob(pth):
                 checksum = checksum_file(f)
                 if checksum:
+                    bb.warn("#3 " + f)
                     checksums.append((f, checksum))
             continue
         elif os.path.isdir(pth):
@@ -977,12 +980,14 @@ def get_file_checksums(filelist, pn):
                     fullpth = os.path.join(root, name)
                     checksum = checksum_file(fullpth)
                     if checksum:
+                        bb.warn("#4 " + fullpth)
                         checksums.append((fullpth, checksum))
             continue
         else:
             checksum = checksum_file(pth)
 
         if checksum:
+            bb.warn("#5 " + pth)
             checksums.append((pth, checksum))
 
     checksums.sort(key=operator.itemgetter(1))

I've tried tracing backwards trying to understand why the '*' gets replaced with a '.', but I get lost in the cache code.
Comment 1 Paul Eggleton 2014-04-08 14:14:33 UTC
I've sent a fix for this to the bitbake-devel list, however it seems like the checksum collection is not the only place where this problem is evident - see bug 6128 (and also bug 6129).