Bug 6127 - Problem with globs in SRC_URI
Summary: Problem with globs in SRC_URI
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Undecided normal
Target Milestone: ---
Assignee: Paul Eggleton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-04-08 02:24 UTC by Jacob Kroon
Modified: 2014-04-09 14:33 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Kroon 2014-04-08 02:24:44 UTC
I've run into a strange problem on master. I have a recipe:

test-bug.bb
===========
LICENSE = "CLOSED"

SRC_URI = "\
    file://a \
    file://dir/* \
"
============

The accompaning test-bug/ folder looks like:
test-bug/a
test-bug/b
test-bug/dir
test-bug/dir/c

Note the extra file "b", which is not specified in the SRC_URI.

Adding debug prints according to the patch at the bottom of this report gives me the following output:

WARNING: #1 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/.
WARNING: #2 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a
WARNING: #5 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/a
WARNING: #2 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/.
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./a
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./b
WARNING: #4 /home/jkroon/Projects/oe-devel/meta-monkey/recipes/test-bug/./dir/c

and running bitbake-diffsigs on the resulting test-bug:do_fetch signature gives me:

This task depends on the checksums of files: [
('a', 'b026324c6904b2a9cb4b88d6d61c81d1'),
('a', 'b026324c6904b2a9cb4b88d6d61c81d1'),
('b', '26ab0db90d72e28ad0ba1e22ee510510'),
('c', '6d7fce9fee471194aa8b5b6e47267f03')
]

'a' is duplicated, and 'b' is included although I assume it shouldn't be.

The debug prints I've added are:

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 8e5342f..15f4081 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -961,13 +961,16 @@ def get_file_checksums(filelist, pn):
         return checksum
 
     checksums = []
+    bb.warn("#1 " + filelist)
     for pth in filelist.split():
         checksum = None
+        bb.warn("#2 " + pth)
         if '*' in pth:
             # Handle globs
             for f in glob.glob(pth):
                 checksum = checksum_file(f)
                 if checksum:
+                    bb.warn("#3 " + f)
                     checksums.append((f, checksum))
             continue
         elif os.path.isdir(pth):
@@ -977,12 +980,14 @@ def get_file_checksums(filelist, pn):
                     fullpth = os.path.join(root, name)
                     checksum = checksum_file(fullpth)
                     if checksum:
+                        bb.warn("#4 " + fullpth)
                         checksums.append((fullpth, checksum))
             continue
         else:
             checksum = checksum_file(pth)
 
         if checksum:
+            bb.warn("#5 " + pth)
             checksums.append((pth, checksum))
 
     checksums.sort(key=operator.itemgetter(1))

I've tried tracing backwards trying to understand why the '*' gets replaced with a '.', but I get lost in the cache code.
Comment 1 Paul Eggleton 2014-04-08 14:14:33 UTC
I've sent a fix for this to the bitbake-devel list, however it seems like the checksum collection is not the only place where this problem is evident - see bug 6128 (and also bug 6129).