Bug 81

Summary:	SRC_URI with http or ftp path having "/download" at end fails to retrieve source
Product:	[Build System, Metadata & Runtime] Meta-yocto	Reporter:	Levant Tinaz <levant.tinaz>
Component:	meta-yocto	Assignee:	Joshua Lock - Disabled <josh>
Status:	RESOLVED FIXED	QA Contact:
Severity:	major
Priority:	Medium	CC:	levant.tinaz, poky.bs.watcher, poky.watcher, sgw
Version:	0.9
Target Milestone:	0.9 M4
Hardware:	All
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	---

Description Levant Tinaz 2010-02-12 19:47:52 UTC

I had in a recipe (.bb) file the following:

SRC_URI = "http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-${PV}-src.tar.gz/download"

which did correctly retrieve the .gz file into sources/CUnit-2.1-0-src.tar.gz
but the fetch failed after trying all default mirrors. The md5 checksum file
sources/CUnit-2.1-0-src.tar.gz.md5 wasn't created either as the fetch failed.
There are download sites that have such URLs.

I fixed this bug with the addition of the lines marked with ">" below in the file bitbake/lib/bb/fetch/wget.py:

    def localpath(self, url, ud, d):
        url = bb.encodeurl([ud.type, ud.host, ud.path, ud.user, ud.pswd, {}])
>       # Handle the special case of download URLs like
>       # http://<path>/<file>/download
>       # Most download URLs are like http://path/file
>       if os.path.basename(url) == "download":
>          ud.basename = os.path.basename(os.path.dirname(ud.path))
        ud.localfile = data.expand(ud.basename, d)

        return os.path.join(data.getVar("DL_DIR", d, True), ud.localfile)

Our Poky/bitbake source appears to be version 1.8.13.

Levant Tinaz

Comment 1 Richard Purdie 2010-02-15 17:38:27 UTC

Hardcoding behaviour like that is not acceptable, its not generic. 

Have you tried a url like:

http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-${PV}-src.tar.gz/download;localpath=${DL_DIR}/CUnit-${PV}-src.tar.gz

?

Comment 2 Levant Tinaz 2010-02-16 13:39:06 UTC

Yes, I've tried "download=" as you suggest. That downloads to the correct location with the correct file name but then I get this error:

NOTE: package cunit-2.1-0: started
NOTE: package cunit-2.1-0-r0: task do_fetch: started
NOTE: fetch http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-2.1-0-src.tar.gz/download
NOTE: localpath=/home/levant/src/PreOlive/Poky/sources/cunit-2.1-0-src.tar.gz
NOTE: Task failed: Unknown fetch Error: 'FetchData' object has no attribute 'basename'
NOTE: package cunit-2.1-0-r0: task do_fetch: failed
ERROR: TaskFailed event exception, aborting
NOTE: package cunit-2.1-0: failed

which is due to ud.localpath being set explicitly in the SRC_URI so the localpath method is not called to set ud.localpath which is where basename is set to the download file name. Any SRC_URI that has "localpath=" option will generate this error due to that reason. I can fix that by adding the following: 

        def fetch_uri(uri, ud, d):
        .
        .
            if ud.localpath == "":
                fetchcmd = fetchcmd.replace("${FILE}", ud.basename)

Then my download works just fine but now the bug to be fixed is as noted above so this bug report Summary should be changed to "SRC_URI with localpath= doesn't work".

Levant Tinaz

Comment 3 Joshua Lock - Disabled 2010-09-06 06:33:20 UTC

I have fixed this issue in commit b78d5368ccd333587ec475be3f3fe058ec9045f2:

fetch: fix setting of localpath in SRC_URI parameters

When setting localpath in the SRC_URI parameters the basename should be set
explicitly as the localpath() method is not called.