Bug 81 - SRC_URI with http or ftp path having "/download" at end fails to retrieve source
Summary: SRC_URI with http or ftp path having "/download" at end fails to retrieve source
Status: RESOLVED FIXED
Alias: None
Product: Meta-yocto
Classification: Build System, Metadata & Runtime
Component: meta-yocto (show other bugs)
Version: 0.9
Hardware: All Multiple
: Medium major
Target Milestone: 0.9 M4
Assignee: Joshua Lock - Disabled
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2010-02-12 19:47 UTC by Levant Tinaz
Modified: 2010-12-21 06:07 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Levant Tinaz 2010-02-12 19:47:52 UTC
I had in a recipe (.bb) file the following:

SRC_URI = "http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-${PV}-src.tar.gz/download"

which did correctly retrieve the .gz file into sources/CUnit-2.1-0-src.tar.gz
but the fetch failed after trying all default mirrors. The md5 checksum file
sources/CUnit-2.1-0-src.tar.gz.md5 wasn't created either as the fetch failed.
There are download sites that have such URLs.

I fixed this bug with the addition of the lines marked with ">" below in the file bitbake/lib/bb/fetch/wget.py:

    def localpath(self, url, ud, d):
        url = bb.encodeurl([ud.type, ud.host, ud.path, ud.user, ud.pswd, {}])
>       # Handle the special case of download URLs like
>       # http://<path>/<file>/download
>       # Most download URLs are like http://path/file
>       if os.path.basename(url) == "download":
>          ud.basename = os.path.basename(os.path.dirname(ud.path))
        ud.localfile = data.expand(ud.basename, d)

        return os.path.join(data.getVar("DL_DIR", d, True), ud.localfile)

Our Poky/bitbake source appears to be version 1.8.13.

Levant Tinaz
Comment 1 Richard Purdie 2010-02-15 17:38:27 UTC
Hardcoding behaviour like that is not acceptable, its not generic. 

Have you tried a url like:

http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-${PV}-src.tar.gz/download;localpath=${DL_DIR}/CUnit-${PV}-src.tar.gz

?

Comment 2 Levant Tinaz 2010-02-16 13:39:06 UTC
Yes, I've tried "download=" as you suggest. That downloads to the correct location with the correct file name but then I get this error:

NOTE: package cunit-2.1-0: started
NOTE: package cunit-2.1-0-r0: task do_fetch: started
NOTE: fetch http://sourceforge.net/projects/cunit/files/CUnit/2.1-0/CUnit-2.1-0-src.tar.gz/download
NOTE: localpath=/home/levant/src/PreOlive/Poky/sources/cunit-2.1-0-src.tar.gz
NOTE: Task failed: Unknown fetch Error: 'FetchData' object has no attribute 'basename'
NOTE: package cunit-2.1-0-r0: task do_fetch: failed
ERROR: TaskFailed event exception, aborting
NOTE: package cunit-2.1-0: failed

which is due to ud.localpath being set explicitly in the SRC_URI so the localpath method is not called to set ud.localpath which is where basename is set to the download file name. Any SRC_URI that has "localpath=" option will generate this error due to that reason. I can fix that by adding the following: 

        def fetch_uri(uri, ud, d):
        .
        .
            if ud.localpath == "":
                fetchcmd = fetchcmd.replace("${FILE}", ud.basename)

Then my download works just fine but now the bug to be fixed is as noted above so this bug report Summary should be changed to "SRC_URI with localpath= doesn't work".

Levant Tinaz

Comment 3 Joshua Lock - Disabled 2010-09-06 06:33:20 UTC
I have fixed this issue in commit b78d5368ccd333587ec475be3f3fe058ec9045f2:

fetch: fix setting of localpath in SRC_URI parameters

When setting localpath in the SRC_URI parameters the basename should be set
explicitly as the localpath() method is not called.