Bug 8802

Summary: meta-fsl-* fails in universe fetch
Product: [Build System, Metadata & Runtime] BSPs Reporter: Ross Burton <ross.burton>
Component: bsps-meta-fsl-armAssignee: Richard Purdie <richard.purdie>
Status: VERIFIED FIXED QA Contact: Alexandru Georgescu <alexandru.c.georgescu>
Severity: major    
Priority: High CC: alexandru.c.georgescu, joshuagloe, richard.purdie, zhenhua.luo
Version: 2.0   
Target Milestone: 2.1 M3   
Hardware: All   
OS: Multiple   
Whiteboard: autobuilder
OS type for building Yocto: --- Type of Regression: Regression (Found by QA test)
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2015-12-14 10:38:03 UTC
The autobuilder does a universe fetch with the meta-fsl-* layers enabled, and this causes errors when MACHINE=qemux86.

[recipe: linux-imx]
ERROR: Function failed: Fetcher failure for URL: 'git://git.freescale.com/imx/linux-2.6-imx.git;branch='. Please set a valid SRCREV for url ['SRCREV_default_pn-linux-imx', 'SRCREV_default', 'SRCREV_pn-linux-imx', 'SRCREV'] (possible key names are git://git.freescale.com/imx/linux-2.6-imx.git;branch=, or use a ;rev=X URL parameter)

[recipe: mfgtool-linux-imx]
ERROR: Function failed: Fetcher failure for URL: 'git://git.freescale.com/imx/linux-2.6-imx.git;branch='. Please set a valid SRCREV for url ['SRCREV_default_pn-mfgtool-linux-imx', 'SRCREV_default', 'SRCREV_pn-mfgtool-linux-imx', 'SRCREV'] (possible key names are git://git.freescale.com/imx/linux-2.6-imx.git;branch=, or use a ;rev=X URL parameter)

[recipe: barebox]
ERROR: Function failed: Fetcher failure for URL: 'file://defconfig'. Unable to fetch URL from any source.

[recipe: udev-rules-qoriq]
ERROR: Function failed: Fetcher failure for URL: 'file://71-fsl-dpaa-persistent-networking.rules'. Unable to fetch URL from any source.

I imagine you have machine-specific files but not machine-specific additions to SRC_URI.  Please don't do this, BSP layers shouldn't cause any changes if the MACHINE isn't selected.
Comment 1 Ross Burton 2015-12-16 09:04:53 UTC
Still breaking the autobuilder.

udev-rules-qoriq sets:

SRC_URI = "\
    file://71-fsl-dpaa-persistent-networking.rules \
    file://72-fsl-dpaa-persistent-networking.rules \
"

But those files are in a machine-specific location so fetch fails for all machines apart from qoriq-ppc.  The SRC_URI should be set using an override.


barebox appears to be broken with a deliberately excluding COMPATIBLE_MACHINE, but fetchall doesn't care about this.  Please remove or fix recipes that are broken.
Comment 2 Zhenhua Luo 2015-12-21 08:31:43 UTC
The patch is submitted. 

http://patches.openembedded.org/patch/110431/
Comment 4 Ross Burton 2016-01-05 13:04:13 UTC
Still broken on the AB: http://errors.yoctoproject.org/Errors/Details/26021/
Comment 5 Zhenhua Luo 2016-01-06 13:44:56 UTC
The patch is submitted. 

http://patches.openembedded.org/patch/111471/
Comment 7 Ross Burton 2016-01-10 09:12:03 UTC
Last nights run produced these errors:

barebox:
ERROR: Function failed: Fetcher failure for URL: 'file://defconfig'. Unable to fetch URL from any source.
http://errors.yoctoproject.org/Errors/Details/26733/

linux-imx:
ERROR: Function failed: Fetcher failure for URL: 'git://git.freescale.com/imx/linux-2.6-imx.git;branch='. Please set a valid SRCREV for url ['SRCREV_default_pn-linux-imx', 'SRCREV_default', 'SRCREV_pn-linux-imx', 'SRCREV'] (possible key names are git://git.freescale.com/imx/linux-2.6-imx.git;branch=, or use a ;rev=X URL parameter)
http://errors.yoctoproject.org/Errors/Details/26732/

mfgtool-linux-imx:
ERROR: Function failed: Fetcher failure for URL: 'git://git.freescale.com/imx/linux-2.6-imx.git;branch='. Please set a valid SRCREV for url ['SRCREV_default_pn-mfgtool-linux-imx', 'SRCREV_default', 'SRCREV_pn-mfgtool-linux-imx', 'SRCREV'] (possible key names are git://git.freescale.com/imx/linux-2.6-imx.git;branch=, or use a ;rev=X URL parameter)
http://errors.yoctoproject.org/Errors/Details/26731/

This was with meta-fsl-arm 1dd6a5b87947e17e986e4326bee277f7bfef0be5 and meta-fsl-ppc cb8d8ffd03645f58b530cc62e9a943a7bf93476c.
Comment 8 Zhenhua Luo 2016-01-11 05:13:26 UTC
Re-assign to the maintainer of meta-fsl-arm layer.
Comment 9 Ross Burton 2016-01-20 21:17:26 UTC
Update as of 20th January:

/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/meta-fsl-arm/recipes-kernel/linux/linux-imx-mfgtool_3.14.38.bb, do_fetch
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/meta-fsl-arm/recipes-kernel/linux/linux-imx_3.14.38.bb, do_fetch
/home/pokybuild/yocto-autobuilder/yocto-worker/nightly/build/meta-fsl-arm/recipes-bsp/barebox/barebox_2015.10.0.bb, do_fetch

Still breaking the autobuilder...
Comment 10 Otavio Salvador 2016-01-25 12:52:24 UTC
The way the universal fetcher works, it discards the compatible machine while does not iterates on the provided overrides, so failing to fetch the source.

The linux-imx, for example, provide SoC family specific branches/revisions so it does not provide a common one (this will change soon but it raises a universal fetch design flaw) triggering the error.
Comment 11 Richard Purdie 2016-01-27 12:11:56 UTC
Firstly, if I have meta-fal-arm in my local layers list and I reparse (rm tmp/cache/* -rf), I see:

WARNING: Unable to get checksum for barebox SRC_URI entry defconfig: file could not be found                                                               | ETA:  --:--:--
WARNING: Unable to get checksum for linux-imx SRC_URI entry defconfig: file could not be found
WARNING: Unable to get checksum for mfgtool-linux-imx SRC_URI entry defconfig: file could not be found

I have MACHINE set to qemux86 in this case but I should not need WARNINGS for this. I should not have to remove the layer to change machine. I have just realised I do also have SOURCE_MIRROR_FETCH = "1" set locally.

If we decide SOURCE_MIRROR_FETCH is broken and can't be supproted, we should remove it.

Secondly, this issue is breaking our autobuilder in the universe fetch. We can't simply "WONTFIX" this. If we can't fix meta-fsl-arm, we can look at other options such as removing the layer from the public autobuilder.
Comment 12 Richard Purdie 2016-01-27 12:25:22 UTC
I've proposed a compromise on the mailing list which should address the issues.
Comment 13 Joshua Lock 2016-02-01 09:38:16 UTC
(In reply to comment #12)
> I've proposed a compromise on the mailing list which should address the
> issues.

For the benefit of those following this issue the proposed solution is:

http://lists.openembedded.org/pipermail/bitbake-devel/2016-January/006978.html
Comment 15 Alexandru Georgescu 2016-02-26 13:04:50 UTC
Verified with master:c638c92d47768e64ceca2c6047bf8313acce77a5