Bug 8757 - Automate the process of checking for dead links in recipes
Summary: Automate the process of checking for dead links in recipes
Status: VERIFIED FIXED
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: 2.1
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 2.1 M1
Assignee: Beth Flanagan
QA Contact: Bogdan Alexandru Voiculescu
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-12-03 20:49 UTC by Juro Bystricky
Modified: 2016-03-21 14:18 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Juro Bystricky 2015-12-03 20:49:18 UTC
URL links in recipes can become obsoleted over the time. Recipes with dead links should be detected by a link-checker running on AB as part of QA.
Comment 1 Paul Eggleton 2015-12-07 19:37:12 UTC
Presumably you mean SRC_URI?
Comment 2 Juro Bystricky 2015-12-07 20:56:39 UTC
(In reply to comment #1)
> Presumably you mean SRC_URI?

Yes, I guess I should have been more specific. SRC_URI may become obsoleted (containing a dead link), but that may be detected quite a bit later, typically when building images from scratch. You will most likely have no problems, as the SRC_URI content will be fetched from a mirror, but still this should be detected and recipes updated accordingly.
Comment 3 Paul Eggleton 2015-12-07 21:41:48 UTC
So in concrete terms, I think this would be a simple matter of running:

  bitbake -c checkuri world

(On a regular basis).
Comment 4 Juro Bystricky 2015-12-07 21:47:29 UTC
(In reply to comment #1)
> Presumably you mean SRC_URI?

Yes, I guess I should have been more specific. SRC_URI may become obsoleted (containing a dead link), but that may be detected quite a bit later, typically when building images from scratch. You will most likely have no problems, as the SRC_URI content will be fetched from a mirror, but still this should be detected and recipes updated accordingly.
(In reply to comment #3)
> So in concrete terms, I think this would be a simple matter of running:
> 
>   bitbake -c checkuri world
> 
> (On a regular basis).

Yes, something like that. The whole idea is to have this check automated (and presumably followed up by SWAT)
Comment 6 Bogdan Alexandru Voiculescu 2016-03-14 10:32:08 UTC
Hi Beth,
I can not find this step on AB. 
https://autobuilder.yoctoproject.org/main/builders/nightly/builds/772

Please verify if patch is applied in production.
Comment 7 Ross Burton 2016-03-18 15:34:21 UTC
Its not a build set but a buildset: https://autobuilder.yoctoproject.org/main/builders/nightly-checkuri.
Comment 8 Ross Burton 2016-03-18 15:35:16 UTC
To save time, verifying myself.
Comment 9 Bogdan Alexandru Voiculescu 2016-03-21 06:35:38 UTC
Hi Ross,
The commit comment was:  
"
nightly.conf: add checkuri step
[YOCTO #8757]

Add a world -c checkuri step to nightly.

Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com>"

for this file" /buildset-config.controller/nightly.conf

So I looked for a build step in nightly.conf job. Also I looked for this commit to be executed in production; meaning this command "bitbake -c checkuri world"

The "bitbake -k universe -c checkuri" command is running for a while, but it is a different commit on a different buildset. I agree "bitbake -k universe -c checkuri" is covering "bitbake -c checkuri world", and it is a solution to verify URL links in recipes. 
Still it is a different solution and not from this enhacement.
Comment 10 Ross Burton 2016-03-21 11:01:47 UTC
You're right but that was just the first iteration that landed in git, it was revised in this commit to be what is currently on the AB:

commit 1c7b54394cf5cc94eca80b5092d0d8d7a4944102
Author: Elizabeth Flanagan <elizabeth.flanagan@intel.com>
Date:   Mon Jan 18 11:28:36 2016 +0000

    nightly-checkuri.conf: move checkuri to nowait

    checkuri takes about an hour to complete. moving it from
    nightly to it's own buildset.
Comment 11 Bogdan Alexandru Voiculescu 2016-03-21 14:18:04 UTC
Ross, thank you for clarification!