Bug 10412 - Specifying milestone with a DISTRO_VERSION in format of MM.mm.pp will cause build to fail on GetDistroVersion
Summary: Specifying milestone with a DISTRO_VERSION in format of MM.mm.pp will cause b...
Status: VERIFIED FIXED
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: unspecified
Hardware: All Multiple
: Medium+ normal
Target Milestone: 2.3 M2
Assignee: Joshua Lock
QA Contact: oscar lopez arandas
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-10-11 21:25 UTC by Tracy Graydon
Modified: 2017-03-27 17:29 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
error log for example of build failure (1.53 KB, text/plain)
2016-10-11 21:25 UTC, Tracy Graydon
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tracy Graydon 2016-10-11 21:25:49 UTC
Created attachment 3468 [details]
error log for example of build failure

When generating a release candidate on the autobuilder, selecting "Milestone" with a DISTRO_VERSION in poky.conf in format of "MM.mm.pp" will cause GetDistroVersion to barf and build will fail. It will throw "exceptions.ValueError: invalid literal for float(): 2.2.0", as an example. 

i.e. https://autobuilder.yoctoproject.org/main/builders/nightly/builds/1046/steps/GetDistroVersion/logs/err.html (text and a screenshot attached)

Autobuilder should probably check for a mismatch between release type and distroversion and warn or otherwise act accordingly. i.e. In this specific case, catch milestone combined with point release distro version and do something smarter.
Comment 1 Tracy Graydon 2016-10-11 21:29:17 UTC
Snippet from poky.conf as an example of what GetDistroVersion.py does not like:

DISTRO = "poky"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
DISTRO_VERSION = "2.2.0"                   <------  Makes GetDistroVersion barf.
Comment 2 Ross Burton 2016-10-11 21:55:42 UTC
FWIW the standard library has a "version" objection in distutils  (IIRC) which can represent versions like 1.2.3-rc2~beta and do comparisons etc, instead of abusing a float.
Comment 3 Joshua Lock 2017-01-11 16:43:07 UTC
(In reply to comment #0)
> Autobuilder should probably check for a mismatch between release type and
> distroversion and warn or otherwise act accordingly. i.e. In this specific
> case, catch milestone combined with point release distro version and do
> something smarter.

Any suggestion on how best to act accordingly? We can fail early here and cause the build to fail, that would certainly be smarter than throwing an exception.
Comment 4 Joshua Lock 2017-01-11 16:50:26 UTC
(In reply to comment #2)
> FWIW the standard library has a "version" objection in distutils  (IIRC)
> which can represent versions like 1.2.3-rc2~beta and do comparisons etc,
> instead of abusing a float.

I don't know if that helps us here. The autobuilder is trying to be smart and automatically increase the version number for a milestone build and then check it matches the yocto number input into the build trigger form.

I think this mostly works because we:
a) we don't do milestone builds for point releases (is this true?)
b) we only bump version number in poky.conf when we're ready to release

so, for example, during milestone releases for pyro (2.3) the poky.conf DISTRO_VERSION is still 2.2 (morty's version number) and we add 0.1 to that and stringify it to get "2.3" for pyro milestone builds.

If we use, i.e. distutils.version.LooseVersion, there's no easy way to bump the version number for milestone releases.

I think what we should probably do is split the value of DISTRO_VERSION on .'s and fail early if there are more than 2 items in the list returned from the split call.
Comment 5 Joshua Lock 2017-01-11 16:53:36 UTC
(In reply to comment #4)
> (In reply to comment #2)
> > FWIW the standard library has a "version" objection in distutils  (IIRC)
> > which can represent versions like 1.2.3-rc2~beta and do comparisons etc,
> > instead of abusing a float.
> 
> I don't know if that helps us here. The autobuilder is trying to be smart
> and automatically increase the version number for a milestone build and then
> check it matches the yocto number input into the build trigger form.
> 
> I think this mostly works because we:
> a) we don't do milestone builds for point releases (is this true?)
> b) we only bump version number in poky.conf when we're ready to release
> 
> so, for example, during milestone releases for pyro (2.3) the poky.conf
> DISTRO_VERSION is still 2.2 (morty's version number) and we add 0.1 to that
> and stringify it to get "2.3" for pyro milestone builds.
> 
> If we use, i.e. distutils.version.LooseVersion, there's no easy way to bump
> the version number for milestone releases.
> 
> I think what we should probably do is split the value of DISTRO_VERSION on
> .'s and fail early if there are more than 2 items in the list returned from
> the split call.

And/or we could just catch the ValueError Exception and fail appropriately then.
Comment 6 Joshua Lock 2017-01-17 12:32:11 UTC
I've updated GetDistroVersion to fail as explained in previous comments and also write a log with the reason for failing.
Comment 8 oscar lopez arandas 2017-03-27 17:29:50 UTC
ENVIRONMENT: genericx86-64 on NUC
STEPS TO REPRODUCE:
 1. Download the core-image-sato-sdk image from autobuilder.

2. Make checkout on the poky commit from the mail: 
     "9fe7a69535f9443175da8289633a761f58c372ff"

3. Source the environment, from poky directory

4. Edit local.conf. PATH: ~/poky/build/conf, at the en of the file adding: 

DISTRO = "poky"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
DISTRO_VERSION = "2.2.0"  
 
5. In ~poky/build directory, in command line, type the following commands:

$ bitbake rpm psplash run-postinsts
$ bitbake package-index

6. $ bitbake core-image-sato-sdk -c testimage