Bug 10412

Summary: Specifying milestone with a DISTRO_VERSION in format of MM.mm.pp will cause build to fail on GetDistroVersion
Product: [Infrastructure] AutoBuilder Reporter: Tracy Graydon <tracy.graydon>
Component: autobuilderAssignee: Joshua Lock <joshuagloe>
Status: VERIFIED FIXED QA Contact: oscar lopez arandas <oscarx.lopez.arandas>
Severity: normal    
Priority: Medium+ CC: infras.ab.watcher, Infras.watcher, jose.perez.carranza, mhalstead, pidge, richard.purdie, ross.burton
Version: unspecified   
Target Milestone: 2.3 M2   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
error log for example of build failure none

Description Tracy Graydon 2016-10-11 21:25:49 UTC 
Created attachment 3468 [details]
error log for example of build failure

When generating a release candidate on the autobuilder, selecting "Milestone" with a DISTRO_VERSION in poky.conf in format of "MM.mm.pp" will cause GetDistroVersion to barf and build will fail. It will throw "exceptions.ValueError: invalid literal for float(): 2.2.0", as an example. 

i.e. https://autobuilder.yoctoproject.org/main/builders/nightly/builds/1046/steps/GetDistroVersion/logs/err.html (text and a screenshot attached)

Autobuilder should probably check for a mismatch between release type and distroversion and warn or otherwise act accordingly. i.e. In this specific case, catch milestone combined with point release distro version and do something smarter.
Comment 1 Tracy Graydon 2016-10-11 21:29:17 UTC 
Snippet from poky.conf as an example of what GetDistroVersion.py does not like:

DISTRO = "poky"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
DISTRO_VERSION = "2.2.0"                   <------  Makes GetDistroVersion barf.
Comment 2 Ross Burton 2016-10-11 21:55:42 UTC 
FWIW the standard library has a "version" objection in distutils  (IIRC) which can represent versions like 1.2.3-rc2~beta and do comparisons etc, instead of abusing a float.
Comment 3 Joshua Lock 2017-01-11 16:43:07 UTC 
(In reply to comment #0)
> Autobuilder should probably check for a mismatch between release type and
> distroversion and warn or otherwise act accordingly. i.e. In this specific
> case, catch milestone combined with point release distro version and do
> something smarter.

Any suggestion on how best to act accordingly? We can fail early here and cause the build to fail, that would certainly be smarter than throwing an exception.
Comment 4 Joshua Lock 2017-01-11 16:50:26 UTC 
(In reply to comment #2)
> FWIW the standard library has a "version" objection in distutils  (IIRC)
> which can represent versions like 1.2.3-rc2~beta and do comparisons etc,
> instead of abusing a float.

I don't know if that helps us here. The autobuilder is trying to be smart and automatically increase the version number for a milestone build and then check it matches the yocto number input into the build trigger form.

I think this mostly works because we:
a) we don't do milestone builds for point releases (is this true?)
b) we only bump version number in poky.conf when we're ready to release

so, for example, during milestone releases for pyro (2.3) the poky.conf DISTRO_VERSION is still 2.2 (morty's version number) and we add 0.1 to that and stringify it to get "2.3" for pyro milestone builds.

If we use, i.e. distutils.version.LooseVersion, there's no easy way to bump the version number for milestone releases.

I think what we should probably do is split the value of DISTRO_VERSION on .'s and fail early if there are more than 2 items in the list returned from the split call.
Comment 5 Joshua Lock 2017-01-11 16:53:36 UTC 
(In reply to comment #4)
> (In reply to comment #2)
> > FWIW the standard library has a "version" objection in distutils  (IIRC)
> > which can represent versions like 1.2.3-rc2~beta and do comparisons etc,
> > instead of abusing a float.
> 
> I don't know if that helps us here. The autobuilder is trying to be smart
> and automatically increase the version number for a milestone build and then
> check it matches the yocto number input into the build trigger form.
> 
> I think this mostly works because we:
> a) we don't do milestone builds for point releases (is this true?)
> b) we only bump version number in poky.conf when we're ready to release
> 
> so, for example, during milestone releases for pyro (2.3) the poky.conf
> DISTRO_VERSION is still 2.2 (morty's version number) and we add 0.1 to that
> and stringify it to get "2.3" for pyro milestone builds.
> 
> If we use, i.e. distutils.version.LooseVersion, there's no easy way to bump
> the version number for milestone releases.
> 
> I think what we should probably do is split the value of DISTRO_VERSION on
> .'s and fail early if there are more than 2 items in the list returned from
> the split call.

And/or we could just catch the ValueError Exception and fail appropriately then.
Comment 6 Joshua Lock 2017-01-17 12:32:11 UTC 
I've updated GetDistroVersion to fail as explained in previous comments and also write a log with the reason for failing.
Comment 8 oscar lopez arandas 2017-03-27 17:29:50 UTC 
ENVIRONMENT: genericx86-64 on NUC
STEPS TO REPRODUCE:
 1. Download the core-image-sato-sdk image from autobuilder.

2. Make checkout on the poky commit from the mail: 
     "9fe7a69535f9443175da8289633a761f58c372ff"

3. Source the environment, from poky directory

4. Edit local.conf. PATH: ~/poky/build/conf, at the en of the file adding: 

DISTRO = "poky"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
DISTRO_VERSION = "2.2.0"  
 
5. In ~poky/build directory, in command line, type the following commands:

$ bitbake rpm psplash run-postinsts
$ bitbake package-index

6. $ bitbake core-image-sato-sdk -c testimage