Bug 11561 - FAIL: test_recipetool_create_github (oeqa.selftest.recipetool.RecipetoolTests)
Summary: FAIL: test_recipetool_create_github (oeqa.selftest.recipetool.RecipetoolTests)
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.1.3
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.1.3
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-05-22 19:32 UTC by Juro Bystricky
Modified: 2017-06-12 22:46 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
console output (27.42 KB, text/x-log)
2017-05-22 19:32 UTC, Juro Bystricky
no flags Details
ubuntu_1604_krogoth (35.92 KB, application/octet-stream)
2017-06-08 21:24 UTC, oscar lopez arandas
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Juro Bystricky 2017-05-22 19:32:32 UTC
Created attachment 3771 [details]
console output

As found in 
https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/896/steps/Running%20oe-selftest/logs/stdio


FAIL: test_recipetool_create_github (oeqa.selftest.recipetool.RecipetoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 457, in test_recipetool_create_github
    self._test_recipe_contents(recipefile, checkvars, inherits)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/devtool.py", line 49, in _test_recipe_contents
    self.assertEqual(value, needvalue, 'values for %s do not match' % var)
AssertionError: Items in the first set but not the second:
'Unknown' : values for LICENSE do not match
Comment 1 Ross Burton 2017-05-24 13:53:05 UTC
Well this is exciting.

Current morty generates this:

LICENSE = "Apache-2.0 Unknown"
LIC_FILES_CHKSUM = "file://COPYING;md5=3b83ef96387f14655fc854ddc3c6bd57 \
                   
file://docs/theme/extra/templates/license.html;md5=8490f984473ca7fa3bdcfe2a919e6249 \
                    file://docs/markdown/legal.md;md5=56b096e9d5a688ac2ecce3670434871a"

Whereas master generates this:

# NOTE: multiple licenses have been detected; they have been separated with &
# in the LICENSE value for now since it is a reasonable assumption that all
# of the licenses apply. If instead there is a choice between the multiple
# licenses then you should change the value to separate the licenses with |
# instead of &. If there is any doubt, check the accompanying documentation
# to determine which situation is applicable.
#
# The following license files were not able to be identified and are
# represented as "Unknown" below, you will need to check them yourself:
#   docs/theme/extra/templates/license.html
#   docs/markdown/legal.md
#
# NOTE: License in setup.py/PKGINFO is: Apache License, Version 2.0
LICENSE = "Apache-2.0"
LIC_FILES_CHKSUM = "file://COPYING;md5=3b83ef96387f14655fc854ddc3c6bd57 \
                    file://docs/theme/extra/templates/license.html;md5=8490f984473ca7fa3bdcfe2a919e6249 \
                    file://docs/markdown/legal.md;md5=56b096e9d5a688ac2ecce3670434871a"


We probe the Meson repository but that has recently changed so those new files are found but not identified.  Master still passes selftest because it fails to put Unknown in the LICENSE field despite the comment it generates saying that it will be.

CCing Paul.

1) can we lock the recipetool create invocation to a specific tag so we don't get caught with upstream changes?
2) Is Unknown not being in LICENSE a bug in master? (I suspect so)
Comment 2 Paul Eggleton 2017-05-25 03:50:29 UTC
1) Yes we can (sorry I should have done that in the first place).

2) This is tricky. When Chris implemented the python module handling, he made it overwrite what's in the LICENSE string to match what the python package information has recorded (if what is there can be translated appropriately). Unfortunately the comments we now write out don't match that and there's a chance that some code with associated license file gets included in the tree and thus picked up by the code and yet it might not be represented if its license isn't mentioned in the package info. In any event that has been the case since the python handling code was introduced - we could decide to go back on that but it hasn't been discussed yet. What has changed since though is that we try to be a bit more intelligent about handing the License and Classifier fields so that handling common licenses is a bit more likely, however that change went in in September so it's a bit of a puzzle as to why it's only breaking now.
Comment 3 Ross Burton 2017-05-25 11:13:14 UTC
1) Cool. What's the syntax?
2) I think the change to meson was only recently made and we haven't ran a morty selftest for a while.
Comment 4 Paul Eggleton 2017-05-25 20:22:20 UTC
You can specify the revision by adding ;rev=<revision> to the end of the URL. Don't forget to enclose the whole thing in quotes or the shell will eat that part though. (FYI Rebecca is adding an explicit option to specify SRCREV so that this isn't as hard to deal with in future.)
Comment 5 Ross Burton 2017-06-07 14:49:59 UTC
Fixed in oe-core master as of b35bedd209092432c560e998043b6a8c5c2e4d34 and backported to the stable branches.
Comment 6 oscar lopez arandas 2017-06-08 21:23:44 UTC
This fault also comes out in krogoth

[krogoth]test_recipetool_create_github(oeqa.selftest.recipetool.RecipetoolTests)

STEPS TO REPRODUCE: 

1.-clone poky
2.-source oe-init-build-env
3.-oe-selftest oe-selftest --run-all 


EXPECTED RESULT

All test cases should be OK 


ACTUAL RESULT

When running selftest on GDC AB Ubuntu16.04 is failing with below output:


==============================================================================
FAIL: test_recipetool_create_github (oeqa.selftest.recipetool.RecipetoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-ubuntu1604/build/meta/lib/oeqa/selftest/recipetool.py", line 457, in test_recipetool_create_github
    self._test_recipe_contents(recipefile, checkvars, inherits)
  File "/home/ab/autobuilder/yocto-worker/nightly-oe-selftest-ubuntu1604/build/meta/lib/oeqa/selftest/devtool.py", line 49, in _test_recipe_contents
    self.assertEqual(value, needvalue, 'values for %s do not match' % var)
AssertionError: Items in the second set but not the first:
'Unknown' : values for LICENSE do not match.
----------------------------------------------------------------------
Full log attached:
Comment 7 oscar lopez arandas 2017-06-08 21:24:17 UTC
Created attachment 3807 [details]
ubuntu_1604_krogoth
Comment 8 Ross Burton 2017-06-08 21:32:39 UTC
What SHA did you use?  The stuff that went to QA was broken, the fix landed immediately afterwards.
Comment 9 oscar lopez arandas 2017-06-08 21:35:18 UTC
BUILD: 2.1.3: rc1-fe7fb002216f7886d49942333ea6d8592339bb05
ENVIRONMENT: ubuntu 16.04 x86_64
Comment 10 Richard Purdie 2017-06-12 14:38:12 UTC
This was fixed in the commit immediately after this, http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=krogoth&id=3565a9697f53ba975a1b7235b802f659418746c3. Its not worth respinning the point release for.
Comment 11 Richard Purdie 2017-06-12 14:38:28 UTC
Resolve as per the last comment
Comment 12 oscar lopez arandas 2017-06-12 22:46:22 UTC
This was resolved in the commit added immediately: 3565a9697f53ba975a1b7235b802f659418746c3


STATUS: IDLE
BUILD: 2.1.3: 3565a9697f53ba975a1b7235b802f659418746c3
ENVIRONMENT: ubuntu 16.04 x86_64 

STEPS TO REPRODUCE: 

1.-clone poky
2.-source oe-init-build-env
3.-oe-selftest oe-selftest --run-all 


EXPECTED RESULT

All test cases should be OK 


ACTUAL RESULT
All test cases should be OK