Bug 11118 - selftest: some testexport tests need a previous build to run
Summary: selftest: some testexport tests need a previous build to run
Status: RESOLVED NOTABUG
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 2.3 M4
Assignee: Jose Perez C
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks: 10937
  Show dependency tree
 
Reported: 2017-03-02 16:09 UTC by Benjamin Esquivel
Modified: 2017-12-06 17:26 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
local AB execution (427.78 KB, application/octet-stream)
2017-03-09 15:59 UTC, Jose Perez C 		no flags Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Benjamin Esquivel 2017-03-02 16:09:42 UTC 
The following tests are failing because of the need of a previous build to run and pass:

> 1. FAIL [104.180s]: test_testexport_sdk (oeqa.selftest.runtime-
> test.TestExport)
> 2. FAIL [128.437s]: test_testexport_basic (oeqa.selftest.runtime-
> test.TestExport)

Tests should be self-sufficient to run by minimizing the user needing to do specific actions that the test can do, like a dependency on an image built prior testing.

Procedure to reproduce:

1. download a fresh yoctproject poky clone
2. init the build environment
3. execute an oe-selftest session with only these tests in the filter.
   $ oe-selftest -r oeqa.selftest.runtime-test.TestExport.test_testexport_basic
   $ oe-selftest -r oeqa.selftest.runtime-test.TestExport.test_testexport_sdk

Expected outcome: the test passes.
Actual outcome: the test fails.
Workaround: prior step 3, build a core-image-sato
Comment 1 Benjamin Esquivel 2017-03-02 16:12:30 UTC 
Above tests were carried on in an AB worker with opensuse421
Comment 2 Benjamin Esquivel 2017-03-02 17:19:19 UTC 
step 3 on the first comment needs an update:

3. execute an oe-selftest session with only these tests in the filter.
   $ oe-selftest -r runtime-test.TestExport.test_testexport_basic
   $ oe-selftest -r runtime-test.TestExport.test_testexport_sdk
Comment 3 Mariano Lopez 2017-03-02 20:29:02 UTC 
I tried to reproduce running selftest manually, using a brand new cloned poky with and without sstates; I wasn't able to trigger the error. Is there a log message that can give us an insight what is going on? I remember that some days ago testexport was broken, perhaps you are seeing these errors?
Comment 4 Benjamin Esquivel 2017-03-03 00:24:02 UTC 
I'll ask for a staging autobuilder job to see if it is fixed.
Comment 5 Jose Perez C 2017-03-09 15:59:33 UTC 
Created attachment 3645 [details]
local AB execution
Comment 6 Jose Perez C 2017-03-09 16:02:04 UTC 
(In reply to comment #5)
> Created attachment 3645 [details]
> local AB execution

Bug was not reproduced anymore on local machine neither local AB, as Mariano says in Comment 3, testexport was broken and this may be the cause of the failures. Closing as not a bug.