<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>13067</bug_id>
          
          <creation_ts>2018-12-07 12:51:20 +0000</creation_ts>
          <short_desc>setupClass failures not being handled correcting for parallel oe-selftest</short_desc>
          <delta_ts>2020-03-08 10:51:47 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>10</classification_id>
          <classification>QA/Testing</classification>
          <product>Functional (self) Testing</product>
          <component>oe-selftest</component>
          <version>unspecified</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>3.1 M3</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Richard Purdie">richard.purdie</reporter>
          <assigned_to name="Chen Qi">Qi.Chen</assigned_to>
          <cc>randy.macleod</cc>
    
    <cc>ross.burton</cc>
    
    <cc>tvgamblin</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>No (bug/feature does not impact docs)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>82432</commentid>
    <comment_count>0</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2018-12-07 12:51:20 +0000</bug_when>
    <thetext>If we run oe-selftest -r and there is a failure in setupClass, it doesn&apos;t get reported to the user clearly. A good example is this:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=6f2ef620d90ab39870bb6c02183249e0e1045aeb

which shows a clear error without the -j options but with it, it only shows:

2018-12-05 18:43:24,262 - oe-selftest - INFO - RESULTS - eSDK.oeSDKExtSelfTest.test_image_generation_binary_feeds - Testcase 1603: UNKNOWN
2018-12-05 18:43:24,262 - oe-selftest - INFO - RESULTS - eSDK.oeSDKExtSelfTest.test_install_libraries_headers - Testcase 1602: UNKNOWN

(from https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/55/steps/7/logs/step2d)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>82433</commentid>
    <comment_count>1</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2018-12-07 12:53:21 +0000</bug_when>
    <thetext>I should note the error is shown in the log but the summary at the end is wrong:

All required tests passed (successes=334, skipped=2, failures=0, errors=0)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>82443</commentid>
    <comment_count>2</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2018-12-07 17:50:23 +0000</bug_when>
    <thetext>Another example: https://autobuilder.yoctoproject.org/typhoon/#/builders/84/builds/61</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85575</commentid>
    <comment_count>3</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2019-11-19 16:30:22 +0000</bug_when>
    <thetext>It&apos;s good to get selftests working reliably sooner rather than later so target milestone is 3.1-M1.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86683</commentid>
    <comment_count>4</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2020-03-08 10:51:47 +0000</bug_when>
    <thetext>I tested and confirmed that Trevor&apos;s recent work did also fix this issue and its no longer present.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>