<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>8170</bug_id>
          
          <creation_ts>2015-08-16 20:00:19 +0000</creation_ts>
          <short_desc>testimage does not correctly support testing LSB images such as full-cmdline</short_desc>
          <delta_ts>2015-09-03 11:52:08 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>10</classification_id>
          <classification>QA/Testing</classification>
          <product>Runtime Testing</product>
          <component>general</component>
          <version>unspecified</version>
          <rep_platform>All</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>VERIFIED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>High</priority>
          <bug_severity>critical</bug_severity>
          <target_milestone>1.9 M3</target_milestone>
          
          <blocked>8130</blocked>
          <everconfirmed>1</everconfirmed>
          <reporter name="Saul Wold">sgw</reporter>
          <assigned_to name="Costin">costin.c.constantin</assigned_to>
          <cc>bluelightning</cc>
    
    <cc>bogdanx.a.voiculescu</cc>
    
    <cc>costin.c.constantin</cc>
    
    <cc>elizabeth.flanagan</cc>
    
    <cc>richard.purdie</cc>
    
    <cc>ross.burton</cc>
          
          <qa_contact name="Cristina Agurida">cristina-danielax.agurida</qa_contact>
          <cf_os>---</cf_os>
          <cf_regression_type>New (Never tested)</cf_regression_type>
          
          <cf_docchange>No (bug/feature does not impact docs)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>53134</commentid>
    <comment_count>0</comment_count>
    <who name="Saul Wold">sgw</who>
    <bug_when>2015-08-16 20:00:19 +0000</bug_when>
    <thetext>Based on Richards request to setup the Autobuilder for lsb testing via testimage, I attempted a build with DISTRO=poky-lsb and core-image-full-cmdline. When I ran the -c testimage, I got the following failures.

This clearly shows that we need some modifications to the test suite for correctly testing LSB images:


| ======================================================================
| FAIL: test_pam (oeqa.runtime.pam.PamBasicTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py&quot;, line 102, in wrapped_f
|     return func(*args)
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py&quot;, line 90, in wrapped_f
|     return f(*args)
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/runtime/pam.py&quot;, line 25, in test_pam
|     self.assertEqual(status, 0, msg = &quot;useradd command does not work as expected. Status and output:%s and %s&quot; %(status, output))
| AssertionError: 127 != 0 : useradd command does not work as expected. Status and output:127 and sh: useradd: command not found
| 
| ======================================================================
| FAIL: test_skeleton_availability (oeqa.runtime.skeletoninit.SkeletonBasicTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py&quot;, line 90, in wrapped_f
|     return f(*args)
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/runtime/skeletoninit.py&quot;, line 19, in test_skeleton_availability
|     self.assertEqual(status, 0, msg = &quot;skeleton init script not found. Output:\n%s &quot; % output)
| AssertionError: 2 != 0 : skeleton init script not found. Output:
| ls: cannot access /etc/init.d/skeleton: No such file or directory
| 
| ======================================================================
| FAIL: test_skeleton_script (oeqa.runtime.skeletoninit.SkeletonBasicTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py&quot;, line 102, in wrapped_f
|     return func(*args)
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/utils/decorators.py&quot;, line 90, in wrapped_f
|     return f(*args)
|   File &quot;/srv/hdd/poky/meta/lib/oeqa/runtime/skeletoninit.py&quot;, line 29, in test_skeleton_script
|     self.assertEqual(status, 0, msg = &quot;Skeleton script could not be started:\n%s\n%s&quot; % (output1, output2))
| AssertionError: 1 != 0 : Skeleton script could not be started:
| sh: /etc/init.d/skeleton: No such file or directory
| 
| 
| ----------------------------------------------------------------------
| Ran 21 tests in 36.737s
| 
This will block the proper implementation of this on the Autobuilder as all LSB images will be red, therefore I am marking this as a Critical High for M3.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53143</commentid>
    <comment_count>1</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2015-08-17 09:47:08 +0000</bug_when>
    <thetext>A minor note - core-image-full-cmdline is not an LSB image. It is used as the basis for LSB images though.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53158</commentid>
    <comment_count>2</comment_count>
    <who name="Saul Wold">sgw</who>
    <bug_when>2015-08-17 18:31:47 +0000</bug_when>
    <thetext>It seems that the oeRuntimeTest.hasPackage(&quot;service&quot;) matches a package group: 
packagegroup-core-full-cmdline-sys-services, not the meta-skeleton service 
package, so part of the problem might be that hasPackage() needs to exact 
matching or use some regular expression format.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53159</commentid>
    <comment_count>3</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2015-08-17 18:36:52 +0000</bug_when>
    <thetext>It does look like it&apos;s doing a re.search() on the entire package list file contents - which seems wrong to me especially given how it is being used in the various tests. It ought to be doing a re.match() on each item individually if we even need regex support.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53245</commentid>
    <comment_count>4</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-08-20 07:11:54 +0000</bug_when>
    <thetext>i made a test by replacing re.search() with re.match(). This time testimage signals skipped packages and tests go on as expected. 
Still i spotted an issue with useadd but i guess this is not related to this bug:

 ======================================================================
| FAIL: test_pam (oeqa.runtime.pam.PamBasicTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File &quot;/media/yp_builds/5poky/meta/lib/oeqa/utils/decorators.py&quot;, line 102, in wrapped_f
|     return func(*args)
|   File &quot;/media/yp_builds/5poky/meta/lib/oeqa/utils/decorators.py&quot;, line 90, in wrapped_f
|     return f(*args)
|   File &quot;/media/yp_builds/5poky/meta/lib/oeqa/runtime/pam.py&quot;, line 25, in test_pam
|     self.assertEqual(status, 0, msg = &quot;useradd command does not work as expected. Status and output:%s and %s&quot; %(status, output))
| AssertionError: 127 != 0 : useradd command does not work as expected. Status and output:127 and sh: useradd: command not found
|
| ----------------------------------------------------------------------</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53247</commentid>
    <comment_count>5</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-08-20 07:33:10 +0000</bug_when>
    <thetext>related to useradd the resulting image contains luseradd so this is why the related assert to the useradd --help fails</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53252</commentid>
    <comment_count>6</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2015-08-20 08:01:22 +0000</bug_when>
    <thetext>It needs to be more than just a swap of re.search() with re.match() - we also need to do the matching on individual items rather than the whole list.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53270</commentid>
    <comment_count>7</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-08-20 14:17:31 +0000</bug_when>
    <thetext>Hi Paul,

I followed your recommendation and did a split by each line.
However i got a situation similar to the initial one for syslog. I built the same image as on comment#1 and in  the .manifest file i can find busybox-syslog. So even if the service exists, testimage is not treating it: &quot;skipped &apos;syslog: No syslog package in image&apos;&quot;
Should i add an exception for busybox-* ?

Thanks</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53308</commentid>
    <comment_count>8</comment_count>
    <who name="Paul Eggleton">bluelightning</who>
    <bug_when>2015-08-21 08:23:03 +0000</bug_when>
    <thetext>We don&apos;t want a hardcoded exception here - if the test genuinely works for both syslog and busybox-syslog then you will have to check *in the test* for both packages.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53319</commentid>
    <comment_count>9</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-08-21 12:23:12 +0000</bug_when>
    <thetext>patch submitted upstream</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53411</commentid>
    <comment_count>10</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-08-26 12:35:09 +0000</bug_when>
    <thetext>patch merged:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=393bd7496d71eb101f21234c1233a2d18fd2c73e</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>53675</commentid>
    <comment_count>11</comment_count>
    <who name="Cristina Agurida">cristina-danielax.agurida</who>
    <bug_when>2015-09-03 11:52:08 +0000</bug_when>
    <thetext>Verified on master: 71b0568fa43285f0946fae93fb43cea5f3bbecec</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>