<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>14328</bug_id>
          
          <creation_ts>2021-03-25 15:35:02 +0000</creation_ts>
          <short_desc>Mitigate AB ptest intermittent failure</short_desc>
          <delta_ts>2021-05-20 15:13:45 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>10</classification_id>
          <classification>QA/Testing</classification>
          <product>Package Testing (ptest)</product>
          <component>ptest</component>
          <version>unspecified</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard>AB-INT</status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>3.4 M1</target_milestone>
          
          <blocked>14163</blocked>
          <everconfirmed>1</everconfirmed>
          <reporter name="Yi Fan Yu">yf3yu</reporter>
          <assigned_to name="Richard Purdie">richard.purdie</assigned_to>
          <cc>alexandre.belloni</cc>
    
    <cc>randy.macleod</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>Don&apos;t know</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>89768</commentid>
    <comment_count>0</comment_count>
    <who name="Yi Fan Yu">yf3yu</who>
    <bug_when>2021-03-25 15:35:02 +0000</bug_when>
    <thetext>It feels like an I/O load issue, work on some sort of `flock`/`dd` timeout is currently done to try to get to the bottom of the cause and verify if it is indeed I/O related.


2 solutions have been suggested in the Mar 25 triage 

IIRC they were: 

* cgroups control to give qemu an advantage over builds
* nice levels to make qemu higher priority (lowering the builds priority)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89841</commentid>
    <comment_count>1</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2021-04-01 14:35:24 +0000</bug_when>
    <thetext>We were doing this anyway but this bug can be used to track things.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89944</commentid>
    <comment_count>2</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2021-04-15 13:35:10 +0000</bug_when>
    <thetext>Most autobuilder workers only have cgroups v1 which doesn&apos;t give the IO controls we&apos;d need. We already set ionice/nice levels. 

The ionice level doesn&apos;t affect async writes.

We&apos;ve now added support for running qemu images within tmpfs.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>89958</commentid>
    <comment_count>3</comment_count>
    <who name="Alexandre Belloni">alexandre.belloni</who>
    <bug_when>2021-04-15 14:58:58 +0000</bug_when>
    <thetext>Following the triage call, it seems necessary to:
 - timestamp the start and end of the image copy to tmpfs
 - double the qemu start timeout</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>90431</commentid>
    <comment_count>4</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2021-05-20 15:13:32 +0000</bug_when>
    <thetext>Many changes including:
 - moving the qemu rootfs to tmpfs
 - limiting glibc self-test memory usage
 - adding smp to qemu image for x86, arm and changing the x86 target machine.
and likely more have improve ptests. Closing this generic bug. Open specific issues for additional problems.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>90432</commentid>
    <comment_count>5</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2021-05-20 15:13:45 +0000</bug_when>
    <thetext>As per previous comment.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>