<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>13716</bug_id>
          
          <creation_ts>2019-12-19 16:26:18 +0000</creation_ts>
          <short_desc>Master] SSTATE_MIRROR hangs if mirror site is down.</short_desc>
          <delta_ts>2021-11-13 16:42:29 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>7</classification_id>
          <classification>Build System, Metadata &amp; Runtime</classification>
          <product>OE-Core</product>
          <component>core</component>
          <version>3.1</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>High</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>4.0 M1</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Armin Kuster">akuster</reporter>
          <assigned_to name="Richard Purdie">richard.purdie</assigned_to>
          <cc>akuster808</cc>
    
    <cc>meta.mr.watcher</cc>
    
    <cc>meta.watcher</cc>
    
    <cc>mhalstead</cc>
    
    <cc>randy.macleod</cc>
    
    <cc>richard.purdie</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>Don&apos;t know</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>85867</commentid>
    <comment_count>0</comment_count>
    <who name="Armin Kuster">akuster</who>
    <bug_when>2019-12-19 16:26:18 +0000</bug_when>
    <thetext>I have these variables defined:

SOURCE_MIRROR_URL ?= &quot;http://downloads.yoctoproject.org/mirror/sources/&quot;
SSTATE_MIRRORS ?= &quot;file://.* http://sstate.yoctoproject.org/PATH&quot;


BB_SIGNATURE_HANDLER = &quot;OEEquivHash&quot;


The mirrors are down so there is a very long ETA. 

Possible need some sort of timeout and quit?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>91828</commentid>
    <comment_count>1</comment_count>
    <who name="Michael Halstead">mhalstead</who>
    <bug_when>2021-10-26 19:55:59 +0000</bug_when>
    <thetext>We are planning a downtime for the YP mirrors late this year to move data centers. If there isn&apos;t a code fix for this by then I&apos;ll find a way to ensure quick timeouts during the downtime. I need to test if a server that responds with 503 temporarily unavailable for all requests allows for quick timeouts.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>91928</commentid>
    <comment_count>2</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2021-11-09 13:26:11 +0000</bug_when>
    <thetext>I&apos;ve sent out a mirror addition for uninative which partially helps some of the issues experienced last time.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>91930</commentid>
    <comment_count>3</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2021-11-09 13:30:46 +0000</bug_when>
    <thetext>I also did some failure analysis. Last time when this happened, the webserver for sstate.yoctoproject.org was alive but it&apos;s filesystem had gone which meant connections to the system were hung open for a long period.

If I change my /etc/hosts to add something like:

192.168.1.123   sstate.yoctoproject.org git.yoctoproject.org downloads.yoctoproject.org

i.e. point at an invalid address, then the connections fail and the build proceeds just fine, either finding things from mirrors or not but it doesn&apos;t hang.

I therefore think the downtime later in the year should be ok as long as we don&apos;t have a half operating webserver hanging connections. We do need to make sure sources are available on mirrors though and Michael and I are putting a plan in place for that, uninative is already covered.

Once we have the mirrors sorted, this bug can drop in priority and we can analyse connection timeouts to address the other issue with a non-responsive server.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>91957</commentid>
    <comment_count>4</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2021-11-11 14:26:48 +0000</bug_when>
    <thetext>Fix for the hang:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=2c46245f449d2716566ef668da0bf48f2109643a

We just need an additional sources mirror url and we should be able to resolve this.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>92002</commentid>
    <comment_count>5</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2021-11-13 16:42:29 +0000</bug_when>
    <thetext>Final mirrors addition:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=33708f024d9b8b273dfe6d5b602f193c0f6a0878</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>