<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>3742</bug_id>
          
          <creation_ts>2013-01-17 11:32:06 +0000</creation_ts>
          <short_desc>bitbake hangs forever when PR serv is enabled</short_desc>
          <delta_ts>2013-04-18 07:13:52 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>7</classification_id>
          <classification>Build System, Metadata &amp; Runtime</classification>
          <product>BitBake</product>
          <component>bitbake</component>
          <version>unspecified</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>VERIFIED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Medium</priority>
          <bug_severity>major</bug_severity>
          <target_milestone>1.4</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Martin Jansa">Martin.Jansa</reporter>
          <assigned_to name="Richard Purdie">richard.purdie</assigned_to>
          <cc>alexandrux.palalau</cc>
    
    <cc>jessica.zhang</cc>
    
    <cc>poky.bs.watcher</cc>
    
    <cc>poky.watcher</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>---</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>28890</commentid>
    <comment_count>0</comment_count>
      <attachid>1014</attachid>
    <who name="Martin Jansa">Martin.Jansa</who>
    <bug_when>2013-01-17 11:32:06 +0000</bug_when>
    <thetext>Created attachment 1014
strace+fd

When building world which takes long time to finish I                                                                                                         
sometimes see bitbake hanging forever. It didn&apos;t happen                                                                                                       
until I&apos;ve enabled PR Service. But maybe it&apos;s not caused                                                                                                      
by PR Service and it&apos;s just coincidence.                                                                                                                      
                                                                                                                                                              
I&apos;ve to kill python processes manually and this traceback shown:                                                                                              
Exception in thread Thread-1:                                                                                                                                 
Traceback (most recent call last):                                                                                                                            
  File &quot;/usr/lib/python2.7/threading.py&quot;, line 551, in __bootstrap_inner                                                                                      
    self.run()                                                                                                                                                
  File &quot;/usr/lib/python2.7/threading.py&quot;, line 504, in run                                                                                                    
    self.__target(*self.__args, **self.__kwargs)                                                                                                              
  File &quot;/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/prserv/serv.py&quot;, line 173, in _work                                                           
    self.prserv.work_forever()                                                                                                                                
  File &quot;/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/prserv/serv.py&quot;, line 93, in work_forever                                                     
    self.handle_request()                                                                                                                                     
  File &quot;/usr/lib/python2.7/SocketServer.py&quot;, line 265, in handle_request                                                                                      
    fd_sets = select.select([self], [], [], timeout)                                                                                                          
error: (4, &apos;Interrupted system call&apos;)

When this happens, there is only one python process doing something                                                                                           
for very long time:
31073 jenkins    20   0  4300   592   504 S  0.0  0.0  0:01.18 │  ├─ tee log.world.20130116_045210.log/bitbake.log
31072 jenkins    20   0  4392   612   588 S  0.0  0.0  0:00.00 │  └─ /bin/sh /home/jenkins/oe/shr-core-branches/shr-core/openembedded-core/scripts/bitbake -k 
31095 jenkins    20   0  210M 92628  4396 S  0.0  0.2  1:59.29 │     └─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
31104 jenkins    20   0  210M 92628  4396 S  0.0  0.2  0:08.43 │        ├─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
31103 jenkins    20   0 1031M  676M  3196 R  7.0  1.4  1h12:11 │        └─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
31106 jenkins    20   0 1031M  676M  3196 S  0.0  1.4  0:16.07 │           ├─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
31105 jenkins    20   0 1031M  676M  3196 S  0.0  1.4  2:00.04 │           ├─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
17586 jenkins    20   0 1031M  675M  1792 S  0.0  1.4  0:00.13 │           ├─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
 2400 jenkins    20   0 1031M  676M  3196 S  0.0  1.4  0:00.00 │           ├─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world
 2399 jenkins    20   0 1031M  676M  3196 S  0.0  1.4  0:00.00 │           └─ python /home/jenkins/oe/shr-core-branches/shr-core/bitbake/bin/bitbake -k world

17586, 2399, 2400, 31105
are waiting in futex
futex(0x7fb9f1475fc8, FUTEX_WAIT_PRIVATE, 2, NULL &lt;unfinished ...&gt;

31106:
select(11, [10], [], [], {0, 500000})   = 0 (Timeout)
futex(0x6d8fc60, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
lrwx------ 1 jenkins jenkins 64 Jan 17 03:23 10 -&gt; socket:[414834257]
lrwx------ 1 jenkins jenkins 64 Jan 17 03:23 11 -&gt; /home/jenkins/oe/shr-core-branches/shr-core/cache/prserv.sqlite3

31095
select(4, [3], NULL, NULL, {0, 249996}) = 0 (Timeout)
lr-x------ 1 jenkins jenkins 64 Jan 17 03:23 3 -&gt; pipe:[414838024]
l-wx------ 1 jenkins jenkins 64 Jan 17 03:23 4 -&gt; pipe:[414838024]

31104
select(10, [9], [], [], {0, 500000})    = 0 (Timeout)
lrwx------ 1 jenkins jenkins 64 Jan 17 03:23 9 -&gt; socket:[414834253]
lrwx------ 1 jenkins jenkins 64 Jan 17 03:23 10 -&gt; /home/jenkins/oe/shr-core-branches/shr-core/cache/prserv.sqlite3

31103:
lrwx------ 1 jenkins jenkins 64 Jan 17 03:14 6 -&gt; socket:[414838023]
lrwx------ 1 jenkins jenkins 64 Jan 17 03:14 7 -&gt; /home/jenkins/oe/shr-core-branches/shr-core/bitbake.lock
lr-x------ 1 jenkins jenkins 64 Jan 17 03:14 24 -&gt; pipe:[429598906]
lr-x------ 1 jenkins jenkins 64 Jan 17 03:14 0 -&gt; pipe:[414831254]

and looping this very fast:
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0
select(0, NULL, NULL, NULL, {0, 100000}) = 0 (Timeout)
select(7, [6], NULL, NULL, {0, 0})      = 0 (Timeout)
read(24, 0x1139be44, 102400)            = -1 EAGAIN (Resource temporarily unavailable)
wait4(-1, 0x7fff157c4110, WNOHANG, NULL) = 0

I&apos;ll attach 7s of strace from each python process + /proc/pid/fd from each. Is there some better way to debug which pipe is from where?

Usually last cooker output is rm_work_all in gimp, but I don&apos;t see
anything special in gimp recipe and it could be some other thread
in background.

NOTE: Running task 22874 of 23570 (ID: 996, /home/jenkins/oe/shr-core-branches/shr-core/meta-openembedded/meta-oe/recipes-graphics/gimp/gimp_2.8.2.bb,
do_rm_work_all)
NOTE: recipe gimp-2.8.2-r0: task do_rm_work_all: Started
NOTE: recipe gimp-2.8.2-r0: task do_rm_work_all: Succeeded
Exception in thread Thread-1:
.. traceback from above</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>29236</commentid>
    <comment_count>1</comment_count>
    <who name="Martin Jansa">Martin.Jansa</who>
    <bug_when>2013-01-28 15:02:14 +0000</bug_when>
    <thetext>Interrupting bitbake with Ctrl+C also hangs now almost every time. Repeating Ctrl+C does not work. kill -9 from different screen window is needed :/.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>29584</commentid>
    <comment_count>2</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2013-02-06 14:50:12 +0000</bug_when>
    <thetext>Interrupting bitbake when its doing which step of the build? When runuqueue is running spawning tasks? I just tried that here and it works fine for me :/</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>29620</commentid>
    <comment_count>3</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2013-02-07 00:23:57 +0000</bug_when>
    <thetext>This looks like its from bad interaction between the threading module and the rest of bitbake. Resolved in : http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=bdfc5207a0fdbed889c4a5997306adab82b1c2d4</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>32335</commentid>
    <comment_count>4</comment_count>
    <who name="Alexandru Palalau">alexandrux.palalau</who>
    <bug_when>2013-04-18 07:13:52 +0000</bug_when>
    <thetext>Verified on master branch.
Commit ID: dab237bc6a7679dfeaa981c14736aedc7b04ecb9</thetext>
  </long_desc>
      
          <attachment
              isobsolete="0"
              ispatch="0"
              isprivate="0"
          >
            <attachid>1014</attachid>
            <date>2013-01-17 11:32:06 +0000</date>
            <delta_ts>2013-01-17 11:32:06 +0000</delta_ts>
            <desc>strace+fd</desc>
            <filename>strace_fd.tar.bz2</filename>
            <type>application/x-bzip</type>
            <size>1768</size>
            <attacher name="Martin Jansa">Martin.Jansa</attacher>
            
              <data encoding="base64">QlpoOTFBWSZTWTxAvPEAIeD/nPxQAQBA5//3I/3X6v////oIAAAEAAMACGAKnx8gqgAUAAAAAAoA
AEcZGmTE0GTJhNMgZDQGgNMmhgBNAY4yNMmJoMmTCaZAyGgNAaZNDACaAxxkaZMTQZMmE0yBkNAa
A0yaGAE0BjjI0yYmgyZMJpkDIaA0Bpk0MAJoDCKFRlEabSDRjUyYmCPRBgATJspkBoNMClKCBATJ
ppJ6BpNpDU8kzUZMTQegnkQyeo2p9/7wzy8/Wz1+zz11303du/DA04l9oZpLEZ0oq9n4GcOmdWud
M6s9nDlxDYGrN7NN+WjXNNdNs20cW7fOGMg9M5QYGKrSo96EEigWpVkF5FwjwE+sC6BoAZ8qfYLB
cohB6C7K2XJTHpaShWghMUIzAzStLcrkVyaTMO3A0wV2B2In8gyStaDzDRafnK5InPvH4yvwL5p6
8veHF3L140HrV6VxXvLguK5c8mTmOI4jnz222nHg4LmOYzfhWM3w6h60PEO0dsVceFegPcrsSTra
CJZZYvqRULklQkgLMrEJEJEJN2qmZPhtNJokm3k22mZMyuA20NmbYSUrLULQlVSzMrKkytIqEysQ
v/Yy0w+q+6zPXUdixWZK8iuYeRykwsENIiaeny7urhpq0111231+IdspPXesZOqVHmhfyGRfwK+A
f1K71J7PQVtK9QZL3ysKeVUs0yE5FeqDwKyvOoxJ2/mV0K0K3K1wrUrGmJO/PRivqKyV8hXOVgZE
aFcq5B8JWq3S3iswML1SJ3DEid8iaFfSVkqO6j5FH/L4PkotVJuPcMDBw9WZmZmZmZmZoNB4P6R/
01h+NxQ0o+1vV8dGsdGUaUY8XAPb+GGNak1gPWwdei0DVV9yk9G+vAt1eMriQM0uqiQK/eVRNEA6
HEfQgMJTjXlIpcAe3pzDT8mb1vtld7B3tc7xpDBwDgVgYraY70kxoxvq0arfRaWVyyZNA31NiwtM
rcrvDBWu3taTXcYOHxFaKowrWtBhXFaFc9+qalcy4Bhamhf8VsONdhW4cC6R8ZWOXFc2jXtDK0cG
OIY4LZyzUrVfsU34hr5cQrSub6c2K57Rrzv0aPHYKbmYT42iuvpt48MdnAdFz0jRmfsdXK/yY2y0
atYb5zYtNxnUSOto1GLwK1aNSt9KtAzQ7GQxdeuuZ6h2704FnaOrVjLax744rkOEkbNue7snU4sY
xjGMY9UNumaDXBTmalc0k3dWw9QbNvLM3Bq7NOHFcxpNOLJOMqOL+4rcriV3hoDu9pXd2BXX3jSn
X1YHXDkVp6Ovq1hXLSrTi7dcrKuGuYNl0G2uzNu4N6cuFm2mYzPBgbhi1K46no0TmNee2Zpw4Zi5
jLbfM0x3FwDx6y2wjdyzlKjzDS2G6NO8r1l2B6HfzHBdMSTxKx3ZU6Bhw37wzMrIOgOQ3WDv+Xx7
O4GdA54C00NNF0wuriW+rmuixmjXrXaNOJypvvitnDFuXQ2q6Vdy1K/mYprhIx2B6hgrAxGYOwfr
KwT2FemaQveHvyo+gr6CvsD/cr7A+QORWB/DQP1lfWV/ErKT7A2lJp/ilL8iUv4laldZW6UuRX2h
9/15mZmZmZmZn8g/GV6lzDqDqq2RNnNKX2hsHQOordeQZaVPopVhV8L4fd834vc2G02m02Tatttv
mzWazSa6a8NTaY+hw4b6lx4b1vW9alYG34mYZPxJfArtz8wdwfqK/QPpj5w+cr9JX3FfSP3bj+cp
OVGgYV9z9ofgHAr9RwH9XkV+sf0K+qjsfAVylvHYVwH8QyLqo0g2clHgqW5WcR098vm7hyq+cPUV
5ldtGg+0aQ9NH9Y76NXlHZ6Ss/chbFdFJxFw8X1qT9upX3PulKdiJ2thwi5UfoK7w3i5jqhW0A+V
70At8MAODm7f+d0RFoLaC6QF8B7D43hF5DKPgo0UeL2le+V+RE8g7B5ylPjHxIXcPlHMr2jqjglJ
2Ue3uHAehpFzlKcato8KL5SuhXjUn2Pwx21fWna5JLkV5PnDuo1RPJ1KtA7JScOhWlGDUbSlMkqc
6uA4B2jzi2o0Ob8NHg8isUchxK93Uie51oXmhaFdcdoxxjYdwdaj947lL3DQYOp2Dzo9BXWVij0l
dFWv7A4i8Y2D0jWOsloHvB3hkqTajmpOYbx4jwUeEXArzjA4B8Xj978v/35duoeA9ik5ompXSDzH
qHiG0Nu4rxHEPAPBsP9ByHqHsfCOdXAr+w5jslf2D7w0RPcG0d1XUV7Cvk+XdiwysLJWWDKrBuGe
uzMzMzGMmDJMmTJkyZMmFhmGWVhGGMH6RGBu+IrHapPq++v7+JWk1/PitKXFagveLuSKcKEgeIF5
4g==
</data>

          </attachment>
      

    </bug>

</bugzilla>