<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>7321</bug_id>
          
          <creation_ts>2015-02-13 14:35:46 +0000</creation_ts>
          <short_desc>Race in do_shared_workdir</short_desc>
          <delta_ts>2015-11-04 13:45:39 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>7</classification_id>
          <classification>Build System, Metadata &amp; Runtime</classification>
          <product>OE-Core</product>
          <component>kernel</component>
          <version>1.8</version>
          <rep_platform>All</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>VERIFIED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>1.9 M3</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Ross Burton">ross.burton</reporter>
          <assigned_to name="Ed Bartosh">eduard.bartosh</assigned_to>
          <cc>alexandru.c.georgescu</cc>
    
    <cc>bruce.ashfield</cc>
    
    <cc>eduard.bartosh</cc>
    
    <cc>richard.purdie</cc>
    
    <cc>tom.zanussi</cc>
          
          <qa_contact name="Costin">costin.c.constantin</qa_contact>
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>No (bug/feature does not impact docs)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>48812</commentid>
    <comment_count>0</comment_count>
    <who name="Ross Burton">ross.burton</who>
    <bug_when>2015-02-13 14:35:46 +0000</bug_when>
    <thetext>This error happened on the autobuilder:

DEBUG: Executing shell function do_shared_workdir
cp: cannot stat &apos;include/generated/utsrelease.h.tmp&apos;: No such file or directory
WARNING: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/qemux86_64-poky-linux-gnux32/linux-yocto/3.14.29+gitAUTOINC+6eddbf4787_f65678ef48-r0/temp/run.do_shared_workdir.30276:1 exit 1 from
  cp -fR include/generated/asm-offsets.h include/generated/autoconf.h include/generated/bounds.h include/generated/compile.h include/generated/uapi include/generated/utsrelease.h $kerneldir/include/generated/
ERROR: Function failed: do_shared_workdir (log file is located at /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/qemux86_64-poky-linux-gnux32/linux-yocto/3.14.29+gitAUTOINC+6eddbf4787_f65678ef48-r0/temp/log.do_shared_workdir.30276)

It was in a MUT run but there wasn&apos;t anything that should have impacted the kernel in the branch over master.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>48819</commentid>
    <comment_count>1</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2015-02-13 15:02:11 +0000</bug_when>
    <thetext>We also have a race on kernel.release being reported on the mailing list.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>48821</commentid>
    <comment_count>2</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-02-13 15:07:33 +0000</bug_when>
    <thetext>I&apos;m out of the office until the 17th, so if anyone has ideas, feel
free to file them here (or take a crack at this).

My suggestion on the mailing list was to save these files at the end
of do_compile() and then finish the work in do_shared_workdir() .. but
then we ask why we still need do_shared_workdir() at all. Removing it
would mean that dependencies needed to be updated again, or we&apos;d keep
it around for a dummy dependency.

Alternatively, can we drop a lock file down in do_compile, and have
it as a dependency for other tasks ? We only then remove it in do_shared_workdir() ?
I&apos;m not sure on the bitbake internals to support a scheme like that.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49383</commentid>
    <comment_count>3</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-11 21:47:20 +0000</bug_when>
    <thetext>Here is my attempt to fix this by getting rid of do_shared_workdir: https://lists.yoctoproject.org/pipermail/yocto/2015-March/024043.html

I haven&apos;t noticed any performance degradation with this change. Here are results of &apos;time bitbake core-image-minimal&apos; running on the same machine before and after the change:

before:
real    30m32.259s
user    135m54.912s
sys     24m48.292s

after:
real    30m23.198s
user    134m52.312s
sys     24m9.100s</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49390</commentid>
    <comment_count>4</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 03:32:42 +0000</bug_when>
    <thetext>As I stated on the list .. I do not want to juggle these task 
definitions again .. unless there&apos;s absolutely no other solution.

They&apos;ve been out there now, and have been adopted already. We are
now forcing more churn.

(In reply to comment #3)
&gt; Here is my attempt to fix this by getting rid of do_shared_workdir:
&gt; https://lists.yoctoproject.org/pipermail/yocto/2015-March/024043.html
&gt; 
&gt; I haven&apos;t noticed any performance degradation with this change. Here are
&gt; results of &apos;time bitbake core-image-minimal&apos; running on the same machine
&gt; before and after the change:
&gt; 
&gt; before:
&gt; real    30m32.259s
&gt; user    135m54.912s
&gt; sys     24m48.292s
&gt; 
&gt; after:
&gt; real    30m23.198s
&gt; user    134m52.312s
&gt; sys     24m9.100s</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49392</commentid>
    <comment_count>5</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-12 08:25:52 +0000</bug_when>
    <thetext>(In reply to comment #2)
&gt; My suggestion on the mailing list was to save these files at the end
&gt; of do_compile() and then finish the work in do_shared_workdir() .. but
&gt; then we ask why we still need do_shared_workdir() at all. Removing it
&gt; would mean that dependencies needed to be updated again, or we&apos;d keep
&gt; it around for a dummy dependency.
&gt;
Can you elaborate on this? Which dependencies need to be updated?

do_shared_workdir runs in parallel with other tasks, which is the reason for race. My idea was that making this functionality part of do_compile would fix the race.

Locking or copying files to yet another place doesn&apos;t look better than removing do_shared_workdir to me. Moreover, locking has the same effect as my change, just more complex and fragile.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49402</commentid>
    <comment_count>6</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 12:17:23 +0000</bug_when>
    <thetext>(In reply to comment #5)
&gt; (In reply to comment #2)
&gt; &gt; My suggestion on the mailing list was to save these files at the end
&gt; &gt; of do_compile() and then finish the work in do_shared_workdir() .. but
&gt; &gt; then we ask why we still need do_shared_workdir() at all. Removing it
&gt; &gt; would mean that dependencies needed to be updated again, or we&apos;d keep
&gt; &gt; it around for a dummy dependency.
&gt; &gt;
&gt; Can you elaborate on this? Which dependencies need to be updated?

This has been in tree for over a month now. You can only see 
the recipes in oe-core, and hence you are breaking a whole
set of recipes that have this dependency.

So no. Whatever we come up with, should not remove that task
definition again. I always get the email when we break other
kernel workflows, and since there are some light weight ways
to not subject more changes on those layers, I see no strong
reason to do it.

&gt; 
&gt; do_shared_workdir runs in parallel with other tasks, which is the reason for
&gt; race. My idea was that making this functionality part of do_compile would
&gt; fix the race.

And the original idea of it was to make it more lightweight, and 
something that can run without a full compile happening. The 
granularity had some intentions. We are breaking those intentions
be re-combining it.

&gt; 
&gt; Locking or copying files to yet another place doesn&apos;t look better than
&gt; removing do_shared_workdir to me. Moreover, locking has the same effect as
&gt; my change, just more complex and fragile.

Not really. Locking is common, and I wouldn&apos;t call it fragile ..
unless you&apos;d like to tag the entire code base this way :)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49403</commentid>
    <comment_count>7</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 12:28:54 +0000</bug_when>
    <thetext>I forgot to add, the route that I was taking was either locking
(and I was still looking at the proper semantics), or one way that
did seem to work was an empty shared_workdir task, with the bulk
of the code put back into do_compile. 

That kept the dependency the same, but avoided the race on the 
files.

I was also looking to patching the kernel build, but that is more
of a long term project, since I&apos;ll need to get those changes into
the mainline kernel.

So maybe to get this into the code base, break as few dependent
layers as possible, having that dummy task (with a big flashing
comment) is the right thing. It has the side benefit of being
able to incremental move blocks of code back to it, or transfer
more blobs to the artifacts dir, without needing to re-compile.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49406</commentid>
    <comment_count>8</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-12 13:05:53 +0000</bug_when>
    <thetext>Thank you for the explanations! Now it&apos;s clear to me why I didn&apos;t see those dependencies :)

(In reply to comment #7)
&gt; I forgot to add, the route that I was taking was either locking
&gt; (and I was still looking at the proper semantics), or one way that
&gt; did seem to work was an empty shared_workdir task, with the bulk
&gt; of the code put back into do_compile. 
&gt; 

I like second approach better. It&apos;s almost 
what I did, but much simpler as doesn&apos;t require removing do_shared_workdir.

&gt; That kept the dependency the same, but avoided the race on the 
&gt; files.
&gt; 
&gt; I was also looking to patching the kernel build, but that is more
&gt; of a long term project, since I&apos;ll need to get those changes into
&gt; the mainline kernel.
&gt; 
&gt; So maybe to get this into the code base, break as few dependent
&gt; layers as possible, having that dummy task (with a big flashing
&gt; comment) is the right thing. It has the side benefit of being
&gt; able to incremental move blocks of code back to it, or transfer
&gt; more blobs to the artifacts dir, without needing to re-compile.
Well, we can move only copying of include/generated to do_compile if you want. This is a minimal change to fix this bug.

Do you want me to update my patch and re-send it to openembedded-core ml?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49407</commentid>
    <comment_count>9</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 13:20:01 +0000</bug_when>
    <thetext>(In reply to comment #8)
&gt; Thank you for the explanations! Now it&apos;s clear to me why I didn&apos;t see those
&gt; dependencies :)
&gt; 
&gt; (In reply to comment #7)
&gt; &gt; I forgot to add, the route that I was taking was either locking
&gt; &gt; (and I was still looking at the proper semantics), or one way that
&gt; &gt; did seem to work was an empty shared_workdir task, with the bulk
&gt; &gt; of the code put back into do_compile. 
&gt; &gt; 
&gt; 
&gt; I like second approach better. It&apos;s almost 
&gt; what I did, but much simpler as doesn&apos;t require removing do_shared_workdir.
&gt; 
&gt; &gt; That kept the dependency the same, but avoided the race on the 
&gt; &gt; files.
&gt; &gt; 
&gt; &gt; I was also looking to patching the kernel build, but that is more
&gt; &gt; of a long term project, since I&apos;ll need to get those changes into
&gt; &gt; the mainline kernel.
&gt; &gt; 
&gt; &gt; So maybe to get this into the code base, break as few dependent
&gt; &gt; layers as possible, having that dummy task (with a big flashing
&gt; &gt; comment) is the right thing. It has the side benefit of being
&gt; &gt; able to incremental move blocks of code back to it, or transfer
&gt; &gt; more blobs to the artifacts dir, without needing to re-compile.
&gt; Well, we can move only copying of include/generated to do_compile if you
&gt; want. This is a minimal change to fix this bug.
&gt; 
&gt; Do you want me to update my patch and re-send it to openembedded-core ml?

Absolutely. You are in the mode and familiar with the code, so lets strike while
the iron is hot!</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49409</commentid>
    <comment_count>10</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-12 13:29:54 +0000</bug_when>
    <thetext>Great!

Can you look at this thread please: https://lists.yoctoproject.org/pipermail/poky/2015-February/010015.html

It looks like related issue to me. Should we also move copying of kernel.release to kernel_do_compile or it&apos;s better to do it in separate patch?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49411</commentid>
    <comment_count>11</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 13:35:43 +0000</bug_when>
    <thetext>(In reply to comment #10)
&gt; Great!
&gt; 
&gt; Can you look at this thread please:
&gt; https://lists.yoctoproject.org/pipermail/poky/2015-February/010015.html

At the time that thread popped up, I did think that it was the same root cause.

&gt; 
&gt; It looks like related issue to me. Should we also move copying of
&gt; kernel.release to kernel_do_compile or it&apos;s better to do it in separate
&gt; patch?

I&apos;d do it in the same patch.  In the end, I&apos;m ok with moving everything 
that is currently in the do_shared_workdir() into compile, as long as 
it is either directly called or the old task stays around .. we&apos;ll have our
flexibility and close the race for now.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49412</commentid>
    <comment_count>12</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-12 13:36:13 +0000</bug_when>
    <thetext>(In reply to comment #10)
&gt; Great!
&gt; 
&gt; Can you look at this thread please:
&gt; https://lists.yoctoproject.org/pipermail/poky/2015-February/010015.html

At the time that thread popped up, I did think that it was the same root cause.

&gt; 
&gt; It looks like related issue to me. Should we also move copying of
&gt; kernel.release to kernel_do_compile or it&apos;s better to do it in separate
&gt; patch?

I&apos;d do it in the same patch.  In the end, I&apos;m ok with moving everything 
that is currently in the do_shared_workdir() into compile, as long as 
it is either directly called or the old task stays around .. we&apos;ll have our
flexibility and close the race for now.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49435</commentid>
    <comment_count>13</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-12 20:29:26 +0000</bug_when>
    <thetext>done: http://lists.openembedded.org/pipermail/openembedded-core/2015-March/102636.html

Just out of curiosity. Bruce, can you point me out to oe-core recipes that use do_shared_workdir? I looked almost everywhere I could and didn&apos;t find anything except of what my first patch was fixing.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49439</commentid>
    <comment_count>14</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-13 03:44:00 +0000</bug_when>
    <thetext>(In reply to comment #13)
&gt; done:
&gt; http://lists.openembedded.org/pipermail/openembedded-core/2015-March/102636.
&gt; html
&gt; 
&gt; Just out of curiosity. Bruce, can you point me out to oe-core recipes that
&gt; use do_shared_workdir? I looked almost everywhere I could and didn&apos;t find
&gt; anything except of what my first patch was fixing.

The layers are in some about to release products, and also from
some queries that I received when the dependency was first changed
(I can&apos;t find them now, but will look again). Admittedly, the
breakage wouldn&apos;t be vast, I&apos;m just trying to minimize any 
disruptions, and then revisit this early in the next cycle.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49441</commentid>
    <comment_count>15</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-13 06:34:12 +0000</bug_when>
    <thetext>Thank you for the explanations. It definitely makes sense to be careful here.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49849</commentid>
    <comment_count>16</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-03-27 09:39:49 +0000</bug_when>
    <thetext>Bruce, my patch has not been accepted yet. Anything wrong with it? Anything is still required from me here?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>49854</commentid>
    <comment_count>17</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-03-27 12:10:01 +0000</bug_when>
    <thetext>This may have slipped through the cracks. I thought it would be
picked up by the normal master under test sweep, since you had
replied to my comments on the first version.

I&apos;ll pick it up and re-send it shortly.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50026</commentid>
    <comment_count>18</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-04-01 16:50:01 +0000</bug_when>
    <thetext>Still don&apos;t see my patch merged. Bruce, let me know if I can do anything about it. Thanks.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50029</commentid>
    <comment_count>19</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-04-01 20:05:47 +0000</bug_when>
    <thetext>I have the patch queued here (and have been testing with it),
but with the feature freeze coming into play, I elected to not
send that part of the queue and instead get it into the master
when it opened again and get a backport to the stable branch.

Unless we really want it sooner, I&apos;ll just fire it back out again
and it can land where it lands!</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50030</commentid>
    <comment_count>20</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-04-01 20:17:58 +0000</bug_when>
    <thetext>I think it&apos;s better to have it merged because it&apos;s scheduled for 1.8 M4 and it&apos;s just a small bugfix.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50037</commentid>
    <comment_count>21</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-04-02 13:01:53 +0000</bug_when>
    <thetext>I&apos;m on a plane today, so can&apos;t easily repost the patch. Can you add my
Signed-off-by along with yours and resend ?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50038</commentid>
    <comment_count>22</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-04-02 13:04:16 +0000</bug_when>
    <thetext>I&apos;ve just discussed it with Richard. He said we&apos;re late for 1.8 release, so no hurry.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50302</commentid>
    <comment_count>23</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-04-14 16:59:49 +0000</bug_when>
    <thetext>I decided to recent my patch. Added your Signed-off-by as you&apos;ve suggested: http://lists.openembedded.org/pipermail/openembedded-core/2015-April/103735.html</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50303</commentid>
    <comment_count>24</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-04-14 17:01:52 +0000</bug_when>
    <thetext>Fine with me!

I&apos;m about a day away from sending my latest series, and I&apos;ll just
drop this from my queue.

(In reply to comment #23)
&gt; I decided to recent my patch. Added your Signed-off-by as you&apos;ve suggested:
&gt; http://lists.openembedded.org/pipermail/openembedded-core/2015-April/103735.
&gt; html</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50335</commentid>
    <comment_count>25</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-04-16 10:53:44 +0000</bug_when>
    <thetext>The fix failed testing on autobuilder: https://autobuilder.yoctoproject.org/main/builders/nightly-rpm/builds/256/steps/Running%20Sanity%20Tests/logs/stdio

I&apos;m trying to reproduce this failure locally.

Bruce, do you have any idea what could be the reason?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50342</commentid>
    <comment_count>26</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2015-04-16 13:00:01 +0000</bug_when>
    <thetext>(In reply to comment #25)
&gt; The fix failed testing on autobuilder:
&gt; https://autobuilder.yoctoproject.org/main/builders/nightly-rpm/builds/256/
&gt; steps/Running%20Sanity%20Tests/logs/stdio
&gt; 
&gt; I&apos;m trying to reproduce this failure locally.
&gt; 
&gt; Bruce, do you have any idea what could be the reason?


argh. Nothing is jumping out. Have you looked at the qa test in
question to see if the kernel dependency is correct ? I&apos;ll pull
it up myself and check as well.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50815</commentid>
    <comment_count>27</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-05-06 09:10:23 +0000</bug_when>
    <thetext>Another build failed. this time differently: https://autobuilder.yoctoproject.org/main/builders/nightly-fsl-ppc-lsb/builds/288/steps/BuildImages/logs/stdio

Looks very strange as task order is good in the log:
NOTE: recipe linux-qoriq-3.12-r0: task do_compile: Succeeded
NOTE: recipe linux-qoriq-3.12-r0: task do_shared_workdir: Started
NOTE: recipe linux-qoriq-3.12-r0: task do_compile_kernelmodules: Started
NOTE: recipe linux-qoriq-3.12-r0: task do_shared_workdir: Succeeded
NOTE: recipe linux-qoriq-3.12-r0: task do_compile_kernelmodules: Succeeded

My guess is that shared dir should be created *after* compiling modules, not before it.

Bruce, does it make sense for you?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>50957</commentid>
    <comment_count>28</comment_count>
    <who name="Ed Bartosh">eduard.bartosh</who>
    <bug_when>2015-05-13 10:07:22 +0000</bug_when>
    <thetext>Merged to master: http://cgit.openembedded.org/openembedded-core/commit/?id=b4ec689f71fe9504559c1921c893f5e096d6b744</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>56183</commentid>
    <comment_count>29</comment_count>
    <who name="Costin">costin.c.constantin</who>
    <bug_when>2015-11-04 13:45:39 +0000</bug_when>
    <thetext>I had a look on both local and public AB machines for any &quot;ERROR: Function failed: do_shared_workdir&quot; message and didn&apos;t found any.(+ discussion with Ed on how to approach) As of this, marking as verified</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>