<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>8129</bug_id>
          
          <creation_ts>2015-08-07 22:41:28 +0000</creation_ts>
          <short_desc>ip absent from qemu args on core-image-sato sanity tests (several builders)</short_desc>
          <delta_ts>2015-08-18 15:12:39 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>10</classification_id>
          <classification>QA/Testing</classification>
          <product>Build Testing</product>
          <component>general</component>
          <version>1.9</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Undecided</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>---</target_milestone>
          
          <blocked>8176</blocked>
          <everconfirmed>1</everconfirmed>
          <reporter name="Alex Franco">alejandro.franco</reporter>
          <assigned_to name="Alejandro Hernandez">alejandro.hernandez</assigned_to>
          <cc>alejandro.hernandez</cc>
    
    <cc>alexandru.c.georgescu</cc>
    
    <cc>richard.purdie</cc>
    
    <cc>stanciux.mihail</cc>
          
          <qa_contact name="Mihail Stanciu">stanciux.mihail</qa_contact>
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>Don&apos;t know</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>52913</commentid>
    <comment_count>0</comment_count>
      <attachid>2640</attachid>
    <who name="Alex Franco">alejandro.franco</who>
    <bug_when>2015-08-07 22:41:28 +0000</bug_when>
    <thetext>Created attachment 2640
Results for failure running sanity tests on nightly-x32 build #422

qemu arguments do not include &quot;ip=&quot; when running Sanity Tests for:

Builder: nightly-qa-skeleton
Branch: master-next
last commit where this occurred: 005a5b60f00a898b701de6df7095adf7440acebf
See first attachment for [https://autobuilder.yoctoproject.org/main/builders/nightly-x32/builds/422/steps/Running%20Sanity%20Tests/logs/stdio]

Builder: nightly-x32
Branch: joshuagl/fido-next
last commit where this occurred: 7706ac25f38e33fe1dcebdcff7e141293308cffb

A recent patch (see the output of nightly-qa-skeleton  logs) improved</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52914</commentid>
    <comment_count>1</comment_count>
      <attachid>2641</attachid>
    <who name="Alex Franco">alejandro.franco</who>
    <bug_when>2015-08-07 22:44:49 +0000</bug_when>
    <thetext>Created attachment 2641
Results of failure running sanity tests for builder nightly-qa-skeleton build #424

Added attachment for [https://autobuilder.yoctoproject.org/main/builders/nightly-qa-skeleton/builds/424/steps/Running%20Sanity%20Tests/logs/stdio]</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52915</commentid>
    <comment_count>2</comment_count>
    <who name="Alejandro Hernandez">alejandro.hernandez</who>
    <bug_when>2015-08-07 23:21:40 +0000</bug_when>
    <thetext>(In reply to comment #0)
&gt; Created attachment 2640 [details]
&gt; Results for failure running sanity tests on nightly-x32 build #422
&gt; 
&gt; qemu arguments do not include &quot;ip=&quot; when running Sanity Tests for:
&gt; 
&gt; Builder: nightly-qa-skeleton
&gt; Branch: master-next
&gt; last commit where this occurred: 005a5b60f00a898b701de6df7095adf7440acebf
&gt; See first attachment for
&gt; [https://autobuilder.yoctoproject.org/main/builders/nightly-x32/builds/422/
&gt; steps/Running%20Sanity%20Tests/logs/stdio]
&gt; 
&gt; Builder: nightly-x32
&gt; Branch: joshuagl/fido-next
&gt; last commit where this occurred: 7706ac25f38e33fe1dcebdcff7e141293308cffb
&gt; 
&gt; A recent patch (see the output of nightly-qa-skeleton  logs) improved

Ok, so this isn&apos;t happening anymore on master-next thanks to the patch submitted, we catch the exception correctly, but we still don&apos;t know why the IP string wasn&apos;t there int he first place</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52916</commentid>
    <comment_count>3</comment_count>
    <who name="Alejandro Hernandez">alejandro.hernandez</who>
    <bug_when>2015-08-07 23:35:06 +0000</bug_when>
    <thetext>Following up, it doesn&apos;t make much sense, since the IP is passed via the runqemu script directly:

KERNEL_NETWORK_CMD=&quot;ip=192.168.7.$n2::192.168.7.$n1:255.255.255.0&quot;

this variable is then included in $KERNCMDLINE, by checking the target&apos;s filesystem, which could potentially be causing the issue, but the cases seem to be correctly defined, in specific, these logs are from a MACHINE=&quot;qemux86-64&quot; and a rootfstype=ext4, (which is what is checked to include $KERNEL_NETWORK_CMD in $KERNCMDLINE)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52917</commentid>
    <comment_count>4</comment_count>
    <who name="Alejandro Hernandez">alejandro.hernandez</who>
    <bug_when>2015-08-07 23:36:53 +0000</bug_when>
    <thetext>There must be something wrong when passing arguments to qemu since the cmdline:

/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-skeleton/build/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-system-x86_64-kernel/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-skeleton/build/build/tmp/deploy/images/qemux86-64/bzImage-qemux86-64.bin-cpucore2duo-vgavmware-no-reboot-serialtcp:127.0.0.1:57673-m256--appendvga=0 uvesafb.mode_option=640x480-32 root= rw mem=256M  oprofile.timer=1 rootfstype=ext4  console=tty1 console=ttyS0,115200n8

seems incomplete and its missing more than just the IPs</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52920</commentid>
    <comment_count>5</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2015-08-09 01:22:19 +0000</bug_when>
    <thetext>It failed with Alejandro&apos;s patch applied: https://autobuilder.yoctoproject.org/main/builers/nightly-arm/builds/428/steps/Running Sanity Tests/logs/stdio

which confirms it doesn&apos;t make it onto the commandline. The question is then why? I added some more debug onto the autobuilder:

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 02ac89c..06bd389 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -100,6 +100,7 @@ class QemuRunner:
         # oe-selftest (this makes stty error out immediately). There ought
         # to be a proper fix but this will suffice for now.
         self.runqemu = subprocess.Popen(launch_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, preexec_fn=os.setpgrp)
+        output = self.runqemu.stdout
 
         logger.info(&quot;runqemu started, pid is %s&quot; % self.runqemu.pid)
         logger.info(&quot;waiting at most %s seconds for qemu pid&quot; % self.runqemutime)
@@ -128,7 +129,10 @@ class QemuRunner:
                     self.ip = ips[0]
                     self.server_ip = ips[1]
             except IndexError, ValueError:
-                logger.info(&quot;Couldn&apos;t get ip from qemu process arguments! Here is the qemu command line used: %s&quot; % cmdline)
+                import fcntl
+                fl = fcntl.fcntl(output, fcntl.F_GETFL)
+                fcntl.fcntl(output, fcntl.F_SETFL, fl | os.O_NONBLOCK)
+                logger.info(&quot;Couldn&apos;t get ip from qemu process arguments! Here is the qemu command line used: %s\n %s&quot; % (cmdline, os.read(output.fileno(), 1000000)))
                 self.stop()
                 return False
             logger.info(&quot;Target IP: %s&quot; % self.ip)

which then shows this for the same failure:

ERROR: Function failed: core-image-minimal - FAILED to start qemu - check the task log and the boot log
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_testimage.6449
Log data follows:
| DEBUG: Executing python function do_testimage
| NOTE: rootfs file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/core-image-minimal-qemuarm-testimage.ext4
| NOTE: Qemu log file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/qemu_boot_log.20150809011726
| NOTE: SSH log file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/ssh_target_log.20150809011726
| NOTE: Created listening socket for qemu serial console on: 127.0.0.1:54588
| NOTE: runqemu started, pid is 6450
| NOTE: waiting at most 60 seconds for qemu pid
| NOTE: qemu started - qemu procces pid is 6509
| NOTE: Couldn&apos;t get ip from qemu process arguments! Here is the qemu command line used: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-system-arm-kernel/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/deploy/images/qemuarm/zImage-qemuarm.bin-Mversatilepb-drivefile=/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/core-image-minimal-qemuarm-testimage.ext4,format=raw-no-reboot-no-reboot-serialtcp:127.0.0.1:54588-m128--appendroot=/dev/sda rw console=ttyAMA0,115200 console=tty  mem=128M highres=off rootfstype=ext4  console=tty1 console=ttyS0,115200n8
|  Please use simplified serial or kvm options instead
| 
| Continuing with the following parameters:
| KERNEL: [/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/deploy/images/qemuarm/zImage-qemuarm.bin]
| ROOTFS: [/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/core-image-minimal-qemuarm-testimage.ext4]
| FSTYPE: [ext4]
| stty: standard input: Inappropriate ioctl for device
| Acquiring lockfile for tap0@NONE...
| Acquiring lockfile for tap1@NONE...
| Acquiring lockfile for tap2@NONE...
| Acquiring lockfile for tap3@NONE...
| Acquiring lockfile for tap4@NONE...
| Using preconfigured tap device &apos;tap4@NONE&apos;
| If this is not intended, touch /tmp/qemu-tap-locks/tap4@NONE.skip to make runqemu skip tap4@NONE.
| /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/scripts/runqemu-internal: line 256: 4@NONE: value too great for base (error token is &quot;4@NONE&quot;)
| Running qemu-system-arm...
| /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-system-arm -kernel /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/deploy/images/qemuarm/zImage-qemuarm.bin -M versatilepb -drive file=/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/build/tmp/work/qemuarm-poky-linux-gnueabi/core-image-minimal/1.0-r0/testimage/core-image-minimal-qemuarm-testimage.ext4,format=raw -no-reboot -no-reboot -serial tcp:127.0.0.1:54588 -m 128 --append &quot;root=/dev/sda rw console=ttyAMA0,115200 console=tty mem=128M highres=off rootfstype=ext4 console=tty1 console=ttyS0,115200n8&quot;
| 
| NOTE: Sending SIGTERM to runqemu
| DEBUG: Python function do_testimage finished
| ERROR: Function failed: core-image-minimal - FAILED to start qemu - check the task log and the boot log
ERROR: Task 0 (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm/build/meta/recipes-core/images/core-image-minimal.bb, do_testimage) failed with exit code &apos;1&apos;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52931</commentid>
    <comment_count>6</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2015-08-09 23:14:43 +0000</bug_when>
    <thetext>Improvement to runqemu script error handling: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=8020a9e44fffb687d4372c59239958776302a8ae

Ensure log output gets seen when runqemu fails:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=37e78463e58c374ba2ff933acfc4441cbb666c1f

Actual fix for issue:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=0968e3a68b0b06caeb827938ddf556f21bfce646

(turns out this only happens on fedora22)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>52933</commentid>
    <comment_count>7</comment_count>
    <who name="Richard Purdie">richard.purdie</who>
    <bug_when>2015-08-09 23:17:37 +0000</bug_when>
    <thetext>*** Bug 8112 has been marked as a duplicate of this bug. ***</thetext>
  </long_desc>
      
          <attachment
              isobsolete="0"
              ispatch="0"
              isprivate="0"
          >
            <attachid>2640</attachid>
            <date>2015-08-07 22:41:28 +0000</date>
            <delta_ts>2015-08-07 22:41:28 +0000</delta_ts>
            <desc>Results for failure running sanity tests on nightly-x32 build #422</desc>
            <filename>file_8129.txt</filename>
            <type>text/plain</type>
            <size>5328</size>
            <attacher name="Alex Franco">alejandro.franco</attacher>
            
              <data encoding="base64">DQojIyMgU2hlbGwgZW52aXJvbm1lbnQgc2V0IHVwIGZvciBidWlsZHMuICMjIw0KDQpZb3UgY2Fu
IG5vdyBydW4gJ2JpdGJha2UgPHRhcmdldD4nDQoNCkNvbW1vbiB0YXJnZXRzIGFyZToNCiAgICBj
b3JlLWltYWdlLW1pbmltYWwNCiAgICBjb3JlLWltYWdlLXNhdG8NCiAgICBtZXRhLXRvb2xjaGFp
bg0KICAgIGFkdC1pbnN0YWxsZXINCiAgICBtZXRhLWlkZS1zdXBwb3J0DQoNCllvdSBjYW4gYWxz
byBydW4gZ2VuZXJhdGVkIHFlbXUgaW1hZ2VzIHdpdGggYSBjb21tYW5kIGxpa2UgJ3J1bnFlbXUg
cWVtdXg4NicNCldpbGwgdXNlIGEgVk5DIHNlcnZlciBhbHJlYWR5IHJ1bm5pbmcgb24gOjENCldB
Uk5JTkc6IEhvc3QgZGlzdHJpYnV0aW9uICJGZWRvcmEtMjIiIGhhcyBub3QgYmVlbiB2YWxpZGF0
ZWQgd2l0aCB0aGlzIHZlcnNpb24gb2YgdGhlIGJ1aWxkIHN5c3RlbTsgeW91IG1heSBwb3NzaWJs
eSBleHBlcmllbmNlIHVuZXhwZWN0ZWQgZmFpbHVyZXMuIEl0IGlzIHJlY29tbWVuZGVkIHRoYXQg
eW91IHVzZSBhIHRlc3RlZCBkaXN0cmlidXRpb24uDQpQYXJzaW5nIHJlY2lwZXMuLi5kb25lLg0K
UGFyc2luZyBvZiA4ODggLmJiIGZpbGVzIGNvbXBsZXRlICgwIGNhY2hlZCwgODg4IHBhcnNlZCku
IDEyODkgdGFyZ2V0cywgNDEgc2tpcHBlZCwgMCBtYXNrZWQsIDAgZXJyb3JzLg0KTk9URTogUmVz
b2x2aW5nIGFueSBtaXNzaW5nIHRhc2sgcXVldWUgZGVwZW5kZW5jaWVzDQoNCkJ1aWxkIENvbmZp
Z3VyYXRpb246DQpCQl9WRVJTSU9OICAgICAgICA9ICIxLjI2LjAiDQpCVUlMRF9TWVMgICAgICAg
ICA9ICJ4ODZfNjQtbGludXgiDQpOQVRJVkVMU0JTVFJJTkcgICA9ICJGZWRvcmEtMjIiDQpUQVJH
RVRfU1lTICAgICAgICA9ICJ4ODZfNjQtcG9reS1saW51eC1nbnV4MzIiDQpNQUNISU5FICAgICAg
ICAgICA9ICJxZW11eDg2LTY0Ig0KRElTVFJPICAgICAgICAgICAgPSAicG9reSINCkRJU1RST19W
RVJTSU9OICAgID0gIjEuOCINClRVTkVfRkVBVFVSRVMgICAgID0gIm14MzIiDQpUQVJHRVRfRlBV
ICAgICAgICA9ICIiDQptZXRhICAgICAgICAgICAgICANCm1ldGEteW9jdG8gICAgICAgIA0KbWV0
YS15b2N0by1ic3AgICAgPSAiam9zaHVhZ2wvZmlkby1uZXh0Ojc3MDZhYzI1ZjM4ZTMzZmUxZGNl
YmRjZmY3ZTE0MTI5MzMwOGNmZmIiDQptZXRhLXF0MyAgICAgICAgICA9ICJmaWRvOjMwMTYxMjlk
OTBiN2FjODUxN2E1MjI3ZDgxOWYxMGFkNDE3YjViNDUiDQoNCk5PVEU6IFByZXBhcmluZyBSdW5R
dWV1ZQ0KTk9URTogRXhlY3V0aW5nIFNldFNjZW5lIFRhc2tzDQpOT1RFOiBFeGVjdXRpbmcgUnVu
UXVldWUgVGFza3MNCk5PVEU6IFJ1bm5pbmcgdGFzayAzMDIgb2YgMzAyIChJRDogMCwgL2hvbWUv
cG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVyL3lvY3RvLXdvcmtlci9uaWdodGx5LXgzMi9idWls
ZC9tZXRhL3JlY2lwZXMtY29yZS9pbWFnZXMvY29yZS1pbWFnZS1taW5pbWFsLmJiLCBkb190ZXN0
aW1hZ2UpDQpOT1RFOiByZWNpcGUgY29yZS1pbWFnZS1taW5pbWFsLTEuMC1yMDogdGFzayBkb190
ZXN0aW1hZ2U6IFN0YXJ0ZWQNCkVSUk9SOiBFcnJvciBleGVjdXRpbmcgYSBweXRob24gZnVuY3Rp
b24gaW4gL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVyL3lvY3RvLXdvcmtlci9uaWdo
dGx5LXgzMi9idWlsZC9tZXRhL3JlY2lwZXMtY29yZS9pbWFnZXMvY29yZS1pbWFnZS1taW5pbWFs
LmJiOg0KDQpUaGUgc3RhY2sgdHJhY2Ugb2YgcHl0aG9uIGNhbGxzIHRoYXQgcmVzdWx0ZWQgaW4g
dGhpcyBleGNlcHRpb24vZmFpbHVyZSB3YXM6DQpGaWxlOiAnZG9fdGVzdGltYWdlJywgbGluZW5v
OiA2LCBmdW5jdGlvbjogPG1vZHVsZT4NCiAgICAgMDAwMjpkZWYgZG9fdGVzdGltYWdlKGQpOg0K
ICAgICAwMDAzOiAgICB0ZXN0aW1hZ2VfbWFpbihkKQ0KICAgICAwMDA0Og0KICAgICAwMDA1Og0K
ICoqKiAwMDA2OmRvX3Rlc3RpbWFnZShkKQ0KICAgICAwMDA3Og0KRmlsZTogJ2RvX3Rlc3RpbWFn
ZScsIGxpbmVubzogMywgZnVuY3Rpb246IGRvX3Rlc3RpbWFnZQ0KICAgICAwMDAxOg0KICAgICAw
MDAyOmRlZiBkb190ZXN0aW1hZ2UoZCk6DQogKioqIDAwMDM6ICAgIHRlc3RpbWFnZV9tYWluKGQp
DQogICAgIDAwMDQ6DQogICAgIDAwMDU6DQogICAgIDAwMDY6ZG9fdGVzdGltYWdlKGQpDQogICAg
IDAwMDc6DQpGaWxlOiAndGVzdGltYWdlLmJiY2xhc3MnLCBsaW5lbm86IDU1LCBmdW5jdGlvbjog
dGVzdGltYWdlX21haW4NCiAgICAgMDA1MTogICAgICAgIGJiLmZhdGFsKCJMb2FkaW5nIHRlc3Rz
IGZhaWxlZDpcbiVzIiAlIHRyYWNlYmFjay5mb3JtYXRfZXhjKCkpDQogICAgIDAwNTI6DQogICAg
IDAwNTM6ICAgIHRhcmdldC5kZXBsb3koKQ0KICAgICAwMDU0Og0KICoqKiAwMDU1OiAgICB0YXJn
ZXQuc3RhcnQoKQ0KICAgICAwMDU2OiAgICB0cnk6DQogICAgIDAwNTc6ICAgICAgICBpZiBleHBv
cnQ6DQogICAgIDAwNTg6ICAgICAgICAgICAgZXhwb3J0VGVzdHMoZCx0YykNCiAgICAgMDA1OTog
ICAgICAgIGVsc2U6DQpGaWxlOiAnL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVyL3lv
Y3RvLXdvcmtlci9uaWdodGx5LXgzMi9idWlsZC9tZXRhL2xpYi9vZXFhL3RhcmdldGNvbnRyb2wu
cHknLCBsaW5lbm86IDE0OCwgZnVuY3Rpb246IHN0YXJ0DQogICAgIDAxNDQ6ICAgICAgICBiYi5u
b3RlKCJRZW11IGxvZyBmaWxlOiAlcyIgJSBzZWxmLnFlbXVsb2cpDQogICAgIDAxNDU6ICAgICAg
ICBzdXBlcihRZW11VGFyZ2V0LCBzZWxmKS5kZXBsb3koKQ0KICAgICAwMTQ2Og0KICAgICAwMTQ3
OiAgICBkZWYgc3RhcnQoc2VsZiwgcGFyYW1zPU5vbmUpOg0KICoqKiAwMTQ4OiAgICAgICAgaWYg
c2VsZi5ydW5uZXIuc3RhcnQocGFyYW1zKToNCiAgICAgMDE0OTogICAgICAgICAgICBzZWxmLmlw
ID0gc2VsZi5ydW5uZXIuaXANCiAgICAgMDE1MDogICAgICAgICAgICBzZWxmLnNlcnZlcl9pcCA9
IHNlbGYucnVubmVyLnNlcnZlcl9pcA0KICAgICAwMTUxOiAgICAgICAgICAgIHNlbGYuY29ubmVj
dGlvbiA9IFNTSENvbnRyb2woaXA9c2VsZi5pcCwgbG9nZmlsZT1zZWxmLnNzaGxvZykNCiAgICAg
MDE1MjogICAgICAgIGVsc2U6DQpGaWxlOiAnL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWls
ZGVyL3lvY3RvLXdvcmtlci9uaWdodGx5LXgzMi9idWlsZC9tZXRhL2xpYi9vZXFhL3V0aWxzL3Fl
bXVydW5uZXIucHknLCBsaW5lbm86IDEwNywgZnVuY3Rpb246IHN0YXJ0DQogICAgIDAxMDM6ICAg
ICAgICAgICAgYmIubm90ZSgicWVtdSBzdGFydGVkIC0gcWVtdSBwcm9jY2VzIHBpZCBpcyAlcyIg
JSBzZWxmLnFlbXVwaWQpDQogICAgIDAxMDQ6ICAgICAgICAgICAgY21kbGluZSA9ICcnDQogICAg
IDAxMDU6ICAgICAgICAgICAgd2l0aCBvcGVuKCcvcHJvYy8lcy9jbWRsaW5lJyAlIHNlbGYucWVt
dXBpZCkgYXMgcDoNCiAgICAgMDEwNjogICAgICAgICAgICAgICAgY21kbGluZSA9IHAucmVhZCgp
DQogKioqIDAxMDc6ICAgICAgICAgICAgaXBzID0gcmUuZmluZGFsbCgiKCg/OlswLTldezEsM31c
Lil7M31bMC05XXsxLDN9KSIsIGNtZGxpbmUuc3BsaXQoImlwPSIpWzFdKQ0KICAgICAwMTA4OiAg
ICAgICAgICAgIGlmIG5vdCBpcHMgb3IgbGVuKGlwcykgIT0gMzoNCiAgICAgMDEwOTogICAgICAg
ICAgICAgICAgYmIubm90ZSgiQ291bGRuJ3QgZ2V0IGlwIGZyb20gcWVtdSBwcm9jZXNzIGFyZ3Vt
ZW50cyEgSGVyZSBpcyB0aGUgcWVtdSBjb21tYW5kIGxpbmUgdXNlZDogJXMiICUgY21kbGluZSkN
CiAgICAgMDExMDogICAgICAgICAgICAgICAgc2VsZi5zdG9wKCkNCiAgICAgMDExMTogICAgICAg
ICAgICAgICAgcmV0dXJuIEZhbHNlDQpFeGNlcHRpb246IEluZGV4RXJyb3I6IGxpc3QgaW5kZXgg
b3V0IG9mIHJhbmdlDQoNCkVSUk9SOiBGdW5jdGlvbiBmYWlsZWQ6IGRvX3Rlc3RpbWFnZQ0KRVJS
T1I6IExvZ2ZpbGUgb2YgZmFpbHVyZSBzdG9yZWQgaW46IC9ob21lL3Bva3lidWlsZC95b2N0by1h
dXRvYnVpbGRlci95b2N0by13b3JrZXIvbmlnaHRseS14MzIvYnVpbGQvYnVpbGQvdG1wL3dvcmsv
cWVtdXg4Nl82NC1wb2t5LWxpbnV4LWdudXgzMi9jb3JlLWltYWdlLW1pbmltYWwvMS4wLXIwL3Rl
bXAvbG9nLmRvX3Rlc3RpbWFnZS4yNjk1OQ0KTk9URTogcmVjaXBlIGNvcmUtaW1hZ2UtbWluaW1h
bC0xLjAtcjA6IHRhc2sgZG9fdGVzdGltYWdlOiBGYWlsZWQNCkVSUk9SOiBUYXNrIDAgKC9ob21l
L3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRlci95b2N0by13b3JrZXIvbmlnaHRseS14MzIvYnVp
bGQvbWV0YS9yZWNpcGVzLWNvcmUvaW1hZ2VzL2NvcmUtaW1hZ2UtbWluaW1hbC5iYiwgZG9fdGVz
dGltYWdlKSBmYWlsZWQgd2l0aCBleGl0IGNvZGUgJzEnDQpOT1RFOiBUYXNrcyBTdW1tYXJ5OiBB
dHRlbXB0ZWQgMzAyIHRhc2tzIG9mIHdoaWNoIDMwMSBkaWRuJ3QgbmVlZCB0byBiZSByZXJ1biBh
bmQgMSBmYWlsZWQuDQpOT1RFOiBUaGUgZXJyb3JzIGZvciB0aGlzIGJ1aWxkIGFyZSBzdG9yZWQg
aW4gL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVyL3lvY3RvLXdvcmtlci9uaWdodGx5
LXgzMi9idWlsZC9idWlsZC90bXAvbG9nL2Vycm9yLXJlcG9ydC9lcnJvcl9yZXBvcnRfMjAxNTA4
MDUxODE1LnR4dA0KWW91IGNhbiBzZW5kIHRoZSBlcnJvcnMgdG8gYSByZXBvcnRzIHNlcnZlciBi
eSBydW5uaW5nOg0KICBzZW5kLWVycm9yLXJlcG9ydCAvaG9tZS9wb2t5YnVpbGQveW9jdG8tYXV0
b2J1aWxkZXIveW9jdG8td29ya2VyL25pZ2h0bHkteDMyL2J1aWxkL2J1aWxkL3RtcC9sb2cvZXJy
b3ItcmVwb3J0L2Vycm9yX3JlcG9ydF8yMDE1MDgwNTE4MTUudHh0IFstcyBzZXJ2ZXJdDQpOT1RF
OiBUaGUgY29udGVudHMgb2YgdGhlc2UgbG9ncyB3aWxsIGJlIHBvc3RlZCBpbiBwdWJsaWMgaWYg
eW91IHVzZSB0aGUgYWJvdmUgY29tbWFuZCB3aXRoIHRoZSBkZWZhdWx0IHNlcnZlci4gUGxlYXNl
IGVuc3VyZSB5b3UgcmVtb3ZlIGFueSBpZGVudGlmeWluZyBvciBwcm9wcmlldGFyeSBpbmZvcm1h
dGlvbiB3aGVuIHByb21wdGVkIGJlZm9yZSBzZW5kaW5nLg0KDQpTdW1tYXJ5OiAxIHRhc2sgZmFp
bGVkOg0KICAvaG9tZS9wb2t5YnVpbGQveW9jdG8tYXV0b2J1aWxkZXIveW9jdG8td29ya2VyL25p
Z2h0bHkteDMyL2J1aWxkL21ldGEvcmVjaXBlcy1jb3JlL2ltYWdlcy9jb3JlLWltYWdlLW1pbmlt
YWwuYmIsIGRvX3Rlc3RpbWFnZQ0KU3VtbWFyeTogVGhlcmUgd2FzIDEgV0FSTklORyBtZXNzYWdl
IHNob3duLg0KU3VtbWFyeTogVGhlcmUgd2VyZSAyIEVSUk9SIG1lc3NhZ2VzIHNob3duLCByZXR1
cm5pbmcgYSBub24temVybyBleGl0IGNvZGUu
</data>

          </attachment>
          <attachment
              isobsolete="0"
              ispatch="0"
              isprivate="0"
          >
            <attachid>2641</attachid>
            <date>2015-08-07 22:44:49 +0000</date>
            <delta_ts>2015-08-07 22:44:49 +0000</delta_ts>
            <desc>Results of failure running sanity tests for builder nightly-qa-skeleton build #424</desc>
            <filename>file_8129.txt</filename>
            <type>text/plain</type>
            <size>4706</size>
            <attacher name="Alex Franco">alejandro.franco</attacher>
            
              <data encoding="base64">IyMjIFNoZWxsIGVudmlyb25tZW50IHNldCB1cCBmb3IgYnVpbGRzLiAjIyMNCg0KWW91IGNhbiBu
b3cgcnVuICdiaXRiYWtlIDx0YXJnZXQ+Jw0KDQpDb21tb24gdGFyZ2V0cyBhcmU6DQogICAgY29y
ZS1pbWFnZS1taW5pbWFsDQogICAgY29yZS1pbWFnZS1zYXRvDQogICAgbWV0YS10b29sY2hhaW4N
CiAgICBhZHQtaW5zdGFsbGVyDQogICAgbWV0YS1pZGUtc3VwcG9ydA0KDQpZb3UgY2FuIGFsc28g
cnVuIGdlbmVyYXRlZCBxZW11IGltYWdlcyB3aXRoIGEgY29tbWFuZCBsaWtlICdydW5xZW11IHFl
bXV4ODYnDQpXaWxsIHVzZSBhIFZOQyBzZXJ2ZXIgYWxyZWFkeSBydW5uaW5nIG9uIDoxDQpXQVJO
SU5HOiBIb3N0IGRpc3RyaWJ1dGlvbiAiRmVkb3JhLTIyIiBoYXMgbm90IGJlZW4gdmFsaWRhdGVk
IHdpdGggdGhpcyB2ZXJzaW9uIG9mIHRoZSBidWlsZCBzeXN0ZW07IHlvdSBtYXkgcG9zc2libHkg
ZXhwZXJpZW5jZSB1bmV4cGVjdGVkIGZhaWx1cmVzLiBJdCBpcyByZWNvbW1lbmRlZCB0aGF0IHlv
dSB1c2UgYSB0ZXN0ZWQgZGlzdHJpYnV0aW9uLg0KUGFyc2luZyByZWNpcGVzLi4uZG9uZS4NClBh
cnNpbmcgb2YgODg5IC5iYiBmaWxlcyBjb21wbGV0ZSAoMCBjYWNoZWQsIDg4OSBwYXJzZWQpLiAx
MzAyIHRhcmdldHMsIDQwIHNraXBwZWQsIDAgbWFza2VkLCAwIGVycm9ycy4NCk5PVEU6IFJlc29s
dmluZyBhbnkgbWlzc2luZyB0YXNrIHF1ZXVlIGRlcGVuZGVuY2llcw0KDQpCdWlsZCBDb25maWd1
cmF0aW9uOg0KQkJfVkVSU0lPTiAgICAgICAgPSAiMS4yNy4xIg0KQlVJTERfU1lTICAgICAgICAg
PSAieDg2XzY0LWxpbnV4Ig0KTkFUSVZFTFNCU1RSSU5HICAgPSAiRmVkb3JhLTIyIg0KVEFSR0VU
X1NZUyAgICAgICAgPSAieDg2XzY0LXBva3ktbGludXgiDQpNQUNISU5FICAgICAgICAgICA9ICJx
ZW11eDg2LTY0Ig0KRElTVFJPICAgICAgICAgICAgPSAicG9reSINCkRJU1RST19WRVJTSU9OICAg
ID0gIjEuOCtzbmFwc2hvdC0yMDE1MDgwNyINClRVTkVfRkVBVFVSRVMgICAgID0gIm02NCBjb3Jl
MiINClRBUkdFVF9GUFUgICAgICAgID0gIiINCm1ldGEgICAgICAgICAgICAgIA0KbWV0YS15b2N0
byAgICAgICAgDQptZXRhLXlvY3RvLWJzcCAgICANCm1ldGEtc2tlbGV0b24gICAgID0gIm1hc3Rl
ci1uZXh0OjAwNWE1YjYwZjAwYTg5OGI3MDFkZTZkZjcwOTVhZGY3NDQwYWNlYmYiDQoNCk5PVEU6
IFByZXBhcmluZyBSdW5RdWV1ZQ0KTk9URTogRXhlY3V0aW5nIFNldFNjZW5lIFRhc2tzDQpOT1RF
OiBFeGVjdXRpbmcgUnVuUXVldWUgVGFza3MNCk5PVEU6IFJ1bm5pbmcgdGFzayAzMTYgb2YgMzE2
IChJRDogMCwgL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVyL3lvY3RvLXdvcmtlci9u
aWdodGx5LXFhLXNrZWxldG9uL2J1aWxkL21ldGEvcmVjaXBlcy1zYXRvL2ltYWdlcy9jb3JlLWlt
YWdlLXNhdG8uYmIsIGRvX3Rlc3RpbWFnZSkNCk5PVEU6IHJlY2lwZSBjb3JlLWltYWdlLXNhdG8t
MS4wLXIwOiB0YXNrIGRvX3Rlc3RpbWFnZTogU3RhcnRlZA0KRVJST1I6IEZ1bmN0aW9uIGZhaWxl
ZDogY29yZS1pbWFnZS1zYXRvIC0gRkFJTEVEIHRvIHN0YXJ0IHFlbXUgLSBjaGVjayB0aGUgdGFz
ayBsb2cgYW5kIHRoZSBib290IGxvZw0KRVJST1I6IExvZ2ZpbGUgb2YgZmFpbHVyZSBzdG9yZWQg
aW46IC9ob21lL3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRlci95b2N0by13b3JrZXIvbmlnaHRs
eS1xYS1za2VsZXRvbi9idWlsZC9idWlsZC90bXAvd29yay9xZW11eDg2XzY0LXBva3ktbGludXgv
Y29yZS1pbWFnZS1zYXRvLzEuMC1yMC90ZW1wL2xvZy5kb190ZXN0aW1hZ2UuMTc2NA0KTG9nIGRh
dGEgZm9sbG93czoNCnwgREVCVUc6IEV4ZWN1dGluZyBweXRob24gZnVuY3Rpb24gZG9fdGVzdGlt
YWdlDQp8IE5PVEU6IHJvb3RmcyBmaWxlOiAvaG9tZS9wb2t5YnVpbGQveW9jdG8tYXV0b2J1aWxk
ZXIveW9jdG8td29ya2VyL25pZ2h0bHktcWEtc2tlbGV0b24vYnVpbGQvYnVpbGQvdG1wL3dvcmsv
cWVtdXg4Nl82NC1wb2t5LWxpbnV4L2NvcmUtaW1hZ2Utc2F0by8xLjAtcjAvdGVzdGltYWdlL2Nv
cmUtaW1hZ2Utc2F0by1xZW11eDg2LTY0LXRlc3RpbWFnZS5leHQ0DQp8IE5PVEU6IFFlbXUgbG9n
IGZpbGU6IC9ob21lL3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRlci95b2N0by13b3JrZXIvbmln
aHRseS1xYS1za2VsZXRvbi9idWlsZC9idWlsZC90bXAvd29yay9xZW11eDg2XzY0LXBva3ktbGlu
dXgvY29yZS1pbWFnZS1zYXRvLzEuMC1yMC90ZXN0aW1hZ2UvcWVtdV9ib290X2xvZy4yMDE1MDgw
NzE4MDgzNw0KfCBOT1RFOiBTU0ggbG9nIGZpbGU6IC9ob21lL3Bva3lidWlsZC95b2N0by1hdXRv
YnVpbGRlci95b2N0by13b3JrZXIvbmlnaHRseS1xYS1za2VsZXRvbi9idWlsZC9idWlsZC90bXAv
d29yay9xZW11eDg2XzY0LXBva3ktbGludXgvY29yZS1pbWFnZS1zYXRvLzEuMC1yMC90ZXN0aW1h
Z2Uvc3NoX3RhcmdldF9sb2cuMjAxNTA4MDcxODA4MzcNCnwgTk9URTogQ3JlYXRlZCBsaXN0ZW5p
bmcgc29ja2V0IGZvciBxZW11IHNlcmlhbCBjb25zb2xlIG9uOiAxMjcuMC4wLjE6NTc2NzMNCnwg
Tk9URTogcnVucWVtdSBzdGFydGVkLCBwaWQgaXMgMTc2Nw0KfCBOT1RFOiB3YWl0aW5nIGF0IG1v
c3QgNjAgc2Vjb25kcyBmb3IgcWVtdSBwaWQNCnwgTk9URTogcWVtdSBzdGFydGVkIC0gcWVtdSBw
cm9jY2VzIHBpZCBpcyAxODQ2DQp8IE5PVEU6IENvdWxkbid0IGdldCBpcCBmcm9tIHFlbXUgcHJv
Y2VzcyBhcmd1bWVudHMhIEhlcmUgaXMgdGhlIHFlbXUgY29tbWFuZCBsaW5lIHVzZWQ6IC9ob21l
L3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRlci95b2N0by13b3JrZXIvbmlnaHRseS1xYS1za2Vs
ZXRvbi9idWlsZC9idWlsZC90bXAvc3lzcm9vdHMveDg2XzY0LWxpbnV4L3Vzci9iaW4vcWVtdS1z
eXN0ZW0teDg2XzY077+9LWtlcm5lbO+/vS9ob21lL3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRl
ci95b2N0by13b3JrZXIvbmlnaHRseS1xYS1za2VsZXRvbi9idWlsZC9idWlsZC90bXAvZGVwbG95
L2ltYWdlcy9xZW11eDg2LTY0L2J6SW1hZ2UtcWVtdXg4Ni02NC5iaW7vv70tY3B177+9Y29yZTJk
dW/vv70tdmdh77+9dm13YXJl77+9LW5vLXJlYm9vdO+/vS1zZXJpYWzvv710Y3A6MTI3LjAuMC4x
OjU3Njcz77+9LW3vv70yNTbvv70tLWFwcGVuZO+/vXZnYT0wIHV2ZXNhZmIubW9kZV9vcHRpb249
NjQweDQ4MC0zMiByb290PSBydyBtZW09MjU2TSAgb3Byb2ZpbGUudGltZXI9MSByb290ZnN0eXBl
PWV4dDQgIGNvbnNvbGU9dHR5MSBjb25zb2xlPXR0eVMwLDExNTIwMG4477+9DQp8IE5PVEU6IFNl
bmRpbmcgU0lHVEVSTSB0byBydW5xZW11DQp8IERFQlVHOiBQeXRob24gZnVuY3Rpb24gZG9fdGVz
dGltYWdlIGZpbmlzaGVkDQp8IEVSUk9SOiBGdW5jdGlvbiBmYWlsZWQ6IGNvcmUtaW1hZ2Utc2F0
byAtIEZBSUxFRCB0byBzdGFydCBxZW11IC0gY2hlY2sgdGhlIHRhc2sgbG9nIGFuZCB0aGUgYm9v
dCBsb2cNCk5PVEU6IHJlY2lwZSBjb3JlLWltYWdlLXNhdG8tMS4wLXIwOiB0YXNrIGRvX3Rlc3Rp
bWFnZTogRmFpbGVkDQpFUlJPUjogVGFzayAwICgvaG9tZS9wb2t5YnVpbGQveW9jdG8tYXV0b2J1
aWxkZXIveW9jdG8td29ya2VyL25pZ2h0bHktcWEtc2tlbGV0b24vYnVpbGQvbWV0YS9yZWNpcGVz
LXNhdG8vaW1hZ2VzL2NvcmUtaW1hZ2Utc2F0by5iYiwgZG9fdGVzdGltYWdlKSBmYWlsZWQgd2l0
aCBleGl0IGNvZGUgJzEnDQpOT1RFOiBUYXNrcyBTdW1tYXJ5OiBBdHRlbXB0ZWQgMzE2IHRhc2tz
IG9mIHdoaWNoIDMxNSBkaWRuJ3QgbmVlZCB0byBiZSByZXJ1biBhbmQgMSBmYWlsZWQuDQpOT1RF
OiBUaGUgZXJyb3JzIGZvciB0aGlzIGJ1aWxkIGFyZSBzdG9yZWQgaW4gL2hvbWUvcG9reWJ1aWxk
L3lvY3RvLWF1dG9idWlsZGVyL3lvY3RvLXdvcmtlci9uaWdodGx5LXFhLXNrZWxldG9uL2J1aWxk
L2J1aWxkL3RtcC9sb2cvZXJyb3ItcmVwb3J0L2Vycm9yX3JlcG9ydF8yMDE1MDgwNzE4MDgzNy50
eHQNCllvdSBjYW4gc2VuZCB0aGUgZXJyb3JzIHRvIGEgcmVwb3J0cyBzZXJ2ZXIgYnkgcnVubmlu
ZzoNCiAgc2VuZC1lcnJvci1yZXBvcnQgL2hvbWUvcG9reWJ1aWxkL3lvY3RvLWF1dG9idWlsZGVy
L3lvY3RvLXdvcmtlci9uaWdodGx5LXFhLXNrZWxldG9uL2J1aWxkL2J1aWxkL3RtcC9sb2cvZXJy
b3ItcmVwb3J0L2Vycm9yX3JlcG9ydF8yMDE1MDgwNzE4MDgzNy50eHQgWy1zIHNlcnZlcl0NCk5P
VEU6IFRoZSBjb250ZW50cyBvZiB0aGVzZSBsb2dzIHdpbGwgYmUgcG9zdGVkIGluIHB1YmxpYyBp
ZiB5b3UgdXNlIHRoZSBhYm92ZSBjb21tYW5kIHdpdGggdGhlIGRlZmF1bHQgc2VydmVyLiBQbGVh
c2UgZW5zdXJlIHlvdSByZW1vdmUgYW55IGlkZW50aWZ5aW5nIG9yIHByb3ByaWV0YXJ5IGluZm9y
bWF0aW9uIHdoZW4gcHJvbXB0ZWQgYmVmb3JlIHNlbmRpbmcuDQoNClN1bW1hcnk6IDEgdGFzayBm
YWlsZWQ6DQogIC9ob21lL3Bva3lidWlsZC95b2N0by1hdXRvYnVpbGRlci95b2N0by13b3JrZXIv
bmlnaHRseS1xYS1za2VsZXRvbi9idWlsZC9tZXRhL3JlY2lwZXMtc2F0by9pbWFnZXMvY29yZS1p
bWFnZS1zYXRvLmJiLCBkb190ZXN0aW1hZ2UNClN1bW1hcnk6IFRoZXJlIHdhcyAxIFdBUk5JTkcg
bWVzc2FnZSBzaG93bi4NClN1bW1hcnk6IFRoZXJlIHdhcyAxIEVSUk9SIG1lc3NhZ2Ugc2hvd24s
IHJldHVybmluZyBhIG5vbi16ZXJvIGV4aXQgY29kZS4=
</data>

          </attachment>
      

    </bug>

</bugzilla>