<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>10974</bug_id>
          
          <creation_ts>2017-01-26 16:45:59 +0000</creation_ts>
          <short_desc>Tests for Build Appliance need to be automated</short_desc>
          <delta_ts>2025-10-23 15:14:48 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>10</classification_id>
          <classification>QA/Testing</classification>
          <product>Build Testing</product>
          <component>general</component>
          <version>unspecified</version>
          <rep_platform>x86</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>DUPLICATE</resolution>
          <dup_id>16016</dup_id>
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard> </status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>enhancement</bug_severity>
          <target_milestone>5.99</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="brian avery">brian.avery</reporter>
          <assigned_to name="Unassigned">unassigned</assigned_to>
          <cc>benjamin.esquivel</cc>
    
    <cc>jose.perez.carranza</cc>
    
    <cc>joshuagloe</cc>
    
    <cc>juro.bystricky</cc>
    
    <cc>laurentiu.serban</cc>
    
    <cc>randy.macleod</cc>
    
    <cc>ross.burton</cc>
    
    <cc>yi.zhao</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>Yes (doc changes required)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>70191</commentid>
    <comment_count>0</comment_count>
    <who name="brian avery">brian.avery</who>
    <bug_when>2017-01-26 16:45:59 +0000</bug_when>
    <thetext>The current tests for build appliance located in https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6552
need to be updated to remove HOB and need to be automated so that the Build Appliance can act as a dogfooding test environment.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>70529</commentid>
    <comment_count>1</comment_count>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-02-08 06:54:09 +0000</bug_when>
    <thetext>I have updated the cases in https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6552</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>70676</commentid>
    <comment_count>2</comment_count>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-02-14 02:31:24 +0000</bug_when>
    <thetext>Hi Brain,
I have no idea about how to automated these cases. Do you mean we should automated all steps include start a vmplayer to launch BA, login and perform the test cases or just automated the test cases in a launched BA ?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>70847</commentid>
    <comment_count>3</comment_count>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-02-21 02:42:44 +0000</bug_when>
    <thetext>Hi Brain,
I have no idea about how to automated these cases. Do you mean we should automated all steps include start a vmplayer to launch BA, login and perform the test cases or just automated the test cases in a launched BA ?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>70995</commentid>
    <comment_count>4</comment_count>
    <who name="brian avery">brian.avery</who>
    <bug_when>2017-02-28 19:06:18 +0000</bug_when>
    <thetext>Ideally, this would be a script that launched the player/VirtualBox, with the BA as a target.  Juro suggests 8 gb mem and 8 cpus.  Caveat: Vmplayer fails for Juro on Linux. 

There is a networking readme in the BA docs, to explain how to set up the BA in nat mode with port forwarding which would be deterministic.  This will also help if we decide to test toaster in the 2.4 timeframe on BA.

Then I&apos;d use a script to ssh in and run the tests like our other tests (testimage for instance) run.  Alternatively, we could just build core-image-minimal and run the testimage itself on the BA. That would be fairly easy and more complete than the current test.

Fair Warning:
Current BA is broken due to a bash etc/alternative move issue.  This approach could be easily fleshed out against morty and moved to master once the BA is back working.

--
Juro said to ask if you hit roadblocks.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>71218</commentid>
    <comment_count>5</comment_count>
    <who name="Stephen K Jolley">sjolley.yp.pm</who>
    <bug_when>2017-03-13 20:15:05 +0000</bug_when>
    <thetext>I believe the question is answered</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>73855</commentid>
    <comment_count>6</comment_count>
      <attachid>3805</attachid>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-06-08 02:27:44 +0000</bug_when>
    <thetext>Created attachment 3805
ba-auto-test.sh

The attachment is a script for ba testing automation.
The script does the following things:
1. Connect the BA VM and set the build environment.
2. Choose a qemu arch randomly and build a minimal image.
2. Run self test for this image.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>73856</commentid>
    <comment_count>7</comment_count>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-06-08 02:29:29 +0000</bug_when>
    <thetext>The script works well on Yocto 2.3 BA</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>73870</commentid>
    <comment_count>8</comment_count>
    <who name="Joshua Lock">joshuagloe</who>
    <bug_when>2017-06-08 10:51:36 +0000</bug_when>
    <thetext>The script is a great start but it&apos;s not enough to mark this issue resolved. We need to ensure that test is used as part of our automated testing so that we can detect when the build appliance is broken.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>74036</commentid>
    <comment_count>9</comment_count>
    <who name="Yi Zhao">yi.zhao</who>
    <bug_when>2017-06-13 05:58:12 +0000</bug_when>
    <thetext>(In reply to comment #8)
&gt; The script is a great start but it&apos;s not enough to mark this issue resolved.
&gt; We need to ensure that test is used as part of our automated testing so that
&gt; we can detect when the build appliance is broken.

Thanks Joshua. I will do more work with it.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>78553</commentid>
    <comment_count>10</comment_count>
    <who name="Libertad">libertad.gonzalez.de.la.cruz</who>
    <bug_when>2017-12-06 18:15:45 +0000</bug_when>
    <thetext>QA Penang needs to analyse this from the beginning, with Yi&apos;s help.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86736</commentid>
    <comment_count>11</comment_count>
    <who name="yeoh ee peng">ee.peng.yeoh</who>
    <bug_when>2020-03-17 08:36:14 +0000</bug_when>
    <thetext>QA team had created the manual build appliance test cases and shared the build appliance testing information to the community. Also QA team had made the decision to discontinue build appliance manual testing given limited resource available. QA team currently looking for community to provide build appliance automation if it was still needed.   
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual/build-appliance.json</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86757</commentid>
    <comment_count>12</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2020-03-19 14:58:24 +0000</bug_when>
    <thetext>Still needs to be done.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>96842</commentid>
    <comment_count>13</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2023-10-24 14:24:28 +0000</bug_when>
    <thetext>Bulk move from 4.99 or 0.00 to 5.99</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>103052</commentid>
    <comment_count>14</comment_count>
    <who name="Ross Burton">ross.burton</who>
    <bug_when>2025-10-23 15:14:48 +0000</bug_when>
    <thetext>Marking as a duplicate (in the future) of 16016 as BA is rarely used now.

*** This bug has been marked as a duplicate of bug 16016 ***</thetext>
  </long_desc>
      
          <attachment
              isobsolete="0"
              ispatch="0"
              isprivate="0"
          >
            <attachid>3805</attachid>
            <date>2017-06-08 02:27:44 +0000</date>
            <delta_ts>2017-06-08 02:27:44 +0000</delta_ts>
            <desc>ba-auto-test.sh</desc>
            <filename>ba-auto-test.sh</filename>
            <type>application/x-shellscript</type>
            <size>3476</size>
            <attacher name="Yi Zhao">yi.zhao</attacher>
            
              <data encoding="base64">IyEvYmluL2Jhc2gKCgp1c2FnZSgpCnsKCWVjaG8gIiQoYmFzZW5hbWUgJDApIDx0YXJnZXQgaXAg
YWRkcmVzcz4iCn0KCiMgQ2hlY2sgaWYgaXAgYWRkcmVzcyBpcyB2YWxpZApjaGVja19pcGFkZHIo
KQp7CgllY2hvICQxIHwgZ3JlcCAiXlswLTldXHsxLDNcfVwuXChbMC05XVx7MSwzXH1cLlwpXHsy
XH1bMC05XVx7MSwzXH0kIiA+IC9kZXYvbnVsbDsKCWlmIFsgJD8gLW5lIDAgXTsgdGhlbgoJCXJl
dHVybiAxCglmaQoKCWlwYWRkcj0kMQoJYT1gZWNobyAkaXBhZGRyfGF3ayAtRiAuICd7cHJpbnQg
JDF9J2AKCWI9YGVjaG8gJGlwYWRkcnxhd2sgLUYgLiAne3ByaW50ICQyfSdgCgljPWBlY2hvICRp
cGFkZHJ8YXdrIC1GIC4gJ3twcmludCAkM30nYAoJZD1gZWNobyAkaXBhZGRyfGF3ayAtRiAuICd7
cHJpbnQgJDR9J2AKCWZvciBudW0gaW4gJGEgJGIgJGMgJGQKCWRvCgkJaWYgWyAkbnVtIC1ndCAy
NTUgXSB8fCBbICRudW0gLWx0IDAgXQoJCXRoZW4KCQkJcmV0dXJuIDEKCQlmaQoJZG9uZQoKCXJl
dHVybiAwCn0KCiMgQ2hlY2sgaWYgdGFyZ2V0IG5ldHdvcmsgaXMgdXAKY2hlY2tfbmV0d29ya191
cCgpCnsKCXBpbmcgLWMxICQxID4gL2Rldi9udWxsIDI+JjEKCWlmIFsgJD8gLW5lIDAgXTsgdGhl
bgoJCWVjaG8gIlBpbmcgdGFyZ2V0IGlwICQxIGZhaWxlZCIKCQlyZXR1cm4gMQoJZWxzZQoJCWVj
aG8gIlBpbmcgdGFyZ2V0IGlwICQxIHN1Y2Nlc3NmdWxseSIKCQlyZXR1cm4gMAoJZmkKfQoKIyBS
dW4gY29tbWFuZCBvbiB0YXJnZXQgdmlhIHNzaApydW5fc3NoKCkKewoJbG9jYWwgdW5hbWU9IiQx
IgoJbG9jYWwgdXBhc3M9IiQyIgoJbG9jYWwgaXBfYWRkcj0iJDMiCglsb2NhbCBydW5fY29tbWFu
ZD0iJDQiCgoJaWYgWyAkIyAtZXEgNSBdOyB0aGVuCgkJbG9jYWwgdGltZV9vdXQ9JDUKCWVsc2UK
CQlsb2NhbCB0aW1lX291dD0yODgwMAoJZmkKCglsb2NhbCByZXQ9MAoJbG9jYWwgZXhwX2NtZD1g
Y2F0IDw8IEVPRgpldmFsIHNwYXduIHNzaCAtbyBVc2VyS25vd25Ib3N0c0ZpbGU9L2Rldi9udWxs
IC1vIFN0cmljdEhvc3RLZXlDaGVja2luZz1ubyAke3VuYW1lfUAke2lwX2FkZHJ9ICIkcnVuX2Nv
bW1hbmQiCnNldCB0aW1lb3V0ICR0aW1lX291dApleHBlY3QgewogICAgIiphc3N3b3JkOiIgeyBz
ZW5kICIke3VwYXNzfVxyIjsgZXhwX2NvbnRpbnVlfQogICAgIiooeWVzL25vKT8iIHsgc2VuZCAi
eWVzXHIiOyBleHBfY29udGludWUgfQogICAgZW9mICAgICAgICAgIHsgZXhpdCBbIGxpbmRleCBb
d2FpdF0gMyBdIH0KfQpFT0ZgCgoJJEVYUEVDVCAtYyAiJGV4cF9jbWQiCglyZXQ9JD8KCXJldHVy
biAkcmV0Cn0KCiMgQ2hlY2sgaWYgc3NoIGlzIHVwIG9uIHRhcmdldApjaGVja19zc2hfdXAoKQp7
Cglsb2NhbCBpcF9hZGRyPSQxCglsb2NhbCB1bmFtZT0icm9vdCIKCWxvY2FsIHVwYXNzPSJyb290
IgoJbG9jYWwgdGltZW91dD02MAoJbG9jYWwgaW50ZXJ2YWw9MAoKCXdoaWxlIFsgJHtpbnRlcnZh
bH0gLWx0ICR7dGltZW91dH0gXQoJZG8KCQlydW5fc3NoICR1bmFtZSAkdXBhc3MgJGlwX2FkZHIg
Imhvc3RuYW1lIiAkdGltZW91dAoJCWlmIFsgJD8gLW5lIDAgXTsgdGhlbgoJCQlpbnRlcnZhbD1g
ZXhwciAkaW50ZXJ2YWwgKyAxMGAKCQkJc2xlZXAgMTAKCQllbHNlCgkJCWVjaG8gIlNTSCBvbiB0
YXJnZXQgJHtpcF9hZGRyfSB3aXRoaW4gJHtpbnRlcnZhbH0gc2Vjb25kcyIKCQkJcmV0dXJuIDAK
CQlmaQoJZG9uZQoKCWVjaG8gIkNhbiBub3QgU1NIIG9uIHRhcmdldCAke2lwX2FkZHJ9IGluICR7
dGltZW91dH0gc2Vjb25kcyIKCXJldHVybiAxCn0KCgppZiBbICQjIC1uZSAxIF07IHRoZW4KCXVz
YWdlCglleGl0IDEKZmkKCkVYUEVDVD1gd2hpY2ggZXhwZWN0YAppZiBbICEgLXggIiRFWFBFQ1Qi
IF07IHRoZW4KCWVjaG8gIiRFWFBFQ1QgY2Fubm90IGJlIGV4ZWN1dGVkIgoJZXhpdCAxCmZpCgp0
YXJnZXRfaXBhZGRyPSIkMSIKdXNlcl9uYW1lPSJidWlsZGVyIgp1c2VyX3Bhc3N3b3JkPSJidWls
ZGVyIgoKCmNoZWNrX2lwYWRkciAkdGFyZ2V0X2lwYWRkcgppZiBbICQ/IC1uZSAwIF07IHRoZW4K
CWVjaG8gIldyb25nIGlwIGFkZHJlc3MiCglleGl0IDEKZmkKCmNoZWNrX25ldHdvcmtfdXAgJHRh
cmdldF9pcGFkZHIKaWYgWyAkPyAtbmUgMCBdOyB0aGVuCgllY2hvICJUYXJnZXQgbmV0d29yayBp
cyBOT1QgcmVhY2hhYmxlIgoJZXhpdCAxCmZpCgpjaGVja19zc2hfdXAgJHRhcmdldF9pcGFkZHIK
aWYgWyAkPyAtbmUgMCBdOyB0aGVuCgllY2hvICJTU0ggb24gJHRhcmdldF9pcGFkZHIgaXMgTk9U
IHVwIgoJZXhpdCAxCmVsc2UKCWVjaG8gIlNTSCBvbiAkdGFyZ2V0X2lwYWRkciBpcyB1cCIKZmkK
CgplY2hvICJTdGFyIHRvIHRlc3QgLi4uIgoKdGVzdF9tYWNoaW5lPSQoZWNobyAicWVtdXg4NiBx
ZW11eDg2LTY0IHFlbXVhcm0gcWVtdWFybTY0IHFlbXVtaXBzIHFlbXVtaXBzNjQgcWVtdXBwYyAk
UkFORE9NIiB8IGF3ayAne3ByaW50ICQoJE5GJShORi0xKSsxKX0nKQoKdXBkYXRlX21hY2hpbmU9
InsgY2QgL2hvbWUvYnVpbGRlci9wb2t5L2J1aWxkL2NvbmY7IGVjaG8gXFxcIk1BQ0hJTkUgPSBc
XFxcXFxcIiR7dGVzdF9tYWNoaW5lfVxcXFxcXFwiXFxcIiA+PiBhdXRvLmNvbmYgfSIKCmVuYWJs
ZV9pbWFnZXRlc3Q9InsgY2QgL2hvbWUvYnVpbGRlci9wb2t5L2J1aWxkL2NvbmY7IGVjaG8gXFxc
IlRFU1RfSU1BR0UgPSBcXFxcXFxcIjFcXFxcXFxcIlxcXCIgPj4gYXV0by5jb25mIH0iCgpidWls
ZF9jb21tYW5kPSJ7IGV4cG9ydCBMQU5HPWVuX1VTLlVURi04OyBleHBvcnQgTENfQUxMPWVuX1VT
LlVURi04OyBjZCAvaG9tZS9idWlsZGVyL3Bva3k7IC4gLi9vZS1pbml0LWJ1aWxkLWVudjsgYml0
YmFrZSBjb3JlLWltYWdlLW1pbmltYWwgfSIKCgpydW5fc3NoICR1c2VyX25hbWUgJHVzZXJfcGFz
c3dvcmQgJHRhcmdldF9pcGFkZHIgIiR1cGRhdGVfbWFjaGluZSIKaWYgWyAkPyAtbmUgMCBdOyB0
aGVuCgllY2hvICJVcGRhdGUgY29uZmlndXJlICh1cGRhdGUgbWFjaGluZSkgZmFpbGVkIgplbHNl
CgllY2hvICJVcGRhdGUgY29uZmlndXJlICh1cGRhdGUgbWFjaGluZSkgc3VjY2Vzc2Z1bGx5Igpm
aQoKcnVuX3NzaCAkdXNlcl9uYW1lICR1c2VyX3Bhc3N3b3JkICR0YXJnZXRfaXBhZGRyICIkZW5h
YmxlX2ltYWdldGVzdCIKaWYgWyAkPyAtbmUgMCBdOyB0aGVuCgllY2hvICJVcGRhdGUgY29uZmln
dXJlIChlbmFibGUgaW1hZ2UgdGVzdCkgZmFpbGVkIgplbHNlCgllY2hvICJVcGRhdGUgY29uZmln
dXJlIChlbmFibGUgaW1hZ2UgdGVzdCkgc3VjY2Vzc2Z1bGx5IgpmaQoKcnVuX3NzaCAkdXNlcl9u
YW1lICR1c2VyX3Bhc3N3b3JkICR0YXJnZXRfaXBhZGRyICIkYnVpbGRfY29tbWFuZCIKaWYgWyAk
PyAtbmUgMCBdOyB0aGVuCgllY2hvICJCdWlsZCBhbmQgdGVzdCBmYWlsZWQiCmVsc2UKCWVjaG8g
IkJ1aWxkIGFuZCB0ZXN0IHN1Y2Nlc3NmdWxseSIKZmkKCmVjaG8gIlRlc3QgY29tcGxldGUiCgo=
</data>

          </attachment>
      

    </bug>

</bugzilla>