Bug 10974 - Tests for Build Appliance need to be automated
Summary: Tests for Build Appliance need to be automated
Status: RESOLVED DUPLICATE of bug 16016
Alias: None
Product: Build Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 5.99
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2017-01-26 16:45 UTC by brian avery
Modified: 2025-10-23 15:14 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)


Attachments
ba-auto-test.sh (3.39 KB, application/x-shellscript)
2017-06-08 02:27 UTC, Yi Zhao
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description brian avery 2017-01-26 16:45:59 UTC
The current tests for build appliance located in https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6552
need to be updated to remove HOB and need to be automated so that the Build Appliance can act as a dogfooding test environment.
Comment 1 Yi Zhao 2017-02-08 06:54:09 UTC
I have updated the cases in https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6552
Comment 2 Yi Zhao 2017-02-14 02:31:24 UTC
Hi Brain,
I have no idea about how to automated these cases. Do you mean we should automated all steps include start a vmplayer to launch BA, login and perform the test cases or just automated the test cases in a launched BA ?
Comment 3 Yi Zhao 2017-02-21 02:42:44 UTC
Hi Brain,
I have no idea about how to automated these cases. Do you mean we should automated all steps include start a vmplayer to launch BA, login and perform the test cases or just automated the test cases in a launched BA ?
Comment 4 brian avery 2017-02-28 19:06:18 UTC
Ideally, this would be a script that launched the player/VirtualBox, with the BA as a target.  Juro suggests 8 gb mem and 8 cpus.  Caveat: Vmplayer fails for Juro on Linux. 

There is a networking readme in the BA docs, to explain how to set up the BA in nat mode with port forwarding which would be deterministic.  This will also help if we decide to test toaster in the 2.4 timeframe on BA.

Then I'd use a script to ssh in and run the tests like our other tests (testimage for instance) run.  Alternatively, we could just build core-image-minimal and run the testimage itself on the BA. That would be fairly easy and more complete than the current test.

Fair Warning:
Current BA is broken due to a bash etc/alternative move issue.  This approach could be easily fleshed out against morty and moved to master once the BA is back working.

--
Juro said to ask if you hit roadblocks.
Comment 5 Stephen K Jolley 2017-03-13 20:15:05 UTC
I believe the question is answered
Comment 6 Yi Zhao 2017-06-08 02:27:44 UTC
Created attachment 3805 [details]
ba-auto-test.sh

The attachment is a script for ba testing automation.
The script does the following things:
1. Connect the BA VM and set the build environment.
2. Choose a qemu arch randomly and build a minimal image.
2. Run self test for this image.
Comment 7 Yi Zhao 2017-06-08 02:29:29 UTC
The script works well on Yocto 2.3 BA
Comment 8 Joshua Lock 2017-06-08 10:51:36 UTC
The script is a great start but it's not enough to mark this issue resolved. We need to ensure that test is used as part of our automated testing so that we can detect when the build appliance is broken.
Comment 9 Yi Zhao 2017-06-13 05:58:12 UTC
(In reply to comment #8)
> The script is a great start but it's not enough to mark this issue resolved.
> We need to ensure that test is used as part of our automated testing so that
> we can detect when the build appliance is broken.

Thanks Joshua. I will do more work with it.
Comment 10 Libertad 2017-12-06 18:15:45 UTC
QA Penang needs to analyse this from the beginning, with Yi's help.
Comment 11 yeoh ee peng 2020-03-17 08:36:14 UTC
QA team had created the manual build appliance test cases and shared the build appliance testing information to the community. Also QA team had made the decision to discontinue build appliance manual testing given limited resource available. QA team currently looking for community to provide build appliance automation if it was still needed.   
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oeqa/manual/build-appliance.json
Comment 12 Randy MacLeod 2020-03-19 14:58:24 UTC
Still needs to be done.
Comment 13 Randy MacLeod 2023-10-24 14:24:28 UTC
Bulk move from 4.99 or 0.00 to 5.99
Comment 14 Ross Burton 2025-10-23 15:14:48 UTC
Marking as a duplicate (in the future) of 16016 as BA is rarely used now.

*** This bug has been marked as a duplicate of bug 16016 ***