Bug 10636 - Improved image testing by extending automated QA tests
Summary: Improved image testing by extending automated QA tests
Status: NEW
Alias: None
Product: Runtime Testing
Classification: QA/Testing
Component: general (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 5.99
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-11-09 15:15 UTC by Ross Burton
Modified: 2025-02-06 16:21 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Burton 2016-11-09 15:15:00 UTC
We need to improve our automated image QA, for example checking that /run is mounted and read/write even if / is read-only.  Assigning to Jussi for further exploratory research and scoping.
Comment 1 Jussi Kukkonen 2016-11-17 10:10:25 UTC
I've looked at the infrastructure and it seems everything needed is there already: this bug is just about defining the things to test for and writing the tests -- this seems fairly straight forward so far. 

I expect there to be two types of tests:
1. file based rootfs tests: See that a specific files exists on rootfs and have the content and permissions we expect
2. runtime tests: same but for files that are only available at runtime

Type 1 could be tested with just the rootfs temporarily mounted somewhere, while type 2 requires starting qemu: need to check if having two separate types is useful or if just doing everything over qemu works fine. 


I've not really scoped out the specific tests yet, but some quick ideas:
 - is "/" read-only when expected? (maybe check a few directories under root?)
 - is "/run" mounted?
 - is "/run" read-write?
 - is "/var/volatile" read-write ?
 - permissions sanity checks for key directories/files?


Related: do we make sure that packages don't install things to /var/run  or /run ? I know I saw people making this mistake in fedora and debian packages a few years ago...
Comment 2 Benjamin Esquivel 2016-11-17 15:49:36 UTC
quick question, is this a bug that will require QA to run these tests regularly? I assume yes so I'd add them to the loop. Pending confirmation.
Comment 3 Jussi Kukkonen 2016-11-17 15:53:50 UTC
(In reply to comment #2)
> quick question, is this a bug that will require QA to run these tests
> regularly? I assume yes so I'd add them to the loop. Pending confirmation.

These would be part of the sanity tests that run on the autobuilder. QA work should not be required.
Comment 4 Ross Burton 2017-09-11 16:18:12 UTC
Reassigning Jussi's bugs.
Comment 5 Randy MacLeod 2025-02-06 16:21:11 UTC
We need to define what these tests should be, then it can go back to a NEWCOMER.