Bug 6068 - Create wic test suite
Summary: Create wic test suite
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: deployment (show other bugs)
Version: 1.9
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 1.9
Assignee: Ed Bartosh
QA Contact: Daniel Istrate
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-03-29 15:43 UTC by Tom Zanussi
Modified: 2015-09-15 13:16 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
.wks example with hardcoded rootfs dir (646 bytes, application/vnd.lotus-1-2-3)
2014-03-29 18:52 UTC, Tom Zanussi
no flags Details
.wks example with both partitions using indirect strings (583 bytes, application/vnd.lotus-1-2-3)
2014-03-29 18:53 UTC, Tom Zanussi
no flags Details
.wks example with one partition using indirect strings (559 bytes, application/vnd.lotus-1-2-3)
2014-03-29 18:54 UTC, Tom Zanussi
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Zanussi 2014-03-29 15:43:20 UTC
wic really needs a (preferably automated) formal testsuite to capture all the usages we expect it to handle, and so we can detect regressions.
Comment 1 Tom Zanussi 2014-03-29 18:52:13 UTC
Created attachment 1866 [details]
.wks example with hardcoded rootfs dir
Comment 2 Tom Zanussi 2014-03-29 18:53:49 UTC
Created attachment 1867 [details]
.wks example with both partitions using indirect strings
Comment 3 Tom Zanussi 2014-03-29 18:54:14 UTC
Created attachment 1868 [details]
.wks example with one partition using indirect strings
Comment 4 Tom Zanussi 2014-03-29 18:58:26 UTC
As a start, these are the commands tested for the multi-rootfs patches (corresponding .wks files attached):

$ wic create directdisk -e core-image-minimal

$ wic create directdisk -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi -e core-image-minimal

$ wic create directdisk-multi -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi-indirect-both --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ -e core-image-minimal

$ wic create directdisk-multi-indirect-both -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi-indirect-second -r /home/trz/yocto/master-cur/buil/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ -e core-image-minimal

$ wic create directdisk-multi-indirect-second -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/
Comment 5 Tom Zanussi 2014-05-08 14:49:24 UTC
And a couple more for the image rootfs option:

wic create directdisk-multi-indirect-both --rootfs-dir rootfs1=core-image-minimal --rootfs-dir rootfs2=core-image-minimal -e core-image-minimal

wic create directdisk-multi-indirect-both -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay-noemgd/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=core-image-minimal
Comment 6 Ed Bartosh 2015-04-29 15:21:36 UTC
Start working with QA team to add wic test cases to oe-selftest
Comment 7 Ed Bartosh 2015-06-26 13:15:08 UTC
Status update. 16 test cases have been created and put into production on AB. Here is how wic testing looks atm:
$ oe-selftest --run-tests wic
2015-06-26 16:06:06,272 - selftest - INFO - Checking that everything is in order before running the tests
2015-06-26 16:06:07,172 - selftest - INFO - Running bitbake -p
2015-06-26 16:06:09,719 - selftest - INFO - Loading tests from: oeqa.selftest.wic
2015-06-26 16:06:09,721 - selftest - INFO - Adding: "include selftest.inc" in local.conf
test01_help (oeqa.selftest.wic.Wic)
Test wic --help ... ok
test02_createhelp (oeqa.selftest.wic.Wic)
Test wic create --help ... ok
test03_listhelp (oeqa.selftest.wic.Wic)
Test wic list --help ... ok
test04_build_image_name (oeqa.selftest.wic.Wic)
Test wic create directdisk --image-name core-image-minimal ... ok
test05_build_artifacts (oeqa.selftest.wic.Wic)
Test wic create directdisk providing all artifacts. ... ok
test06_gpt_image (oeqa.selftest.wic.Wic)
Test creation of core-image-minimal with gpt table and UUID boot ... ok
test07_unsupported_subcommand (oeqa.selftest.wic.Wic)
Test unsupported subcommand ... ok
test08_no_command (oeqa.selftest.wic.Wic)
Test wic without command ... ok
test09_help_kickstart (oeqa.selftest.wic.Wic)
Test wic help overview ... ok
test10_help_plugins (oeqa.selftest.wic.Wic)
Test wic help plugins ... ok
test11_help_kickstart (oeqa.selftest.wic.Wic)
Test wic help kickstart ... ok
test12_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with gzip ... ok
test13_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with bzip2 ... ok
test14_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with xz ... ok
test15_wrong_compressor (oeqa.selftest.wic.Wic)
Test how wic breaks if wrong compressor is provided ... ok
test16_rootfs_indirect_recipes (oeqa.selftest.wic.Wic)
Test usage of rootfs plugin with 'rootfsN=<recipe>' syntax ... ok

----------------------------------------------------------------------
Ran 16 tests in 90.257s

OK
2015-06-26 16:07:39,979 - selftest - INFO - Finished
2015-06-26 16:07:39,979 - selftest - INFO - Removing the include from local.conf

I'm going to close this bug and I'll continue to add new test cases for new and old functionality.
Comment 8 Ed Bartosh 2015-06-29 08:24:48 UTC
15 test cases are in master now: http://cgit.openembedded.org/openembedded-core/commit/?id=0ffc5c38ef28605abf815a47eb41923802d31df7

2 more are waiting for review. I think it's enough to say that wic testing suite is created and put into production.
Comment 9 Daniel Istrate 2015-09-15 13:16:25 UTC
As of now in master: ea562de57590c966cd5a75fda8defecd397e6436 there are 23 test cases in meta/lib/oeqa/selftest/wic.py