Bug 6068

Summary: Create wic test suite
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Tom Zanussi <tom.zanussi>
Component: deploymentAssignee: Ed Bartosh <eduard.bartosh>
Status: VERIFIED FIXED QA Contact: Daniel Istrate <daniel.alexandrux.istrate>
Severity: enhancement    
Priority: Medium CC: alexandru.c.georgescu, bluelightning, bogdanx.a.voiculescu, jessica.zhang
Version: 1.9   
Target Milestone: 1.9   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
.wks example with hardcoded rootfs dir
none
.wks example with both partitions using indirect strings
none
.wks example with one partition using indirect strings none

Description Tom Zanussi 2014-03-29 15:43:20 UTC
wic really needs a (preferably automated) formal testsuite to capture all the usages we expect it to handle, and so we can detect regressions.
Comment 1 Tom Zanussi 2014-03-29 18:52:13 UTC
Created attachment 1866 [details]
.wks example with hardcoded rootfs dir
Comment 2 Tom Zanussi 2014-03-29 18:53:49 UTC
Created attachment 1867 [details]
.wks example with both partitions using indirect strings
Comment 3 Tom Zanussi 2014-03-29 18:54:14 UTC
Created attachment 1868 [details]
.wks example with one partition using indirect strings
Comment 4 Tom Zanussi 2014-03-29 18:58:26 UTC
As a start, these are the commands tested for the multi-rootfs patches (corresponding .wks files attached):

$ wic create directdisk -e core-image-minimal

$ wic create directdisk -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi -e core-image-minimal

$ wic create directdisk-multi -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi-indirect-both --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ -e core-image-minimal

$ wic create directdisk-multi-indirect-both -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/

$ wic create directdisk-multi-indirect-second -r /home/trz/yocto/master-cur/buil/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ -e core-image-minimal

$ wic create directdisk-multi-indirect-second -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux -r /home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=/home/trz/yocto/master-cur/build/tmp/work/crownbay-poky-linux/core-image-minimal/1.0-r0/rootfs/
Comment 5 Tom Zanussi 2014-05-08 14:49:24 UTC
And a couple more for the image rootfs option:

wic create directdisk-multi-indirect-both --rootfs-dir rootfs1=core-image-minimal --rootfs-dir rootfs2=core-image-minimal -e core-image-minimal

wic create directdisk-multi-indirect-both -b /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay-noemgd/usr/share -k /home/trz/yocto/master-cur/build/tmp/sysroots/crownbay-noemgd/usr/src/kernel -n /home/trz/yocto/master-cur/build/tmp/sysroots/x86_64-linux --rootfs-dir rootfs1=/home/trz/yocto/master-cur/build/tmp/work/crownbay_noemgd-poky-linux/core-image-minimal/1.0-r0/rootfs/ --rootfs-dir rootfs2=core-image-minimal
Comment 6 Ed Bartosh 2015-04-29 15:21:36 UTC
Start working with QA team to add wic test cases to oe-selftest
Comment 7 Ed Bartosh 2015-06-26 13:15:08 UTC
Status update. 16 test cases have been created and put into production on AB. Here is how wic testing looks atm:
$ oe-selftest --run-tests wic
2015-06-26 16:06:06,272 - selftest - INFO - Checking that everything is in order before running the tests
2015-06-26 16:06:07,172 - selftest - INFO - Running bitbake -p
2015-06-26 16:06:09,719 - selftest - INFO - Loading tests from: oeqa.selftest.wic
2015-06-26 16:06:09,721 - selftest - INFO - Adding: "include selftest.inc" in local.conf
test01_help (oeqa.selftest.wic.Wic)
Test wic --help ... ok
test02_createhelp (oeqa.selftest.wic.Wic)
Test wic create --help ... ok
test03_listhelp (oeqa.selftest.wic.Wic)
Test wic list --help ... ok
test04_build_image_name (oeqa.selftest.wic.Wic)
Test wic create directdisk --image-name core-image-minimal ... ok
test05_build_artifacts (oeqa.selftest.wic.Wic)
Test wic create directdisk providing all artifacts. ... ok
test06_gpt_image (oeqa.selftest.wic.Wic)
Test creation of core-image-minimal with gpt table and UUID boot ... ok
test07_unsupported_subcommand (oeqa.selftest.wic.Wic)
Test unsupported subcommand ... ok
test08_no_command (oeqa.selftest.wic.Wic)
Test wic without command ... ok
test09_help_kickstart (oeqa.selftest.wic.Wic)
Test wic help overview ... ok
test10_help_plugins (oeqa.selftest.wic.Wic)
Test wic help plugins ... ok
test11_help_kickstart (oeqa.selftest.wic.Wic)
Test wic help kickstart ... ok
test12_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with gzip ... ok
test13_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with bzip2 ... ok
test14_compress_gzip (oeqa.selftest.wic.Wic)
Test compressing an image with xz ... ok
test15_wrong_compressor (oeqa.selftest.wic.Wic)
Test how wic breaks if wrong compressor is provided ... ok
test16_rootfs_indirect_recipes (oeqa.selftest.wic.Wic)
Test usage of rootfs plugin with 'rootfsN=<recipe>' syntax ... ok

----------------------------------------------------------------------
Ran 16 tests in 90.257s

OK
2015-06-26 16:07:39,979 - selftest - INFO - Finished
2015-06-26 16:07:39,979 - selftest - INFO - Removing the include from local.conf

I'm going to close this bug and I'll continue to add new test cases for new and old functionality.
Comment 8 Ed Bartosh 2015-06-29 08:24:48 UTC
15 test cases are in master now: http://cgit.openembedded.org/openembedded-core/commit/?id=0ffc5c38ef28605abf815a47eb41923802d31df7

2 more are waiting for review. I think it's enough to say that wic testing suite is created and put into production.
Comment 9 Daniel Istrate 2015-09-15 13:16:25 UTC
As of now in master: ea562de57590c966cd5a75fda8defecd397e6436 there are 23 test cases in meta/lib/oeqa/selftest/wic.py