| Summary: | Increase oe-selftest coverage of wic codebase | ||
|---|---|---|---|
| Product: | [QA/Testing] Test Plans/Suite | Reporter: | Ed Bartosh <eduard.bartosh> |
| Component: | test-plans/suite | Assignee: | Ed Bartosh <eduard.bartosh> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | benjamin.esquivel, corneliux.stoicescu |
| Version: | unspecified | ||
| Target Milestone: | 2.3 M4 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Ed Bartosh
2016-11-08 09:38:00 UTC
Just a small note that testing all the argument options is one of the objectives of this request. There are two types of argument options in wic - command line arguments and arguments in .wks files. The former are mostly covered by tests already, the latter are covered only partly. Covering options as such shouldn't be a main goal of this task in my opinion. The goal should be to cover as much functionality as possible and increase test coverage. Hi Ed, As you mentioned on your last comment, currently almost all command line options are tested, and .wks options are indirectly tested too by generating the different images on the existing test cases. Also all the current different canned-wks images, and the source plugins they use are tested for executing without errors in this process. Besides this, could you please provide 2 or 3 examples of functionality that you consider that requires to be tested? Looking at current coverage report I'd say creator.py, filemap.py partition.py, rawcopy.py, rootfs_pcbios_ext.py and syslinux.py modules are covered much less than the rest of code. I'd start from those, look at which lines are not covered and decide if it's important functionality to cover or not. Here is the current coverage report for your reference: $ coverage3 report --skip-covered Name Stmts Miss Branch BrPart Cover ---------------------------------------------------------------------------------------------------------------- /home/ed/git/yocto/poky/scripts/lib/wic/conf.py 48 6 12 5 82% /home/ed/git/yocto/poky/scripts/lib/wic/creator.py 62 19 28 11 62% /home/ed/git/yocto/poky/scripts/lib/wic/engine.py 108 13 60 12 85% /home/ed/git/yocto/poky/scripts/lib/wic/filemap.py 247 96 52 6 58% /home/ed/git/yocto/poky/scripts/lib/wic/help.py 49 4 18 3 90% /home/ed/git/yocto/poky/scripts/lib/wic/imager/direct.py 187 18 84 13 87% /home/ed/git/yocto/poky/scripts/lib/wic/ksparser.py 121 34 38 7 65% /home/ed/git/yocto/poky/scripts/lib/wic/msger.py 133 51 52 17 57% /home/ed/git/yocto/poky/scripts/lib/wic/partition.py 213 75 58 14 61% /home/ed/git/yocto/poky/scripts/lib/wic/plugin.py 84 7 35 6 89% /home/ed/git/yocto/poky/scripts/lib/wic/pluginbase.py 25 1 2 0 96% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/imager/direct_plugin.py 36 3 6 2 88% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/bootimg-efi.py 120 17 28 10 79% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/bootimg-partition.py 54 13 24 5 67% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/bootimg-pcbios.py 100 8 30 10 86% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/fsimage.py 22 13 6 0 32% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/isoimage-isohybrid.py 275 84 94 33 61% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/rawcopy.py 31 21 10 0 24% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/rootfs.py 31 10 12 3 65% /home/ed/git/yocto/poky/scripts/lib/wic/plugins/source/rootfs_pcbios_ext.py 72 59 24 0 14% /home/ed/git/yocto/poky/scripts/lib/wic/utils/misc.py 40 3 24 8 83% /home/ed/git/yocto/poky/scripts/lib/wic/utils/oe/misc.py 103 17 44 8 82% /home/ed/git/yocto/poky/scripts/lib/wic/utils/partitionedfs.py 164 24 87 20 80% /home/ed/git/yocto/poky/scripts/lib/wic/utils/runner.py 57 16 31 11 65% /home/ed/git/yocto/poky/scripts/lib/wic/utils/syslinux.py 18 15 12 0 10% /home/ed/git/yocto/poky/scripts/wic 171 46 78 23 71% ---------------------------------------------------------------------------------------------------------------- TOTAL 2582 673 949 227 69% Jair, would you mind if I take this bug? I'm going to refactor wic codebase in M3 time frame, so I'll be changing tests quite actively. It will probably slow both of us down if we need to synchronise our work. Ed, sure, go ahead. I was still about to start the implementation stage, so no problems there. thanks. I'll take it then. The coverage is now 70% and going to be increased after these two patchsets are accepted: https://patchwork.openembedded.org/series/4708/# https://patchwork.openembedded.org/series/5320/# moving to M4 as there is still some work to be done to increase coverage to the target 80-90% Increased test coverage to 81% by these 2 patchsets: http://lists.openembedded.org/pipermail/openembedded-core/2017-March/134460.html http://lists.openembedded.org/pipermail/openembedded-core/2017-March/134761.html please, review both patchsets have been merged into master |