Bug 6359

Summary: Include QA tests for kernel-dev manual tasks
Product: [Yocto Project Subprojects] Kernel Reporter: Darren Hart <dvhart>
Component: kernel-toolingAssignee: Francisco Pedraza <francisco.j.pedraza.gonzalez>
Status: VERIFIED FIXED QA Contact: Jose Perez C <jose.perez.carranza>
Severity: enhancement    
Priority: Medium+ CC: alexandru.c.georgescu, bluelightning, costin.c.constantin, georgex.l.musat, juan.fernandox.ramos.frayle, sonia.delgado, yp.kernel.watcher, yp.watcher
Version: 2.3   
Target Milestone: 2.3 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Bug Depends on: 10753, 10754    
Bug Blocks: 11059    

Description Darren Hart 2014-05-21 20:10:31 UTC
Currently, Bruce has a number of QA type tests he runs, but they need to be incorporated into the official QA process. We also need to identify gaps and write new tests to cover all the tasks covered by the kernel-dev manual. For example:

* linux-yocto-custom
  * local source
  * local source with parallel meta
  * local source with recipe-space meta
* External Source
* defconfig
* defconfig + fragments
* linux-yocto meta data + local fragments
* building external modules (hello-mod)

A full audit is necessary to verify to manual with each new release.
Comment 1 Darren Hart 2014-05-21 20:11:15 UTC
Bruce, first task is to you to get what you have over to QA. From there, we will need to work closely with QA to develop the remaining tests.
Comment 2 Darren Hart 2014-07-31 14:12:00 UTC
Adding a requirement:

Minimum linux kernel version supported - ensure the linux-yocto-custom recipe can be used to build a basic recipe of whatever this version is.
Comment 3 Alexandru Georgescu 2014-11-20 09:13:08 UTC
To be discussed internally with Lucian and Costin.
Comment 4 Bruce Ashfield 2016-03-21 23:30:50 UTC
Did anything ever come of the discussions ? We can revisit this in 2.2
Comment 5 Alexandru Georgescu 2016-04-12 08:00:20 UTC
Un(In reply to comment #4)
> Did anything ever come of the discussions ? We can revisit this in 2.2

Hi Bruce, we had some local discussions but we didn't have yet the required resources to get this done into 2.1.
Comment 6 Francisco Pedraza 2016-09-20 19:05:24 UTC
As discussed I am going to take care of this. An initial review has started.
Comment 7 Sonia Delgado 2016-11-08 22:30:02 UTC
@Francisco, what is the result of your analysis for the missing test cases?
Comment 8 Francisco Pedraza 2016-11-22 19:57:28 UTC
(In reply to comment #7)
> @Francisco, what is the result of your analysis for the missing test cases?

Hi,
I will automate the commont tasks on kernel development. Also I will create a test plan for this on testopia.

Until now I have covered only 
1. Creating and preparing a layer
I am working with:
2. Modifying an Existing Recipe
I will add:
3. Creating the Append File
4. Applying Patches
5. Changing the Configuration
6. Using an "In-Tree"  defconfig File
7. Generating Configuration Files
8. Working with Out-of-Tree Modules
9. Building Out-of-Tree Modules on the Target
10. Incorporating Out-of-Tree Modules
11. Adding Recipe-Space Kernel Features
Comment 9 Francisco Pedraza 2016-12-01 13:36:38 UTC
I am going to split this one with the basic functionality for kernel-dev

1: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10753
1. Creating and preparing a layer
2. Modifying an Existing Recipe
3. Creating the Append File
4. Applying Patches

2: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10754
5. Changing the Configuration
6. Using an "In-Tree"  defconfig File
7. Generating Configuration Files
8. Working with Out-of-Tree Modules
9. Building Out-of-Tree Modules on the Target
10. Incorporating Out-of-Tree Modules
11. Adding Recipe-Space Kernel Features

3: This functionality is going to be added on 2.3 M2
* linux-yocto-custom
  * local source
  * local source with parallel meta
  * local source with recipe-space meta
* External Source
* defconfig
* defconfig + fragments
* linux-yocto meta data + local fragments
* building external modules (hello-mod)
Comment 10 Francisco Pedraza 2017-01-24 01:17:35 UTC
Some test cases already designed (will be only manual tasks).
Test Case ID: TC_KD_01	
Test Case Name: TC_KD_Applying_patches	
Description:
"As a Developer I want to be able to apply single patch to Linux Kernel Source"
Corresponded to: "* linux-yocto-custom"

Test Case ID: TC_KD_02
Test Case Name: TC_KD_linux-yocto-custom
Description: "As a Developer I want to to be able to work with your own sources"

Test Case ID: TC_KD_03
Test Case Name: TC_KD_local_parallel_meta
Description:"As a Developer I want to be able to work with local source with parallel meta."

Requirements covered now:
* linux-yocto-custom
* External Source
Comment 11 Francisco Pedraza 2017-01-25 23:08:42 UTC
One more test designed:
TC ID: TC_KD_defconfig
Description: "As a Developer I want to be able to work generate and change  configuration files (defconfig)"
Covered Requirement: " * defconfig"
Comment 12 Francisco Pedraza 2017-01-31 16:42:01 UTC
I have submitted some test cases to Testopia [1], also I created a wiki as well (first version)[2]. I hope to have this finished the next week, with the final version of the test cases and the design of the remaining.
[1]- https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6713
[2] - https://wiki.yoctoproject.org/wiki/Kernel_Development_QA#About_Kernel_Development
Comment 13 Francisco Pedraza 2017-01-31 16:44:11 UTC
(In reply to comment #12)
> I have submitted some test cases to Testopia [1], also I created a wiki as
> well (first version)[2]. I hope to have this finished the next week, with
> the final version of the test cases and the design of the remaining.
> [1]- https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6713
> [2] -
> https://wiki.yoctoproject.org/wiki/
> Kernel_Development_QA#About_Kernel_Development

Adding the template as well: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6714
Comment 14 Francisco Pedraza 2017-02-15 23:24:43 UTC
Test Cases added to https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=6714 testopia and updated on Test Plan as well. https://wiki.yoctoproject.org/wiki/Kernel_Development_QA#About_Kernel_Development
Comment 15 Juan Ramos 2017-09-01 17:10:30 UTC
Verified. https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=7788