Bug 12274

Summary:	Automate Kernel Development test cases
Product:	[QA/Testing] Test Plans/Suite	Reporter:	Jair Gonzalez <jair.de.jesus.gonzalez.plascencia>
Component:	test-plans/suite	Assignee:	Mazliana <mazliana.mohamad>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	Medium+	CC:	mazliana.mohamad, randy.macleod, richard.purdie
Version:	2.5
Target Milestone:	2.8 M3
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Jair Gonzalez 2017-10-25 17:12:42 UTC

Currently, test cases for Kernel Development features are executed manually, as documented on the corresponding test plan, which can be found on https://wiki.yoctoproject.org/wiki/Kernel_Development_QA.
The objective of this bug is to track the development of automated test cases to replace their manual counterparts.
The release criteria is to cover at least current test cases (https://wiki.yoctoproject.org/wiki/Kernel_Development_Test_Cases), that apply to the following features:
1. linux-yocto-custom
1.1 local source
1.2 local source with parallel meta
1.3 local source with recipe-space meta
2. External Source
3. defconfig
4. defconfig + fragments
5. building external modules (hello-mod)

Comment 1 Jair Gonzalez 2017-11-06 23:01:19 UTC

To achieve this task there might be some obstacles to solve, as for example, some current test cases require comparing two files to try to determine if they are similar enough to consider the output originates from a reference defconfig file. This may be partially solved by using a differentiation percentage criteria.
Another issue may be that it’s possible that a default configuration option changes in the future, not being able to detect any intended configuration change during a test. For this, a procedure could be documented to update the test if it is not working as expected anymore.

Comment 2 Mazliana 2019-01-16 07:14:20 UTC

A script for test case applying patches is completed.
Currently, in progress for code review before sending a patch to openembedded.

Comment 3 Randy MacLeod 2019-05-30 15:07:38 UTC

Moved to 2.8M1, is this going to submitted soon?

Comment 4 Mazliana 2019-07-12 01:22:32 UTC

The script was done. Status now reviews by the internal team.

Comment 5 Randy MacLeod 2019-08-15 15:01:38 UTC

Moved to M3. Any news on the internal review?

Comment 6 Mazliana 2019-08-26 06:46:02 UTC

Completed automate one of the tests which is TC_KD_01 – Applying Patches.
The patch has been sent upstream. Here the link.

http://lists.openembedded.org/pipermail/openembedded-core/2019-August/286035.html