Bug 9244

Summary:	patchtest-oe: Create tests to validate recipes, classes or config files
Product:	[Yocto Project Subprojects] Patchwork/Patchtest	Reporter:	Alex Franco <alejandro.franco>
Component:	Patchtest	Assignee:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status:	RESOLVED FIXED	QA Contact:	Alexandru Georgescu <alexandru.c.georgescu>
Severity:	enhancement
Priority:	Medium+	CC:	leonardo.sandoval.gonzalez, meta.mr.watcher, meta.watcher
Version:	2.2
Target Milestone:	2.2 M2
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Yes (doc changes required)
Bug Depends on:	8648
Bug Blocks:

Description Alex Franco 2016-03-11 17:30:05 UTC

Split from bug 4078

For bug 4078, pt-suites was written as a test suite to work with patchtest

The tests done there work for generic python files and patches, but not properly recipes, classes and config files. The tests we need for this bug are:

If **recipe** files modified by patch, recipe checks
  - Required for these tests:
    - list of mbox-formatted patches
    - list of files modified per patch
    - bitbake environment: use tinfoil? bitbake -e? new oe-selftest test?

  - Tests:
    1. SRC_URI - removed files, but files still around
    2. SRC_URI - points to something other than an SCM, but checksums not changed
    3. LIC_FILES_CHKSUM - missing
    4. LIC_FILES_CHKSUM - changed but commit message doesn't include "lic"

Additional tests should warrant additional bugs

Comment 1 Leonardo Sandoval Gonzalez 2016-06-14 14:13:05 UTC

This is expected to be finished early 2.2M2. These are the test cases considered right now:


patch.signed_off_by.test_signed_off_by.OEPatchSignedOffBy.test_signed_off_by_format
patch.signed_off_by.test_signed_off_by.OEPatchSignedOffBy.test_signed_off_by_presence
patch.upstream_status.test_upstream_status.OEPatchUpstreamStatus.test_upstream_status_presence
patch.upstream_status.test_upstream_status.OEPatchUpstreamStatus.test_upstream_status_valid_status
bitbake.parse.test_parse.OEBitbakeParse.test_bitbake_parse
mbox.payload.test_payload.OEPayload.test_payload_presence
mbox.signed_off_by.test_signed_off_by.OESignedOffBy.test_signed_off_by_format
mbox.signed_off_by.test_signed_off_by.OESignedOffBy.test_signed_off_by_presence
mbox.subject.test_subject.OESubject.test_subject_format
mbox.subject.test_subject.OESubject.test_subject_presence
mbox.description.test_description.OEDescription.test_presence
python.lint.test_pylint.OEPyLint.test_pylint
metadata.lic_files_chksum.test_lic_files_chksum.OELicFilesChksum.test_modified_not_mentioned
metadata.lic_files_chksum.test_lic_files_chksum.OELicFilesChksum.test_presence
metadata.src_uri.test_src_uri.OESrcUri.test_checksums_not_changed
metadata.src_uri.test_src_uri.OESrcUri.test_left_files

Comment 2 Leonardo Sandoval Gonzalez 2016-06-23 19:51:30 UTC

My fault, moving it to M1 again. CCB should agree on any milestone movement.

Comment 3 Leonardo Sandoval Gonzalez 2016-07-07 15:16:23 UTC

Basically the same comment as https://bugzilla.yoctoproject.org/show_bug.cgi?id=4078#c17

Moving to 2.2 M2 so we can have this time window for testing.

Comment 4 Leonardo Sandoval Gonzalez 2016-08-04 17:32:26 UTC

Checks completed on repo ssh://git@git.yoctoproject.org/patchtest-oe, folder tests. Errors from this code should be reported in a separate bug.