Bug 9244 - patchtest-oe: Create tests to validate recipes, classes or config files
Summary: patchtest-oe: Create tests to validate recipes, classes or config files
Status: RESOLVED FIXED
Alias: None
Product: Patchwork/Patchtest
Classification: Yocto Project Subprojects
Component: Patchtest (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 2.2 M2
Assignee: Leonardo Sandoval Gonzalez
QA Contact: Alexandru Georgescu
URL:
Whiteboard:
Depends on: 8648
Blocks:
  Show dependency tree
 
Reported: 2016-03-11 17:30 UTC by Alex Franco
Modified: 2016-08-04 17:32 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Yes (doc changes required)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Franco 2016-03-11 17:30:05 UTC 
Split from bug 4078

For bug 4078, pt-suites was written as a test suite to work with patchtest

The tests done there work for generic python files and patches, but not properly recipes, classes and config files. The tests we need for this bug are:

If **recipe** files modified by patch, recipe checks
  - Required for these tests:
    - list of mbox-formatted patches
    - list of files modified per patch
    - bitbake environment: use tinfoil? bitbake -e? new oe-selftest test?

  - Tests:
    1. SRC_URI - removed files, but files still around
    2. SRC_URI - points to something other than an SCM, but checksums not changed
    3. LIC_FILES_CHKSUM - missing
    4. LIC_FILES_CHKSUM - changed but commit message doesn't include "lic"

Additional tests should warrant additional bugs
Comment 1 Leonardo Sandoval Gonzalez 2016-06-14 14:13:05 UTC 
This is expected to be finished early 2.2M2. These are the test cases considered right now:


patch.signed_off_by.test_signed_off_by.OEPatchSignedOffBy.test_signed_off_by_format
patch.signed_off_by.test_signed_off_by.OEPatchSignedOffBy.test_signed_off_by_presence
patch.upstream_status.test_upstream_status.OEPatchUpstreamStatus.test_upstream_status_presence
patch.upstream_status.test_upstream_status.OEPatchUpstreamStatus.test_upstream_status_valid_status
bitbake.parse.test_parse.OEBitbakeParse.test_bitbake_parse
mbox.payload.test_payload.OEPayload.test_payload_presence
mbox.signed_off_by.test_signed_off_by.OESignedOffBy.test_signed_off_by_format
mbox.signed_off_by.test_signed_off_by.OESignedOffBy.test_signed_off_by_presence
mbox.subject.test_subject.OESubject.test_subject_format
mbox.subject.test_subject.OESubject.test_subject_presence
mbox.description.test_description.OEDescription.test_presence
python.lint.test_pylint.OEPyLint.test_pylint
metadata.lic_files_chksum.test_lic_files_chksum.OELicFilesChksum.test_modified_not_mentioned
metadata.lic_files_chksum.test_lic_files_chksum.OELicFilesChksum.test_presence
metadata.src_uri.test_src_uri.OESrcUri.test_checksums_not_changed
metadata.src_uri.test_src_uri.OESrcUri.test_left_files
Comment 2 Leonardo Sandoval Gonzalez 2016-06-23 19:51:30 UTC 
My fault, moving it to M1 again. CCB should agree on any milestone movement.
Comment 3 Leonardo Sandoval Gonzalez 2016-07-07 15:16:23 UTC 
Basically the same comment as https://bugzilla.yoctoproject.org/show_bug.cgi?id=4078#c17

Moving to 2.2 M2 so we can have this time window for testing.
Comment 4 Leonardo Sandoval Gonzalez 2016-08-04 17:32:26 UTC 
Checks completed on repo ssh://git@git.yoctoproject.org/patchtest-oe, folder tests. Errors from this code should be reported in a separate bug.