| Summary: | Implement script to test new patches coming to oe/yocto mailing lists | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Ed Bartosh <eduard.bartosh> |
| Component: | Scripts and Tools | Assignee: | Ed Bartosh <eduard.bartosh> |
| Status: | RESOLVED DUPLICATE | QA Contact: | |
| Severity: | normal | ||
| Priority: | Undecided | CC: | leonardo.sandoval.gonzalez, richard.purdie, ross.burton |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
Ed Bartosh
2016-11-08 10:15:36 UTC
This is patchtest, which has been in development for a cycle or two now and should be going live shortly. If you want to find out more you should have the slides for the presentation about it in your email, or speak to Leo/Jose in GDC. Running oe-selftest might be a little excessive as it takes about 8 hours, which isn't really "fast feedback", so until we can speed up/trim selftest that will have to wait. Every patch that goes into master does have to pass oe-selftest on the AB though. Marking as a dup of 10376 (deploy patchtest) as I can't seem to find the original master bug. *** This bug has been marked as a duplicate of bug 10376 *** This is not patchset. Patchset, when matured, will run a lot of tests that I'm as a maintainer of wic is not interested. It's not simple, I can't configure it to my needs and I don't want to wait hours for results it will produce as most of them are not interested to me. I want simple tool that does exactly what I need and does it fast. > Running oe-selftest might be a little excessive as it takes about 8 hours, which isn't really "fast feedback"
oe-selftest --coverage -r wic won't take 8 hours to run.
Your first line: "The idea is to provide maintainers of oe/yocto software components a simple tool that fetches new patchsets from patchwork.openembedded.org, applies them to master, runs tests and sends results and logs to maintainer e-mail if tests fail" That is an exact description of patchtest. How is what you are suggesting not patchtest? As I suggested by "until we trim selftest" we could easily add the subset of selftest modules that execute quickly to patchtest, though I was thinking more about the ones that perform tests without actually building anything. If a patch being tested changes gcc or autotools.bbclass, wic's selftest will still a significant amount of time as it has to rebuild core-image-minimal from scratch. I'd happy to be wrong on this. Will see. So far I can't use patchset and can't confirm it does what I want. I thought it will run tests on machines I have no access to. If I can use it and tweak to my needs then I agree this bug is a duplicate of patchset deployment bug. However, I'm not sure I understand why simple tool like I want has to be deployed somewhere. I want to get it, install on my machine and run, that's it. Can I do that? If you're thinking about something for personal use where you don't have to worry about a malicious patch adding "python() {os.system("rf -rf /")} to a recipe, then a quick script to poll patchworks and grab patches should be fairly trivial.
But given the incredibly wide scope for exploits I think "we" need to do this properly.
I'll give patchtest a try and close this bug as duplicate if it does what I need.
Regarding malicious patches - I was going to run script in docker container(s), so trivial things like os.system("rf -rf /") won't do a lot of damage. I doubt that patchtest can harden my system better than docker.
Patchtest is intended to do what you describe. Its not meant to take long, it should be able to determine the status of a patch in a matter of minutes, if that long. I'd strongly suggest you work with the people working on patchtest so that if it doesn't do what you need right now, we make it do that. This sounds like 100% duplication and I do not want to see that within the project. OK, feel free to close this then. As I said - I'll give patchwork a try. If it does what I want - perfect, I'll be happy to use it. Closing. *** This bug has been marked as a duplicate of bug 10376 *** (In reply to comment #5) > I'd happy to be wrong on this. Will see. So far I can't use patchset and > can't confirm it does what I want. I thought it will run tests on machines I > have no access to. If I can use it and tweak to my needs then I agree this > bug is a duplicate of patchset deployment bug. However, I'm not sure I > understand why simple tool like I want has to be deployed somewhere. In order to automatically process patches, a cronjob needs to be set somewhere. The latter and the proper patchtest setup will be done by in a LF infrastructure. >I want to get it, install on my machine and run, that's it. Can I do that? Sure. Please follow the README located at the patchtest repo (now at git.yp.org) and follow the host installation and execution section. For the moment, we are not launching any Poky script (bitbake or oe-selftest) because the qemu machine we are using is not prepared, but the test cases are there, just these are being skipped by the moment. patchtest is using the unittest python module, and patchtest-oe, the test suite, is based on it. Check the suite and augment it to your needs, then run the tool with some wic mboxes/patches. Documentation is not the best but please give it a try and let me know if you get stuck in any step. One last thing that was already touched in the comments: patchtest in production is not intended for deep testing, we are mainly focused on parsing the commit metadata and patch payload and provide feedback to the developer as soon as possible. |