| Summary: |
Improve configuration parser - better error reporting and more lenient |
| Product: |
[Infrastructure] AutoBuilder
|
Reporter: |
Joshua Lock <joshuagloe> |
| Component: |
autobuilder | Assignee: |
Joshua Lock <joshuagloe> |
| Status: |
RESOLVED
OBSOLETE
|
QA Contact: |
|
| Severity: |
enhancement
|
|
|
| Priority: |
Medium
|
CC: |
infras.ab.watcher, Infras.watcher, pidge
|
| Version: |
2.4 | |
|
| Target Milestone: |
2.5 | |
|
| Hardware: |
x86 | |
|
| OS: |
Multiple | |
|
| Whiteboard: |
|
|
OS type for building Yocto:
|
---
|
Type of Regression:
|
---
|
|---|
|
Verified:
|
|
Documentation change:
|
Yes (doc changes required)
|
|---|
The buildset configuration parser is very unforgiving of syntax errors such as a stray comma[1]. Said stray comma results in a long Traceback which doesn't provide many clues for fixing the problem short of digging into the code, i.e. (from[2]): Traceback (most recent call last): File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1187, in unwindGenerator return _inlineCallbacks(None, gen, Deferred()) File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 1045, in _inlineCallbacks result = g.send(result) File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py", line 169, in upgradeMaster master_cfg = loadConfig(config, configFile) File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py", line 53, in loadConfig config['basedir'], configFileName) --- <exception caught here> --- File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in loadConfig exec f in localDict File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg", line 94, in <module> yocto_buildsets.createBuildsets() File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 65, in createBuildsets self.parseBuildSet(buildset) File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet self.parseRepos(buildset) File "/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py", line 127, in parseRepos if layer.iterkeys().next() not in self.repos: exceptions.AttributeError: 'list' object has no attribute 'iterkeys' Errors loading configuration: error while parsing config file: 'list' object has no attribute 'iterkeys' (traceback in logfile) The yocto-autobuilder will be friendlier to administrators if it can help them diagnose such issues more easily by providing more informative reporting of failures, such as pointing to the specific file and line that caused the parser failure. For extra convenience we could: * make the parser more lenient of stray punctuation (like setLenient() in Android's JsonReader[3]) * switch to using a pure data format, such as JSON, rather than parsing Python AST's. If we were to switch to JSON a user could use their editor of choice's built in linter to detect such issues at editing time. 1. http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991 2. https://lists.yoctoproject.org/pipermail/yocto/2017-March/035274.html 3. https://developer.android.com/reference/android/util/JsonReader.html#setLenient(boolean)