Bug 12700 - [2.5 M3 rc1][poky-tiny][Centos7][autobuilder2] - Clobber build dir - failed to load utils
Summary: [2.5 M3 rc1][poky-tiny][Centos7][autobuilder2] - Clobber build dir - failed t...
Status: RESOLVED WONTFIX
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: 2.5
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 2.5 M4
Assignee: Aaron Chan
QA Contact: Aaron Chan
URL: http://centaurus02.png.intel.com:8010...
Whiteboard:
: 12701 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-04-20 02:28 UTC by Aaron Chan
Modified: 2018-05-03 08:25 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: CentOS
Type of Regression: Regression (Found by QA test)
Verified:
Documentation change: Don't know


Attachments
BuildStep - Clobber build dir (3.76 KB, application/octet-stream)
2018-04-20 02:28 UTC, Aaron Chan
no flags Details
[yocto-autobuilder-helper] config.json configfile (30.28 KB, application/octet-stream)
2018-04-26 18:45 UTC, Aaron Chan
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Chan 2018-04-20 02:28:53 UTC
Created attachment 4277 [details]
BuildStep - Clobber build dir

=======================================================================================================
BUILDSTEP - Clobber build dir
=======================================================================================================
Traceback (most recent call last):
  File "/home/ab/yocto-autobuilder-helper/janitor/clobberdir", line 20, in <module>
    ourconfig = utils.loadconfig(os.path.dirname(__file__) + "/../scripts/dummy")
  File "/home/ab/yocto-autobuilder-helper/janitor/../scripts/utils.py", line 79, in loadconfig
    ourconfig = json.load(f)
  File "/usr/lib64/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 717 column 1 (char 30284)
program finished with exit code 1

There are 2 possible reason for this error.
1. utils.loadconfig unable to source the scripts directory to load utils module.
2. error in Json content @ ourconfig = json.load(f)

It appears that this error is only found on this distribution (centos7) and seems to be consistent over other jobs executed as well.
Need to further investigate this to narrow down the root cause.
Comment 1 Richard Purdie 2018-04-26 07:39:23 UTC
Hi Aaron, This looks like it might be caused by an error in the config.json file. Could you check that and if you can't find the issue, attach the config.json please?
Comment 2 Aaron Chan 2018-04-26 18:45:08 UTC
Created attachment 4288 [details]
[yocto-autobuilder-helper] config.json configfile
Comment 3 Aaron Chan 2018-04-26 18:50:16 UTC
Hi Richard,

Was able to locate and find the syntax error from the config.json file since its a long list.
So what i did was to recopy the entire yocto-autobuilder-helper into the worker home directory.

# Just a proposal that we should give a better error handling when loading the json contents.

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/tree/scripts/utils.py#n79

#
# Helper to load the config.json file for scripts in the scripts directory (pass in __file__)
#
def loadconfig(f):
    scriptsdir = os.path.dirname(os.path.realpath(f))

    try:
        with open(os.path.join(scriptsdir, '..', 'config.json')) as f:
            ourconfig = json.load(f)
    except:
        print "[autobuilder2-helper] has found a syntax error in config.json"
        sys.exit(1)

    # Expand templates in the configuration
    ourconfig = expandtemplates(ourconfig)

    return ourconfig
Comment 4 Richard Purdie 2018-05-03 08:22:26 UTC
*** Bug 12701 has been marked as a duplicate of this bug. ***
Comment 5 Ross Burton 2018-05-03 08:24:00 UTC
The syntax error is that you're missing a trailing }
Comment 6 Richard Purdie 2018-05-03 08:25:04 UTC
Unfortunately this isn't something we can easily fix. We're using the standard upstream python json library and any parser issues would need to be fixed there. Much as I'd like to improve this, its therefore out of scope of what we can do within the codebases we're writing.