Bug 9803

Summary: cooker.py blew up on failed build (serialization issue?)
Product: [Build System, Metadata & Runtime] BitBake Reporter: Jussi Kukkonen <jku>
Component: bitbakeAssignee: Ed Bartosh <eduard.bartosh>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Undecided CC: bill.randle, eduard.bartosh, joshuagloe, mhalstead, pidge, poky.bs.watcher, poky.watcher
Version: unspecified   
Target Milestone: 2.2 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Jussi Kukkonen 2016-06-18 13:28:04 UTC
Recent nightly-no-x11 build failed (cmake failure is handled on mailing list), and something went wrong in cooker.py. The resulting log is is almost 400MB of python traceback that looks like this:

b&#39;\x80\x03cbb.cooker\nCookerExit\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00pidq\x03K\x00sb.&#39; is not JSON serializable Traceback (most recent call last):
  File &#34;/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-no-x11/build/bitbake/lib/bb/cooker.py&#34;, line 330, in write_event
    f.write(&#34;%s\n&#34; % json.dumps({&#34;class&#34;:event.__module__ + &#34;.&#34; + event.__class__.__name__, &#34;vars&#34;:json.dumps(pickle.dumps(event)) }))
  File &#34;/usr/lib/python3.4/json/__init__.py&#34;, line 230, in dumps
    return _default_encoder.encode(obj)
  File &#34;/usr/lib/python3.4/json/encoder.py&#34;, line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File &#34;/usr/lib/python3.4/json/encoder.py&#34;, line 250, in iterencode
    return _iterencode(o, 0)
  File &#34;/usr/lib/python3.4/json/encoder.py&#34;, line 173, in default
    raise TypeError(repr(o) + &#34; is not JSON serializable&#34;)
TypeError: b&#39;\x80\x03cbb.cooker\nCookerExit\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00pidq\x03K\x00sb.&#39; is not JSON serializable
Comment 1 Joshua Lock 2016-06-20 08:03:56 UTC
I think we may need to fix up the -w/--write-log option for Python3.

However I'm not sure if/where this option is being set for use on the autobuilder. Looking at:

yocto-autobuilder/lib/python2.7/site-packages/autobuilder\
/buildsteps/BuildImages.py

we pass -w 'bitbake_eventlog.json' if the images variable includes #TOASTER, but I don't see that being set in the buildset.controller/nightly-no-x11.conf

CC'ing some of the build & release folks who may be able to shed light on that
Comment 2 Ed Bartosh 2016-06-20 09:56:08 UTC
Looks like I fixed it while working on bug #9585 https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=ed/toaster/eventplay-9585

I'll submit the fix soon.
Comment 3 Ed Bartosh 2016-06-20 09:56:50 UTC
moving to M1
Comment 4 Ed Bartosh 2016-06-20 11:04:43 UTC
Patchset has been sent for review: http://lists.openembedded.org/pipermail/bitbake-devel/2016-June/007602.html
Comment 5 Ed Bartosh 2016-06-21 07:08:31 UTC
The fix has been merged to poky master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e28b36e1e8b2d6b1531698d6d50658e3fc73adc0