Bug 9803 - cooker.py blew up on failed build (serialization issue?)
Summary: cooker.py blew up on failed build (serialization issue?)
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Undecided normal
Target Milestone: 2.2 M1
Assignee: Ed Bartosh
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-06-18 13:28 UTC by Jussi Kukkonen
Modified: 2016-06-21 07:08 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jussi Kukkonen 2016-06-18 13:28:04 UTC
Recent nightly-no-x11 build failed (cmake failure is handled on mailing list), and something went wrong in cooker.py. The resulting log is is almost 400MB of python traceback that looks like this:

b'\x80\x03cbb.cooker\nCookerExit\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00pidq\x03K\x00sb.' is not JSON serializable Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-no-x11/build/bitbake/lib/bb/cooker.py", line 330, in write_event
    f.write("%s\n" % json.dumps({"class":event.__module__ + "." + event.__class__.__name__, "vars":json.dumps(pickle.dumps(event)) }))
  File "/usr/lib/python3.4/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.4/json/encoder.py", line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.4/json/encoder.py", line 250, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: b'\x80\x03cbb.cooker\nCookerExit\nq\x00)\x81q\x01}q\x02X\x03\x00\x00\x00pidq\x03K\x00sb.' is not JSON serializable
Comment 1 Joshua Lock 2016-06-20 08:03:56 UTC
I think we may need to fix up the -w/--write-log option for Python3.

However I'm not sure if/where this option is being set for use on the autobuilder. Looking at:

yocto-autobuilder/lib/python2.7/site-packages/autobuilder\
/buildsteps/BuildImages.py

we pass -w 'bitbake_eventlog.json' if the images variable includes #TOASTER, but I don't see that being set in the buildset.controller/nightly-no-x11.conf

CC'ing some of the build & release folks who may be able to shed light on that
Comment 2 Ed Bartosh 2016-06-20 09:56:08 UTC
Looks like I fixed it while working on bug #9585 https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=ed/toaster/eventplay-9585

I'll submit the fix soon.
Comment 3 Ed Bartosh 2016-06-20 09:56:50 UTC
moving to M1
Comment 4 Ed Bartosh 2016-06-20 11:04:43 UTC
Patchset has been sent for review: http://lists.openembedded.org/pipermail/bitbake-devel/2016-June/007602.html
Comment 5 Ed Bartosh 2016-06-21 07:08:31 UTC
The fix has been merged to poky master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=e28b36e1e8b2d6b1531698d6d50658e3fc73adc0