Bug 7263 - Error Reporting Tool can't cope with binary data in the logs
Summary: Error Reporting Tool can't cope with binary data in the logs
Status: RESOLVED FIXED
Alias: None
Product: Error Reporting tool
Classification: Yocto Project Subprojects
Component: Error Reporting Tool (show other bugs)
Version: 1.7
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.8 M3
Assignee: Michael Wood
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-01-29 11:12 UTC by Richard Purdie
Modified: 2015-02-10 15:24 UTC (History)
0 users

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Purdie 2015-01-29 11:12:04 UTC
For example:

diff --git a/meta/recipes-extended/bash/bash_4.3.bb b/meta/recipes-extended/bash/bash_4.3.bb
index 23d3197..19b06a1 100644
--- a/meta/recipes-extended/bash/bash_4.3.bb
+++ b/meta/recipes-extended/bash/bash_4.3.bb
@@ -109,3 +109,8 @@ SRC_URI[patch030.md5sum] = "efb709fdb1368945513de23ccbfae053"
 SRC_URI[patch030.sha256sum] = "85434f8a2f379d0c49a3ff6d9ffa12c8b157188dd739e556d638217d2a58385b"
 
 BBCLASSEXTEND = "nativesdk"
+
+python do_test () {
+    bb.fatal("åäö")
+}
+addtask test

then bitbake bash -c test

NOTE: Executing RunQueue Tasks
ERROR: åäö
ERROR: Function failed: do_test
ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/bash/4.3-r1/temp/log.do_test.26280
ERROR: Task 0 (/media/build1/poky/meta/recipes-extended/bash/bash_4.3.bb, do_test) failed with exit code '1'

You can send the errors to an upstream server by running:
  send-error-report /media/build1/poky/build/tmp/log/error-report/error_report_201501291053.txt [server]

$ send-error-report /media/build1/poky/build/tmp/log/error-report/error_report_201501291053.txt 

Sends an error report (if the report-error class was enabled) to a remote server.

This scripts sends the contents of the error to a public upstream server.

Please remove any identifying information before sending.
500 Internal Server Error
There was a problem submiting your data, response written in /media/build1/poky/build/tmp/log/error-report/error_report_201501291053.txt.response.html

There is also a problem with the report-error.bbclass itself, it calls jsondata['failures'].append(taskdata) and if taskdata['log'] had binary data in it, it could crash, there was no encoding.

You can demonstrate this with something like:

+python do_test () {
+    bindata = open("/bin/bash", "rb").read()
+
+    bb.fatal(bindata)
+}

which obvioulsy messes up the console but you'll also see a traceback from the JSON code when the error-report function is called (you can look at the cooker logs in log/cooker/).
Comment 1 Richard Purdie 2015-01-29 11:15:39 UTC
An attempted fix which would require a change to the transmission format:

From 8ce292fb7e6c777918ea1f1e7fdd0a772f7066fc Mon Sep 17 00:00:00 2001
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Wed, 27 Aug 2014 14:16:58 +0000
Subject: report-error: Fix binary data encoding in logs

If the log file contains binary data the report-error class currently
crashes. This safely encodes the data to deal correctly with binary
characters.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/classes/report-error.bbclass b/meta/classes/report-error.bbclass
index 8b30422..6ba09eb 100644
--- a/meta/classes/report-error.bbclass
+++ b/meta/classes/report-error.bbclass
@@ -47,8 +47,8 @@ python errorreport_handler () {
             taskdata['package'] = e.data.expand("${PF}")
             taskdata['task'] = task
             if log:
-                logFile = open(log, 'r')
-                taskdata['log'] = logFile.read()
+                logFile = open(log, 'rb')
+                taskdata['log'] = logFile.read().encode('base64')
                 logFile.close()
             else:
                 taskdata['log'] = "No Log"
-- 
cgit v0.10.2
Comment 2 Michael Wood 2015-02-03 11:51:31 UTC
As discussed a try/except may actually be the best solution

http://lists.openembedded.org/pipermail/openembedded-core/2015-January/101204.html
Comment 3 Michael Wood 2015-02-03 17:46:32 UTC
(In reply to comment #2)
> As discussed a try/except may actually be the best solution
> 
> http://lists.openembedded.org/pipermail/openembedded-core/2015-January/
> 101204.html

I've added a v2 of the patch - I had meant to put the file open() inside the try as well so that any IOError would also be handled.