Bug 14208

Summary: send-error-report fails with 500: INTERNAL SERVER ERROR
Product: [Infrastructure] AutoBuilder Reporter: Alexandre Belloni <alexandre.belloni>
Component: autobuilderAssignee: Thomas Perrot <thomas.perrot>
Status: RESOLVED DUPLICATE QA Contact:
Severity: normal    
Priority: Medium+ CC: chee.yang.lee, infras.ab.watcher, Infras.watcher, pidge, randy.macleod, sgw
Version: 3.3   
Target Milestone: 3.4 M2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
error_report_bitbake_1611770503.txt
none
error_report_bitbake_1611770519.txt
none
fix "false"
none
correct error reporting
none
error_report_bitbake_1619453707.txt none

Description Alexandre Belloni 2021-01-29 20:40:58 UTC 
Created attachment 4766 [details]
error_report_bitbake_1611770503.txt

There is a reproducible failure at the send-error-report invocation:

ERROR: HTTP Error 500: INTERNAL SERVER ERROR
Preparing to send errors to: errors.yoctoproject.org
ERROR: HTTP Error 500: INTERNAL SERVER ERROR
Preparing to send errors to: errors.yoctoproject.org
ERROR: Command ['/home/pokybuild/yocto-worker/check-layer/yocto-autobuilder-helper/scripts/upload-error-reports', '/home/pokybuild/yocto-worker/check-layer/build/build', 'https://autobuilder.yoctoproject.org/typhoon/#builders/39/builds/2990'] failed

Link to the log (appears above too):
https://autobuilder.yoctoproject.org/typhoon/#/builders/39/builds/2990/steps/20

Attached are both error reports.
Comment 1 Alexandre Belloni 2021-01-29 20:41:30 UTC 
Created attachment 4767 [details]
error_report_bitbake_1611770519.txt
Comment 2 Yi Fan Yu 2021-02-09 16:28:04 UTC 
Yop I can reproduce, wasn't able to send the json file to the server calling the script locally.
Comment 3 Yi Fan Yu 2021-02-10 23:20:28 UTC 
tried forging the requests with a simple curl command
curl  errors.yoctoproject.org/ClientPost/ -L -H "Content-Type: application/json" --user-agent "send-error-report/0.3" --data "`cat ~/Downloads/post_request.txt`"

the server spits back 

Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/csrf.py" in wrapped_view
  58.         return view_func(*args, **kwargs)
File "/var/www/error-report-web/Post/views.py" in addData
  61.         result = p.parse(request)
File "/var/www/error-report-web/Post/parser.py" in parse
  40.         if self.contains_tags(jsondata) == True:
File "/var/www/error-report-web/Post/parser.py" in contains_tags
  28.             if '<' in val:


turns out one of the fields is `"machine":False`, which is not an iterable so `'<' in val` is invalid
Comment 4 Yi Fan Yu 2021-03-09 15:57:05 UTC 
I want also to submit a patch to not send MACHINE as false
moving to M4
Comment 5 Yi Fan Yu 2021-03-09 15:59:44 UTC 
bumping m4
Comment 7 Yi Fan Yu 2021-04-20 17:59:43 UTC 
sent 2 patches to the yocto mailing list to fix some of the weirdness around 
yocto-check-layer-wrapper

this is the error msg seen:

https://errors.yoctoproject.org/Errors/Details/576832/
Comment 8 Yi Fan Yu 2021-04-23 01:33:04 UTC 
Created attachment 4790 [details]
fix "false"
Comment 9 Yi Fan Yu 2021-04-23 01:34:10 UTC 
Created attachment 4791 [details]
correct error reporting
Comment 10 Alexandre Belloni 2021-04-26 21:37:14 UTC 
Another occurrence but probably a different issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/25/steps/14

I'm attaching error_report_bitbake_1619453707.txt
Comment 11 Alexandre Belloni 2021-04-26 21:37:44 UTC 
Created attachment 4792 [details]
error_report_bitbake_1619453707.txt
Comment 17 Thomas Perrot 2021-06-22 17:06:14 UTC 
I reproduced this failure, with the following command, to send the report to a local instance of error-report-web:

send-error-report -n foo --no-ssl -y -s 127.0.0.1:8000 ../error_report_bitbake_1619453707.txt

And from error-report-web side, the following stack trace has been obtained:

Internal Server Error: /ClientPost/
Traceback (most recent call last):
  File "/src/Post/parser.py", line 66, in parse
    b.save()
  File "/src/Post/models.py", line 53, in save
    self.ERROR_TYPE)
Post.models.InvalidErrorType: Error type check-layer is not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/src/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/src/venv/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/src/Post/views.py", line 63, in addData
    result = p.parse(request)
  File "/src/Post/parser.py", line 69, in parse
    return { 'error' : "Problem reading json payload, %s" % e.message }
AttributeError: 'InvalidErrorType' object has no attribute 'message'
[22/Jun/2021 11:57:06] "POST /ClientPost/ HTTP/1.1" 500 83858
Comment 18 Thomas Perrot 2021-06-23 06:01:22 UTC 
The issue occurs because "check-layer" isn't an entry of "ERROR_TYPE_CHOICES":

class Build(models.Model):
    ERROR_TYPE_CHOICES = (
            (ErrorType.RECIPE, 'Recipe'),
            (ErrorType.CORE, 'Core'),
            (ErrorType.BITBAKE_SELFTEST, 'Bitbake selftest'),
            (ErrorType.OE_SELFTEST, 'OE selftest'),
    )

When I add a "check-layer" in ERROR_TYPE_CHOICES and ErrorType, the report is sent with success:

send-error-report -n toto --no-ssl -y -s 127.0.0.1:8000 ../error_report_bitbake_1619453707.txt
Preparing to send errors to: 127.0.0.1:8000
Your entry can be found here: http://127.0.0.1:8000/Errors/Build/2/

[23/Jun/2021 00:55:19] "GET /ClientPost/JSON HTTP/1.1" 301 0
[23/Jun/2021 00:55:19] "GET /ClientPost/JSON/ HTTP/1.1" 200 62
[23/Jun/2021 00:55:19] "POST /ClientPost/ HTTP/1.1" 200 67


diff --git a/Post/models.py b/Post/models.py
index 3fa66f2198a4..b7a913c82359 100644
--- a/Post/models.py
+++ b/Post/models.py
@@ -15,6 +15,7 @@ import Levenshtein
 
 class ErrorType(object):
     RECIPE = 'recipe'
+    CHECK_LAYER = 'check-layer'
     CORE = 'core'
     BITBAKE_SELFTEST = 'bitbake-selftest'
     OE_SELFTEST = 'oe-selftest'
@@ -26,6 +27,7 @@ class InvalidErrorType(Exception):
 class Build(models.Model):
     ERROR_TYPE_CHOICES = (
             (ErrorType.RECIPE, 'Recipe'),
+            (ErrorType.CHECK_LAYER, 'check-layer'),
             (ErrorType.CORE, 'Core'),
             (ErrorType.BITBAKE_SELFTEST, 'Bitbake selftest'),
             (ErrorType.OE_SELFTEST, 'OE selftest'),
Comment 19 Thomas Perrot 2021-07-05 07:17:47 UTC 
Changes above has been merged.
Comment 20 Randy MacLeod 2021-07-15 19:11:48 UTC 
Thanks Thomas!

For anyone looking, here's a link to the commit:
  http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/commit/?id=7fa9441d4498baf9e4655d946fffb7217a6ce437
Comment 21 Saul Wold 2021-08-31 21:43:03 UTC 
Seems to be happening again.

2 recent builds that had layer-check failures are getting HTTP Error 500.

https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/232
https://autobuilder.yoctoproject.org/typhoon/#/builders/121/builds/231


Internal Server Error: /ClientPost/
Traceback (most recent call last):
  File "/var/www/error-report-web/Post/parser.py", line 66, in parse
    b.save()
  File "/var/www/error-report-web/Post/models.py", line 53, in save
    self.ERROR_TYPE)
Post.models.InvalidErrorType: Error type check-layer is not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
    return view_func(*args, **kwargs)
  File "/var/www/error-report-web/Post/views.py", line 63, in addData
    result = p.parse(request)
  File "/var/www/error-report-web/Post/parser.py", line 69, in parse
    return { 'error' : "Problem reading json payload, %s" % e.message }
AttributeError: 'InvalidErrorType' object has no attribute 'message'
Comment 22 Alexandre Belloni 2021-09-02 14:52:17 UTC 
A new bug #14506 has been opened to track the new failure.

*** This bug has been marked as a duplicate of bug 14506 ***
Comment 23 Randy MacLeod 2025-02-14 16:46:26 UTC 
bulk change to add Saul's non-WR address.
Comment 24 Randy MacLeod 2025-02-14 16:48:27 UTC 
Bulk change: Remove Saul's old WR address.