Bug 13936

Summary: Intermittent HTTP Error 500: INTERNAL SERVER ERROR on autobuilder
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Steve Sakoman <steve>
Component: oe-core otherAssignee: Steve Sakoman <steve>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: mhalstead, randy.macleod
Version: 3.1   
Target Milestone: 3.2 M3   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Steve Sakoman 2020-06-10 15:35:38 UTC 
Typically in core-image-sato-sdk.bb:do_testimage

Seen in both dunfell and master

Recent examples:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/906
https://autobuilder.yoctoproject.org/typhoon/#/builders/107/builds/595
Comment 1 Steve Sakoman 2020-08-04 07:09:14 UTC 
After disappearing for quite some time it has occurred again on dunfell:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/1143
Comment 2 Steve Sakoman 2020-08-08 15:54:00 UTC 
Occurred again on dunfell:

https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/787

This time I was able to log into the builder before the build disappeared and run the failing command again manually:

[sakoman@centos7-ty-2 ~]$ /home/pokybuild/yocto-worker/qemumips-alt/yocto-autobuilder-helper/scripts/upload-error-reports /home/pokybuild/yocto-worker/qemumips-alt/build/build https://autobuilder.yoctoproject.org/typhoon/#builders/102/builds/787
[sakoman@centos7-ty-2 ~]$

The command completed without error messages, so perhaps these are intermittent infrastructure issues???
Comment 3 Steve Sakoman 2020-08-17 20:21:42 UTC 
Richard and I found a couple of error reports that would consistently trigger the 500 error.  There were no errors in the http server log, however there was an error in the gunicorn server log that indicated an issue with encodings.

Michael Halstead updated the default collation on the DB to utf8_unicode_ci and now it handles the triggering error report without error.
Comment 4 Michael Halstead 2020-08-20 07:52:46 UTC 
The default collation did not allow storage of all needed unicode characters. Altering the settings as below took about 12 minutes to complete and resolved the errors. Django doesn't have a way to specify the collation for a table so I'll add this to the README.

ALTER DATABASE `error_report_web` CHARACTER SET 'utf8' COLLATE 'utf8_unicode_ci'
ALTER TABLE  Post_buildfailure convert to character set DEFAULT COLLATE DEFAULT;
ALTER TABLE  Post_build convert to character set DEFAULT COLLATE DEFAULT;