Bug 10399 - bitbake is unable to handle exceptions past the clean_environment() call
Summary: bitbake is unable to handle exceptions past the clean_environment() call
Status: RESOLVED FIXED
Alias: None
Product: BitBake
Classification: Build System, Metadata & Runtime
Component: bitbake (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: High critical
Target Milestone: 2.2 M4
Assignee: Aníbal Limón
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-10-07 22:10 UTC by Benjamin Esquivel
Modified: 2017-12-06 17:11 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Benjamin Esquivel 2016-10-07 22:10:20 UTC 
When bitbake --server-only is called, any exception, handled or unhandled; that occurs after this line: 

  cleanedvars = bb.utils.clean_environment()

from bitbake/lib/bb/main.py, produces an infinite loop of cleanliness.

An example of this is bug 10397.
Comment 1 Benjamin Esquivel 2016-10-07 22:18:59 UTC 
An induced example of this bug can be found at the poky-contrib branch besquive/such-clean:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=besquive/such-clean

which places a handled exception right after the point where it will loop endlessly.

it is worth noting that the exception could happen on further lines than this induced example, like with bug 10397.
Comment 2 Benjamin Esquivel 2016-10-13 21:11:41 UTC 
Forgot to add the steps to reproduce:

1. clone the git repo
2. get the code from the besquive/such-clean branch (stimulates an exception)
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=besquive/such-clean
3. run the init build env script
4. start a bitbake server, i.e.
 $ bitbake --server-only -t xmlrpc -B localhost:1234
5. watch the fireworks
Comment 3 Aníbal Limón 2016-10-13 21:34:04 UTC 
The loop is on, 

File "/home/alimon/repos/pokyro/bitbake/lib/bb/event.py", line 158, in print_ui_queue

Due of that function is called atexit, if you move the registry of bb.event.LogHandler after bb.utils.clean_environment() it will fix the problem because clean_environment() is calling logger.debug, more investigation is needed.

diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py
index 19ee729..992e879 100755
--- a/bitbake/lib/bb/main.py
+++ b/bitbake/lib/bb/main.py
@@ -435,19 +435,15 @@ def bitbake_main(configParams, configuration):
     bb.msg.init_msgconfig(configParams.verbose, configuration.debug,
                           configuration.debug_domains)
 
+    # Clear away any spurious environment variables while we stoke up the cooker
+    cleanedvars = bb.utils.clean_environment()
+
     # Ensure logging messages get sent to the UI as events
     handler = bb.event.LogHandler()
     if not configParams.status_only:
         # In status only mode there are no logs and no UI
         logger.addHandler(handler)
Comment 4 Aníbal Limón 2016-10-13 22:04:20 UTC 
The bug is caused because bb.event.LogHandler adds events to ui_queue when is called causing an infinite loop when interate over ui_queue at print_ui_queue.
Comment 5 Aníbal Limón 2016-10-13 22:05:04 UTC 
Also happens in bitbake proceess not only server mode.