Bug 10399

Summary: bitbake is unable to handle exceptions past the clean_environment() call
Product: [Build System, Metadata & Runtime] BitBake Reporter: Benjamin Esquivel <benjamin.esquivel>
Component: bitbakeAssignee: Aníbal Limón <anibal.limon>
Status: RESOLVED FIXED QA Contact:
Severity: critical    
Priority: High CC: brian.avery, poky.bs.watcher, poky.watcher
Version: unspecified   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10397
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Benjamin Esquivel 2016-10-07 22:10:20 UTC 
When bitbake --server-only is called, any exception, handled or unhandled; that occurs after this line: 

  cleanedvars = bb.utils.clean_environment()

from bitbake/lib/bb/main.py, produces an infinite loop of cleanliness.

An example of this is bug 10397.
Comment 1 Benjamin Esquivel 2016-10-07 22:18:59 UTC 
An induced example of this bug can be found at the poky-contrib branch besquive/such-clean:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=besquive/such-clean

which places a handled exception right after the point where it will loop endlessly.

it is worth noting that the exception could happen on further lines than this induced example, like with bug 10397.
Comment 2 Benjamin Esquivel 2016-10-13 21:11:41 UTC 
Forgot to add the steps to reproduce:

1. clone the git repo
2. get the code from the besquive/such-clean branch (stimulates an exception)
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=besquive/such-clean
3. run the init build env script
4. start a bitbake server, i.e.
 $ bitbake --server-only -t xmlrpc -B localhost:1234
5. watch the fireworks
Comment 3 Aníbal Limón 2016-10-13 21:34:04 UTC 
The loop is on, 

File "/home/alimon/repos/pokyro/bitbake/lib/bb/event.py", line 158, in print_ui_queue

Due of that function is called atexit, if you move the registry of bb.event.LogHandler after bb.utils.clean_environment() it will fix the problem because clean_environment() is calling logger.debug, more investigation is needed.

diff --git a/bitbake/lib/bb/main.py b/bitbake/lib/bb/main.py
index 19ee729..992e879 100755
--- a/bitbake/lib/bb/main.py
+++ b/bitbake/lib/bb/main.py
@@ -435,19 +435,15 @@ def bitbake_main(configParams, configuration):
     bb.msg.init_msgconfig(configParams.verbose, configuration.debug,
                           configuration.debug_domains)
 
+    # Clear away any spurious environment variables while we stoke up the cooker
+    cleanedvars = bb.utils.clean_environment()
+
     # Ensure logging messages get sent to the UI as events
     handler = bb.event.LogHandler()
     if not configParams.status_only:
         # In status only mode there are no logs and no UI
         logger.addHandler(handler)
Comment 4 Aníbal Limón 2016-10-13 22:04:20 UTC 
The bug is caused because bb.event.LogHandler adds events to ui_queue when is called causing an infinite loop when interate over ui_queue at print_ui_queue.
Comment 5 Aníbal Limón 2016-10-13 22:05:04 UTC 
Also happens in bitbake proceess not only server mode.