Bug 10849

Summary: esdk creation on Toaster has traceback errors from buildinfohelper
Product: [Build System, Metadata & Runtime] Toaster Reporter: brian avery <brian.avery>
Component: toasterAssignee: David Reyna <david.reyna>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: normal    
Priority: Medium CC: jessica.zhang, libertad.gonzalez.de.la.cruz, ross.burton, stanciux.mihail
Version: 2.3   
Target Milestone: 2.3 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
toaster_ui.log for core-image-minimal:do_populate_sdk_ext none

Description brian avery 2016-12-22 20:38:12 UTC
Created attachment 3575 [details]
toaster_ui.log for core-image-minimal:do_populate_sdk_ext

To replicate:
build core-image-minimal:do_populate_sdk_ext
machine=qemux86,system=x86-64


Errors in build-toaster-2/toaster_ui.log:

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_configure.9521
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_compile.21346
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_compile.21346
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

---
There are others but this is a sample. Log file attached.
Comment 1 David Reyna 2017-01-19 07:29:42 UTC
Here is a simple Reproducer:
  bitbake nativesdk-chrpath -c cleansstate 
  bitbake nativesdk-chrpath

It appears the problem is that there is both a 'virtual:native' and a 'virtual:nativesdk' version of certain packages, and that the guessing code in update_and_store_task() is only expecting one candidate. The solution appears to be to add awareness of 'native' versus 'nativesdk' to break ties.

Example failing event lookup for self.internal_state['taskdata']:
  identifier = /<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch
  candidates= 'virtual:native:/<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch': {'outcome': 1},
              'virtual:nativesdk:/<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch': {'outcome': -1}
Comment 2 David Reyna 2017-01-20 06:53:46 UTC
The core problem is that some events are missing the "virtual:native[sdk]:" prefix for respective identifiers, and the compensating guessing code in Toaster is not set up to resolve between these two.

It turns out that the broken event's have a "_package" member that consistently has a 'native[sdk]-' prefix that can be used to break that tie, as per:

In update_and_store_task():
                if len(candidates) == 1:
                    identifier = candidates[0]
+               # break tie if missing 'native[sdk]' prefix
+               if (len(candidates) == 2) and hasattr(event,'_package'):
+                   if (0 <= str(event._package).find('native-')):
+                       identifier = 'native:' + identifier
+                   if (0 <= str(event._package).find('nativesdk-')):
+                       identifier = 'nativesdk:' + identifier
+                   candidates = [x for x in self.internal_state['taskdata'].keys() if x.endswith(identifier)]
+                   if len(candidates) == 1:
+                       identifier = candidates[0]

The proposal is to fix this special case now in Toaster, and then come back later and fix the broken events in the bb core code.
Comment 3 brian avery 2017-03-06 22:04:51 UTC
submitted to bitbake
https://patchwork.openembedded.org/patch/137831/
Comment 4 Ross Burton 2017-03-20 17:35:31 UTC
Merged in oe-core e455e40ba309837903b9e2d5f1dff55cce1135de.