Bug 10849 - esdk creation on Toaster has traceback errors from buildinfohelper
Summary: esdk creation on Toaster has traceback errors from buildinfohelper
Status: RESOLVED FIXED
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: 2.3
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.3 M3
Assignee: David Reyna
QA Contact: Libertad
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-12-22 20:38 UTC by brian avery
Modified: 2017-04-28 17:49 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
toaster_ui.log for core-image-minimal:do_populate_sdk_ext (615.87 KB, application/octet-stream)
2016-12-22 20:38 UTC, brian avery
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description brian avery 2016-12-22 20:38:12 UTC
Created attachment 3575 [details]
toaster_ui.log for core-image-minimal:do_populate_sdk_ext

To replicate:
build core-image-minimal:do_populate_sdk_ext
machine=qemux86,system=x86-64


Errors in build-toaster-2/toaster_ui.log:

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_configure.9521
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_compile.21346
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

NOTE: Logfile for task /workdir/build-toaster-2/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-chrpath/0.16-r0/temp/log.do_compile.21346
ERROR: 
Traceback (most recent call last):
  File "/home/usersetup/poky/bitbake/lib/bb/ui/toasterui.py", line 271, in main
    buildinfohelper.update_and_store_task(event)
  File "/home/usersetup/poky/bitbake/lib/bb/ui/buildinfohelper.py", line 1262, in update_and_store_task
    assert identifier in self.internal_state['taskdata']
AssertionError

---
There are others but this is a sample. Log file attached.
Comment 1 David Reyna 2017-01-19 07:29:42 UTC
Here is a simple Reproducer:
  bitbake nativesdk-chrpath -c cleansstate 
  bitbake nativesdk-chrpath

It appears the problem is that there is both a 'virtual:native' and a 'virtual:nativesdk' version of certain packages, and that the guessing code in update_and_store_task() is only expecting one candidate. The solution appears to be to add awareness of 'native' versus 'nativesdk' to break ties.

Example failing event lookup for self.internal_state['taskdata']:
  identifier = /<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch
  candidates= 'virtual:native:/<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch': {'outcome': 1},
              'virtual:nativesdk:/<path>/recipes-devtools/chrpath/chrpath_0.16.bb:do_fetch': {'outcome': -1}
Comment 2 David Reyna 2017-01-20 06:53:46 UTC
The core problem is that some events are missing the "virtual:native[sdk]:" prefix for respective identifiers, and the compensating guessing code in Toaster is not set up to resolve between these two.

It turns out that the broken event's have a "_package" member that consistently has a 'native[sdk]-' prefix that can be used to break that tie, as per:

In update_and_store_task():
                if len(candidates) == 1:
                    identifier = candidates[0]
+               # break tie if missing 'native[sdk]' prefix
+               if (len(candidates) == 2) and hasattr(event,'_package'):
+                   if (0 <= str(event._package).find('native-')):
+                       identifier = 'native:' + identifier
+                   if (0 <= str(event._package).find('nativesdk-')):
+                       identifier = 'nativesdk:' + identifier
+                   candidates = [x for x in self.internal_state['taskdata'].keys() if x.endswith(identifier)]
+                   if len(candidates) == 1:
+                       identifier = candidates[0]

The proposal is to fix this special case now in Toaster, and then come back later and fix the broken events in the bb core code.
Comment 3 brian avery 2017-03-06 22:04:51 UTC
submitted to bitbake
https://patchwork.openembedded.org/patch/137831/
Comment 4 Ross Burton 2017-03-20 17:35:31 UTC
Merged in oe-core e455e40ba309837903b9e2d5f1dff55cce1135de.