Bug 6332

Summary: bbclass handlers that handle MetadataEvent's clobber that event's 'data' member
Product: [Build System, Metadata & Runtime] BitBake Reporter: Dave Lerner <dave.lerner>
Component: bitbakeAssignee: Alexandru Damian <alexandru.damian>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: poky.bs.watcher, poky.watcher
Version: 1.6   
Target Milestone: 1.7   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
a bbclass that is dispatched with MetadataEvent or all events none

Description Dave Lerner 2014-05-14 21:21:26 UTC
Created attachment 1958 [details]
a bbclass that is dispatched with MetadataEvent or all events

As tested in daisy release.

If the function bb.event.execute_handler() is called to run a registered
event handler, and if the event is a MetadataEvent, then the event member
variable 'data', is destroyed by execute_handler() which causes failures
for other handlers registered for that event, such as the toaster
event handlers.

This happens when either
- a bbclass invokes 'addhandler' without setting an event mask, and
consequently gets all events, or
- a bbclass adds the handler and sets the handler's event mask to MetadataEvent.

To reproduce the case where a handler receiving all events clobbers
the MetadataEvent.data member variable:
1) copy the attached file bug_event.bbclass into poky/meta/classes
2) run the init script
    source oe-init-build-env
3) append to conf/local.conf
    INHERIT += "bug_event"
4) start toaster which listens for metadata events
    source toaster start
5) generate some events
    bitbake quilt-native
    ...
    NOTE: bug_event: data was found in vars as dispatched by execute_handler
    ...
The event handler reports that all MetadataEvents have the 'data' member
variable, as shown by the above highlighted "NOTE: bug_event", only the "'data' in vars(e)" clause executes in the code below:
    python bug_event() {
        if isinstance(e, bb.event.MetadataEvent):
            if 'data' in vars(e):
                bb.note("bug_event: data was found in vars as dispatched by ...
            else:
                bb.note("bug_event: data was NOT found in vars as dispatched by.
    }

6) While the bug_event.bbclass reports that MetadataEvent.data existed,
the toaster event handlers reports that the metadata events have lost
the 'data' member variable:
    grep assert toaster_ui.log
    assert 'data' in vars(event)
    ...

To reproduce the case where a handler receiving metadata events first
clobbers the 'data' field for subsequent handlers
1) edit the poky/meta/classes/bug_event.bbclass to uncomment
the eventmask line
    - #bug_events[eventmask] = "bb.event.MetadataEvent"
    + bug_events[eventmask] = "bb.event.MetadataEvent"
2) reset toaster and toaster logs
    source toaster stop ; rm toaster* ; source toaster start
3) generate some events
    bitbake quilt-native
    ...
    NOTE: bug_event: data was found in vars as dispatched by execute_handler
    ...
4) the toaster logs still assert MetadataEvent.data is missing:
    grep assert toaster_ui.log
    assert 'data' in vars(event)
    ...
Comment 1 Richard Purdie 2014-05-15 15:04:05 UTC
The data member is only available in certain events. If you look at the server event code, this is added for some classes of events, and not added for others.
Comment 2 Dave Lerner 2014-05-15 15:49:35 UTC
I don't understand your comment.  Should MetadataEvent constructor not setup a 'data' member variable that is potentially destroyed by execute_handler if a bbclass registers for all events?

The way execute_handler is written it assumes that _NO_ event has a 'data' member.

1) If a handler does not set an event mask, then the MetadataEvent will be dispatched to that handler via execute_handler
2) During the dispatch in execute_handler the MetadataEvent.data member is
  a) first overwritten,
  b) then deleted
3) When the MetadataEvent is then sent to the UI handler, the MetadataEvent.data field expected by the toaster handlers has been destroyed and toaster throws an assert.
Comment 3 Richard Purdie 2014-05-22 15:40:08 UTC
Basically, we *cannot* send the data store over serialization. This means that while we can have it available for class events, it cannot be passed over to the UI and therefore it will not be present in the events the UI sees.

We don't want this to happen either, the data store is *huge* and it would be bad to try and serialize and transfer it repeatedly through something like xmlrpc.

So any data store present in events gets killed off by bb.event.execute_handler(), this is by design and isn't something we can easily change.
Comment 4 Richard Purdie 2014-05-22 15:45:10 UTC
Sorry, Paul clarified this. The "data" namespace in events is reserved for the datastore. The MetadataEvent is going to need to use a different name. Assigning to Alex to resolve.
Comment 5 Alexandru Damian 2014-07-18 13:12:20 UTC
Patch sent to mailing lists.
Comment 6 Alexandru Damian 2014-10-28 10:47:09 UTC
Merged and Fixed.