10886 2017-01-06 11:13:15 +0000 Improve event documentation 2017-01-24 00:10:07 +0000 1 1 9 Documentation BitBake User Manual bitbake-manual unspecified x86 Multiple RESOLVED FIXED 23 Jan 2017: RESOLVED Medium normal 2.3 M2 1 richard.purdie srifenbark patrick.ohly --- --- Done (doc changes complete) oldest_to_newest 69565 0 richard.purdie 2017-01-06 11:13:15 +0000 The current "Events" section of the bitbake manual is a bit weak and ambiguous and the example is actually buggy. I'd propose we revise it to read as follows: """ BitBake allows installation of event handlers within recipe and class files. Events are triggered at certain points during operation, such as the beginning of operation against a given .bb, the start of a given task, task failure, task success, and so forth. The intent is to make it easy to do things like email notification on build failure. Following is an example event handler that prints the name of the event and the content of the FILE variable: addhandler myclass_eventhandler python myclass_eventhandler() { from bb.event import getName from bb import data print("The name of the Event is %s" % getName(e)) print("The file we run for is %s" % d.getVar('FILE')) } myclass_eventhandler[eventmask] = "bb.event.BuildStarted bb.event.BuildCompleted" In this example, an eventmask has been set so that the handler only sees "BuildStarted" and "BuildCompleted" events. This event handler gets called every time an event matching the eventmask is triggered. A global variable "e" is defined which represents the current event. With the getName(e) method, one can get the name of the triggered event. The global datastore is available as "d". In old code, "e.data" may be used but it is deprecated and "d" should be used going forward. The context of the datastore will be appropriate to the event in question. For example, "BuildStarted" and "BuildCompleted" events run before any tasks are executed so would be in the global configuration datastore namespace, there will be no recipe specific metadata there. They also run in the main cooker/server process rather than any worker so changes made to the datastore would be seen by other cooker/server events within the current build but not outside of that build or in the workers. Task events run in the actual tasks in question so have recipe and task specific contents. These are run in the worker context and are discarded at the end of task execution. During a standard build, the following common events might occur: bb.event.ConfigParsed() bb.event.ParseStarted() bb.event.ParseProgress() bb.event.ParseCompleted() bb.event.BuildStarted() bb.build.TaskStarted() bb.build.TaskInvalid() bb.build.TaskFailedSilent() bb.build.TaskFailed() bb.build.TaskSucceeded() bb.event.BuildCompleted() bb.cooker.CookerExit() These events are the most common kinds of events that most metadata might have an interest in viewing. There are also events that occur based on specific requests to the server, some common examples are: bb.event.TreeDataPreparationStarted() bb.event.TreeDataPreparationProgress() bb.event.TreeDataPreparationCompleted() bb.event.DepTreeGenerated() bb.event.CoreBaseFilesFound() bb.event.ConfigFilePathFound() bb.event.FilesMatchingFound() bb.event.ConfigFilesFound() bb.event.TargetsTreeGenerated() These are often used to communicate larger pieces of information from the bitbake server to other parts of bitbake such as UIs. """ This still isn't perfect but it documents event scope, the eventmask and fixes the example. 69566 1 richard.purdie 2017-01-06 11:41:47 +0000 Updated event descriptions: bb.event.ConfigParsed() - Fired when the base configuration has been parsed, bitbake.conf, base.bbclass and any global INHERITs. You can see multiple such events, when each of the workers parse the base configuration of if the server changes configuration and reparses. Any given datastore will only have one such event executed agaisnt it however. If BB_INVALIDCONF is set in the datastore by the event handler, the configuration will be reparsed and a new event triggered, allowing the metadata to update configuration. bb.event.ParseStarted() - Fired when bitbake is about to start parsing recipes, has a 'total' attribute which represents the number of recipes bitbake plans to parse. bb.event.ParseProgress() - Fired as parsing progresses, has a 'current' attribute of the number of recipes parsed as well as a 'total' attribute. bb.event.ParseCompleted() - Fired when parsing is complete, has 'cached', 'parsed', 'skipped', 'virtuals', 'masked', and 'errors' attributes giving statistics about the parsing results. bb.event.BuildStarted() - Fired when a new build is started bb.build.TaskStarted() - Fired when a task starts, has a 'taskfile' attribute which points to the recipe the task is from, a 'taskname' attribute for the task's name (including the 'do_' prefix), a 'logfile' attribute pointing to where the task's output is stored and a 'time' attribute for the task execution start time. bb.build.TaskInvalid() - Fired if bitbake tries to execute a task which does not exist. bb.build.TaskFailedSilent() - Fired for setscene tasks which fail and shouldn't be presented to the user verbosely bb.build.TaskFailed() - Fired for normal tasks which fail bb.build.TaskSucceeded() - Fired when a task completes successfully bb.event.BuildCompleted() - Fired when a build has finished bb.cooker.CookerExit() - Fired when the bitbake server/cooker shuts down. Usually only seen by the UIs as a sign they should shutdown too. 69567 2 patrick.ohly 2017-01-06 12:05:02 +0000 Do all of these events fire in the same process? When working on system statistics sampling, it wasn't clear whether writing into the shared files had to be protected against concurrent writes. 69568 3 patrick.ohly 2017-01-06 12:13:50 +0000 (In reply to comment #0) > During a standard build, the following common events might occur: > > bb.event.ConfigParsed() Add here: bb.event.HeartbeatEvent() And as description: bb.event.BuildStarted() - Fires at regular time intervals of 1 second, with the interval configurable with BB_HEARTBEAT_EVENT. Has a 'time' attribute which is the time.time() value when the event was triggered. Useful for activities like system state monitoring. 69661 4 richard.purdie 2017-01-10 17:29:23 +0000 (In reply to comment #2) > Do all of these events fire in the same process? > > When working on system statistics sampling, it wasn't clear whether writing > into the shared files had to be protected against concurrent writes. I was trying to explain that with: """ The context of the datastore will be appropriate to the event in question. For example, "BuildStarted" and "BuildCompleted" events run before any tasks are executed so would be in the global configuration datastore namespace, there will be no recipe specific metadata there. They also run in the main cooker/server process rather than any worker so changes made to the datastore would be seen by other cooker/server events within the current build but not outside of that build or in the workers. Task events run in the actual tasks in question so have recipe and task specific contents. These are run in the worker context and are discarded at the end of task execution. """ not sure if that answers your question or not. If not, suggestions on how to make it clearer would likely help Scott. 69663 5 srifenbark 2017-01-10 18:10:32 +0000 Hi, Here is the link to the updated "Events" section - https://bugzilla.yoctoproject.org/show_bug.cgi?id=10886. Please look it over and let me know of any corrections. Thanks, Scott 69961 6 srifenbark 2017-01-20 13:50:37 +0000 Ooops.. bad link in Comment 5 for the section to review. Here is the right link: http://www.yoctoproject.org/docs/temp/bitbake-user-manual/bitbake-user-manual.html#events Sorry, Scott 69962 7 richard.purdie 2017-01-20 14:06:32 +0000 Looks good. Some quick tweaks: You can drop the "from bb import data" line from the example. "For legacy code, you can use "e.data" to get the name of the triggered event" -> "In legacy code, you may see "e.data" used to get the datastore". " The "taskname" attribute, which includes the do_ prefix," -> "The "taskname" attribute, which is the task's name, includes the do_ prefix," Otherwise looks good to go thanks. 70070 8 srifenbark 2017-01-24 00:10:07 +0000 After Richard reviewed these chanages, all is good once a stray quote character was removed. Setting the status to RESOLVED and putting the doc flag to "done." Scott