10676 2016-11-16 08:19:54 +0000 cooker: event queue overflowed 2016-12-14 21:21:01 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple RESOLVED FIXED Medium+ normal 2.3 M1 1 patrick.ohly richard.purdie jose.perez.carranza poky.bs.watcher poky.watcher davidx.lopez.barriba --- --- No (bug/feature does not impact docs) oldest_to_newest 68352 0 patrick.ohly 2016-11-16 08:19:54 +0000 When producing a lot of output from shell commands which run in parallel (see bug #10668) as part of a task, I ended up with this failure: ERROR: Running idle function Traceback (most recent call last): File "/work/ostro-os/bitbake/lib/bb/server/process.py", line 142, in ProcessServer.idle_commands(delay=0.1, fds=[<multiprocessing.connection.Connection object at 0x7f67d0fe65f8>, <multiprocessing.connection.Connection object at 0x7f67d0b37080>]): try: > retval = function(self, data, False) if retval is False: File "/work/ostro-os/bitbake/lib/bb/cooker.py", line 211, in _process_inotify_updates(server=<ProcessServer(ProcessServer-1, started)>, notifier_list=[<pyinotify.Notifier object at 0x7f67d0b3d6a0>, <pyinotify.Notifier object at 0x7f67d0b3d7f0>], abort=False): n.read_events() > n.process_events() return 1.0 File "/work/ostro-os/bitbake/lib/pyinotify.py", line 1275, in Notifier.process_events(): else: > self._default_proc_fun(revent) self._sys_proc_fun.cleanup() # remove olds MOVED_* events records File "/work/ostro-os/bitbake/lib/bb/cooker.py", line 262, in BBCooker.notifications(event=<Event mask=0x4000 maskname=IN_Q_OVERFLOW >): def notifications(self, event): > if not event.pathname in self.inotify_modified_files: self.inotify_modified_files.append(event.pathname) AttributeError: 'Event' object has no attribute 'pathname' IN_Q_OVERFLOW is "Event queue overflowed". It seems reasonable that such an event doesn't have a pathname. Not sure what bitbake can do in this case. At least it shouldn't crash as above. 68353 1 patrick.ohly 2016-11-16 09:05:43 +0000 Handling a shell command like echo foo | sed -u 's/^/bar: /' & wait $! required a slightly more complex workaround. In that case, the offending token is a list of three tuples. Here's the updated workaround which handles that. Changing indention makes it fairly large. The basic idea is that if a token is a list, process_token_list will be called recursively for the members of that list. diff --git a/bitbake/lib/bb/codeparser.py b/bitbake/lib/bb/codeparser.py index 41484ec..8673b77 100644 --- a/bitbake/lib/bb/codeparser.py +++ b/bitbake/lib/bb/codeparser.py @@ -389,25 +389,24 @@ class ShellParser(): "case_clause": case_clause, } - for token in tokens: - try: + def process_token_list(tokens): + for token in tokens: + if isinstance(token, list): + process_token_list(token) + continue name, value = token - except ValueError: - if isinstance(token, list) and len(token) == 1: - bb.warn('Unexpected token (need list or tuple with two entries): %s' % token) - name, value = token[0] - else: - bb.fatal('Cannot split %s into name and value (need list or tuple with two entries)' % token) - try: - more_tokens, words = token_handlers[name](value) - except KeyError: - raise NotImplementedError("Unsupported token type " + name) + try: + more_tokens, words = token_handlers[name](value) + except KeyError: + raise NotImplementedError("Unsupported token type " + name) + + if more_tokens: + self.process_tokens(more_tokens) - if more_tokens: - self.process_tokens(more_tokens) + if words: + self.process_words(words) - if words: - self.process_words(words) + process_token_list(tokens) def process_words(self, words): """Process a set of 'words' in pyshyacc parlance, which includes 68354 2 patrick.ohly 2016-11-16 09:06:11 +0000 Oops, wrong bug. Please ignore comment #1. 68928 3 richard.purdie 2016-12-08 14:09:07 +0000 http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=e10da7d9dfc4da714092a63987d564ef67f73718