10945 2017-01-20 10:39:56 +0000 data_smart.py: _remove causes extra work 2018-10-18 12:42:17 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake unspecified x86 Multiple RESOLVED FIXED Medium enhancement 4.99 1 patrick.ohly richard.purdie poky.bs.watcher poky.watcher --- --- No (bug/feature does not impact docs) oldest_to_newest 69954 0 patrick.ohly 2017-01-20 10:39:56 +0000 getVarFlag() does some work related to _remove and the entire _remove multiple times. This is not wrong, but might have a performance impact. There are two reasons for this: - applying the split/join when there is nothing to remove - removing when parsing=True This can be reproduced with "bitbake -e qemu-native" and an entry in local.conf: SRC_URI_remove_pn-foobar = "removeme" It happens to break qemu-native because of an invalid _prepend without trailing space - that's how I noticed. Here's a patch which might fix the issues. According to Richard, this needs good testing (bitbake selftest, parsing OE) because he has a feeling that getVarFlags() behaves this way for a reason. diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py index 79d591a23..16840069b 100644 --- a/lib/bb/data_smart.py +++ b/lib/bb/data_smart.py @@ -772,7 +772,7 @@ class DataSmart(MutableMapping): cachename = var + "[" + flag + "]" value = self.expand(value, cachename) - if value and flag == "_content" and local_var is not None and "_remove" in local_var: + if not parsing and value and flag == "_content" and local_var is not None and "_remove" in local_var: removes = [] self.need_overrides() for (r, o) in local_var["_remove"]: @@ -784,9 +784,10 @@ class DataSmart(MutableMapping): if match: removes.extend(self.expand(r).split()) - filtered = filter(lambda v: v not in removes, - value.split()) - value = " ".join(filtered) + if removes: + filtered = filter(lambda v: v not in removes, + value.split()) + value = " ".join(filtered) if expand and var in self.expand_cache: # We need to ensure the expand cache has the correct value # flag == "_content" here 70007 1 richard.purdie 2017-01-23 14:48:21 +0000 I did take part of the fix: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=45df694a9f472ac2f684aadac4d864c3dfdc48a7 since that part was obviously correct. The other piece (parsing=True) needs further investigation. 81897 2 richard.purdie 2018-10-18 12:42:17 +0000 Remaining optimisation added in http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=58e3f45d4b7d69d663ce7c94933d4da7b5f071e3