10680 2016-11-16 13:04:15 +0000 multiconfig globbing support 2017-03-19 20:22:25 +0000 1 1 7 Build System, Metadata & Runtime BitBake bitbake 2.3 x86 Multiple RESOLVED FIXED Medium+ enhancement 2.2.2 1 richard.purdie juro.bystricky henry.bruce jose.perez.carranza poky.bs.watcher poky.watcher randy.macleod jair.de.jesus.gonzalez.plascencia --- --- Yes (doc changes required) oldest_to_newest 68359 0 richard.purdie 2016-11-16 13:04:15 +0000 Currently you can't build a target for all the configured multiconfigs without specifying a list. This enhancement is to support globbing so that you can do this, e.g. instead of: bitbake multiconfig:A:bash multiconfig:B:bash bash bitbake multiconfig:*:bash 69254 1 juro.bystricky 2016-12-21 01:23:12 +0000 Currently testing this patch: diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index 30131fb..a6841e7 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -666,7 +666,23 @@ class BBCooker: if not task.startswith("do_"): task = "do_%s" % task - fulltargetlist = self.checkPackages(pkgs_to_build, task) + targetlist = self.checkPackages(pkgs_to_build, task) + fulltargetlist = [] + + # Wild card expansion: + # Replace string such as "multiconfig:*:bash" + # into "multiconfig:A:bash multiconfig:B:bash bash" + for k in targetlist: + if k.startswith("multiconfig:") and k.split(":")[1] == "*": + for mc in self.multiconfigs: + if mc: + fulltargetlist.append(k.replace('*', mc)) + # default task + else: + fulltargetlist.append(k.split(":")[2]) + else: + fulltargetlist.append(k) + taskdata = {} localdata = {} 69317 2 richard.purdie 2016-12-22 14:47:48 +0000 Patch looks good to me, please send to the bitbake list and we can get this closed. 69319 3 juro.bystricky 2016-12-22 16:02:02 +0000 Before I send this to the mailing list, I have one suggestion: We do this: BB_MULTICONFIG = "A B" $ bitbake multiconfig:*:bash expands to: $ bitbake multiconfig:A:bash multiconfig:B:bash bash But what to do in this case, something like this: $ bitbake multiconfig:*:bash core-image-minimal I suggest we expand it like this: $ bitbake multiconfig:A:bash multiconfig:B:bash core-image-minimal In other words, if "default" is explicitly specified, we don't override it. This would make the syntax more flexible as well. 69326 4 juro.bystricky 2016-12-22 16:45:51 +0000 In particular, I would find syntax support for this very useful: $ bitbake multiconfig:*:meta-toolchain hosttols (building a set of baremetal toolchains + hostools) 69781 5 juro.bystricky 2017-01-13 00:05:16 +0000 I ran into a few real world use cases where it is desirable to use multiconfig with two different tasks (recipes). For example, I frequently build SDKs with several toolchains + additional host tools. Another use is to build one image for main CPU(s) and a different co-image for a companion CPU. The additional logic is simple: if the user specified the "default" task explicitly, use it, otherwise assume the "default" task to be the same as the multiconfig task(s). Currently supported multiconfig sytax: BBMULTICONFIG = "A B C ..." $ bitbake multiconfig:A:task1 multiconfig:B:task2 multiconfig:C:task3 ... taskN (where task1, task2,...,taskN,... may or may not be the same.) Suported is also any subset of tasks, i.e.: $ bitbake multiconfig:A:task1 $ bitbake multiconfig:A:task1 taskM $ bitbake multiconfig:A:task1 multiconfig:B:task2 $ bitbake taskN Proposed new syntax addition implemented in this patch (two formats): 1. Default task implicit: $ bitbake multiconfig:*:task expands as: $ bitbake multiconfig:A:task multiconfig:B:task multiconfig:C:task ... task 2. Default task explicit $ bitbake multiconfig:*:task1 task2 expands as: $ bitbake multiconfig:A:task1 multiconfig:B:task1 multiconfig:C:task1 ... task2 This patch should allow all of the above: diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index f6bad36..bcd4dca 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -666,7 +666,37 @@ class BBCooker: if not task.startswith("do_"): task = "do_%s" % task - fulltargetlist = self.checkPackages(pkgs_to_build, task) + targetlist = self.checkPackages(pkgs_to_build, task) + fulltargetlist = [] + defaulttask_implicit = '' + defaulttask_explicit = False + wildcard = False + + # Wild card expansion: + # Replace string such as "multiconfig:*:bash" + # into "multiconfig:A:bash multiconfig:B:bash bash" + for k in targetlist: + if k.startswith("multiconfig:"): + if wildcard: + bb.fatal('multiconfig conflict') + if k.split(":")[1] == "*": + wildcard = True + for mc in self.multiconfigs: + if mc: + fulltargetlist.append(k.replace('*', mc)) + # implicit default task + else: + defaulttask_implicit = k.split(":")[2] + else: + fulltargetlist.append(k) + else: + defaulttask_explicit = True + fulltargetlist.append(k) + + if not defaulttask_explicit and defaulttask_implicit != '': + fulltargetlist.append(defaulttask_implicit) + + bb.debug(1,"fulltargetlist: %s" % (str(fulltargetlist))) taskdata = {} localdata = {} 69782 6 juro.bystricky 2017-01-13 00:09:22 +0000 It takes forever to test this properly.... 69822 7 juro.bystricky 2017-01-15 20:42:44 +0000 sent patch to the mailing list: http://lists.openembedded.org/pipermail/bitbake-devel/2017-January/008395.html 70046 8 juro.bystricky 2017-01-23 20:30:45 +0000 merged with master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=7d18d05d451d4a69d827f7f78e7e17b69f950269 71323 9 henry.bruce 2017-03-15 22:13:41 +0000 multiconfig is a powerful new feature whose usability (and thus rate of adoption) is greatly enhanced by the globbing feature. Requesting backport for morty so that those not ready to jump on pyro can benefit form this enhancement. 71360 10 sjolley.yp.pm 2017-03-16 15:33:00 +0000 Triage team agreed to backport to 2.2.2. 71436 11 juro.bystricky 2017-03-19 20:22:25 +0000 Backported to morty: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?h=morty&id=ca66b08b3b7254dea23d1a8ac9eedbe2396e7f12