The % wildcard (as used in PREFERRED_VERSION and in bbappend filenames) can only match at the end. Whilst this is almost certainly by design, we don't actually state that anywhere in the documentation that I can find - not anywhere we refer to the wildcard usage in the bitbake manual or ref manual, at any rate. We should either change the code to allow the wildcard to be anywhere (easiest I can think of would be to substitute the % with * and use fnmatch); or alternatively we update the documentation so it is clear that there is this limitation.
Scott, could we at least ensure the current docs reflect the current behavior?
Yes - I will stop making things up and try to reflect the behavior accurately :) Of course, I strive to be accurate. Thank you Paul for finding this defect in the docs. Scott
I think we need a little section in the "Syntax and Operators" (file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#bitbake-user-manual-metadata) section of the BitBake manual just for wildcard explanation. My understanding is that you can use two symbols (* and %) for wildcarding. From looking through all the docs, the "*" symbol matches one or more of any character. It also seems that the "%" symbol does the same. How do these work? Specifically, Can I get replies here: 1. How does the "*" character work? And, are there limitations to using it? 2. How does the "%" character work? And, is the limitation Paul pointed out the only limitation? Also, wildcard usage is sporadically reference in both the BitBake manual and the YP Reference Manual: * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#var-BB_ALLOWED_NETWORKS - both in the BB manual and the ref glossary (file:///home/scottrif/yocto-docs/documentation/ref-manual/ref-manual.html#var-BB_ALLOWED_NETWORKS) * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#var-PREFERRED_VERSION. Also, in the YP Ref glossary (file:///home/scottrif/yocto-docs/documentation/ref-manual/ref-manual.html#var-PREFERRED_VERSION) * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#append-bbappend-files * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#locating-and-parsing-recipes * file:///home/scottrif/yocto-docs/documentation/ref-manual/ref-manual.html#general-migration-considerations - loosely mentioned. * file:///home/scottrif/yocto-docs/documentation/ref-manual/ref-manual.html#var-BINCONFIG_GLOB - I don't really understand this one. * file:///home/scottrif/yocto-docs/documentation/ref-manual/ref-manual.html#var-COMPLEMENTARY_GLOB - I don't understand this one either.
This does already appear to be documented in https://www.yoctoproject.org/docs/2.4.1/bitbake-user-manual/bitbake-user-manual.html#var-PREFERRED_VERSION however it doesn't state it only works on the end of the expression. We need to state that the % matching only works on the end of PREFERRED_VERSION. The % match is only for PREFERRED_VERSION too, it is not used in other variables. We don't support * wildcarding in this variable at this time, this bug could be about possibly adding that support although I'm reluctant as it has potential performance implications for parsing. You are correct that there are other variables that use other forms of wildcard or "globing". BB_ALLOWED_NETWORKS only appears to support things starting "*." and does not handle generic wildcards. We should document that explicitly. bbappend file naming uses % to match end of expressions but it only works in the same way PREFERRED_VERSION does, it can only be used at the end (before the final .bbappend extension), not mid way within the value. BBMASK takes regular expressions (see python's re module) BBFILES uses the python glob module syntax "BBFILES is a space-separated list of available files and supports wildcards" should say "BBFILES is a space-separated list of available files and supports wildcards using python's glob module's format" The OE FILES variable uses python glob format COMPLEMENTARY_GLOB uses python's fnmatch (which is similar to glob) BINCONFIG_GLOB uses shell globbing which is basically like glob/fnmatch.
Great information Richard.. .thanks. Here are some changes to look at. First, in the BitBake Manual * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#wildcard-support - A new little section in the BB manual to give some service to wilcarding support in general. * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#var-PREFERRED_VERSION - Updated the PREFERRED_VERSION description. * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#var-BB_ALLOWED_NETWORKS - Updated BB_ALLOWED_NETWORKS description. * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#append-bbappend-files - Updated the section on .bbappend files. * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#var-BBMASK - See the third paragraph. This was mainly in place already. I did some re-wording. * https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#var-BBFILES - Updated the BBFILES variable description. Now for the YP Reference Manual: * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-PREFERRED_VERSION - Updated description of the PREFERRED_VERSION. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-BB_ALLOWED_NETWORKS - Updated the BB_ALLOWED_NETWORKS variable description. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#ref-terms - Updated the "append file" term to include description of limited wildcarding support. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-BBMASK - Updated the BBMASK variable desciption. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-BBFILES - Updated the BBFILES variable description. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-FILES - Updated the FILES variable description. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-COMPLEMENTARY_GLOB - updated COMPLEMENTRAY_GLOB with a note. I don't really understand this variable so I just added the note. * https://yoctoproject.org/docs/2.6/ref-manual/ref-manual.html#var-BINCONFIG_GLOB - added a note to the BINCONFIG-GLOB variable.
In https://yoctoproject.org/docs/2.6/bitbake-user-manual/bitbake-user-manual.html#wildcard-support we should title it "Wildcard support in variables". Then "support for wildcard use varies" -> "support for wildcard use in variables varies". You can link to https://docs.python.org/3/library/re.html#re for "re" and the links need to have "3" in them for python3, not python2. The wording of the second paragraph is horrible and doesn't read well. "Where a variable has some form of wildcard suport, the documentation will describe which form of wildcards, its use and limitations. For BB_ALLOWED_NETWORKS, the only supported form is "*.foo.bar" so "*aa.foo.bar" will not work, it has to start "*.". For BBMASK, we should point at python3, not 2.3 which is older than anything we support now. For BBFILES, need python 3 link. Looking through the rest I think we're ok, as long as all the python2 links are moved to python3 ones.
Richard, Thanks for the feedback. * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#wildcard-support-in-variables * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#var-BB_ALLOWED_NETWORKS * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#var-BBMASK * file:///home/scottrif/bitbake/doc/bitbake-user-manual/bitbake-user-manual.html#var-BBFILES
Thanks Scott, the bitbake manual looks better now!
Thanks Richard, Marking RESOLVED and setting doc flag to "done". Scott