Bug 8150

Summary: Implement whitelisting for packages from imported layers
Product: [Build System, Metadata & Runtime] BitBake Reporter: Alexander Kanevskiy <alexander.kanevskiy>
Component: bitbakeAssignee: Paul Eggleton <bluelightning>
Status: VERIFIED WONTFIX QA Contact: Lucian Musat <georgex.l.musat>
Severity: enhancement    
Priority: High CC: bluelightning, bogdanx.a.voiculescu, jackie.huang, liezhi.yang, patrick.ohly, poky.bs.watcher, poky.watcher, randy.macleod
Version: 1.9   
Target Milestone: 1.9 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Alexander Kanevskiy 2015-08-11 12:44:58 UTC 
In our custom distribution we need to provide enough content in our binary package feed repositories.
To populate those, we would like to use already existing layers like meta-oe from OpenEmbedded.org, but downside is that those repositories are providing so many recipes that we don't need to build or want to ship and respectively don't allow developer to build (unsupported set of recipes).

Using blacklisting feature is not really possible, as package set can be often changed and list would grow fast.
Thus needed functionality with behaviour  "skip all recipes in layer XXX except ones explicitly allowed".
Those options can be located in distro specific configuration files, and should be layer specific. E.g. allow package "gtest" from custom layer, but skip it from "meta-oe".
Comment 1 Paul Eggleton 2015-08-13 15:07:23 UTC 
FWIW, a function was added to bitbake's lib/bb/utils.py in order to make this possible - see bug 7723. With that it's pretty easy to add a class that you can inherit globally that raises SkipRecipe if the recipe is from a particular layer and isn't whitelisted.
Comment 2 Alexander Kanevskiy 2015-08-13 15:16:25 UTC 
Yes, bug 7723 was prework for this feature. It shouldn't be hard to implement as we already discussed offline with Richard and Paul. However, I've created that item specifically to get proper design on how to implement it from ideological point of view and make feature really usable.

E.g. how to name variables, how to make sure that it can be properly controlled on distro configuration level, etc.
Comment 3 Randy MacLeod 2015-08-13 21:27:06 UTC 
We wrote a script to replay all commits and output a white-listed subset of a layer, preserving the original commit id in the long log. 

That works but it's slow, tedious and doesn't let naive users easily discover what they _could_ add without needing to clone meta-oe or find and use the layers index. We may drop the meta-oe-subset approach if a good white-list work-flow can be agreed upon.
Comment 4 Paul Eggleton 2015-08-14 11:21:31 UTC 
I've put together an initial implementation here:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=paule/whitelist
Comment 5 Alexander Kanevskiy 2015-08-14 14:03:38 UTC 
Looks good enough for me to get to community review and initial testing.
Comment 6 Jackie Huang 2015-08-17 10:59:30 UTC 
I tried with the patch but it doesn't work with multilib and -native cases, you may need something like:

-            if not d.getVar('PN', True) in whitelist.split():
+            if not d.getVar('BPN', True) in whitelist.split():

or other way to properly handle multilib.
Comment 7 Paul Eggleton 2015-08-19 13:35:47 UTC 
OK, I've switched to BPN, tweaked the comments a bit and sent it as an RFC patch to the OE-Core list.
Comment 8 Paul Eggleton 2015-08-27 12:38:57 UTC 
Have sent a v2 with some changes:

http://lists.openembedded.org/pipermail/openembedded-core/2015-August/109520.html
Comment 9 Paul Eggleton 2015-09-15 09:26:47 UTC 
The patch implementing this has been rejected; the community would rather not support this kind of thing in OE-Core. Distro layers can carry and use the class if needed though.
Comment 10 Lucian Musat 2015-10-14 07:39:53 UTC 
Setting as verified as per last comment.