Bug 12747 - PACKAGE_EXCLUDE should also be able to uninstall complementary packages
Summary: PACKAGE_EXCLUDE should also be able to uninstall complementary packages
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 2.6
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: Future
Assignee: Unassigned
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2018-05-14 19:07 UTC by Chen Qi
Modified: 2018-10-17 01:46 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chen Qi 2018-05-14 19:07:46 UTC
For now PACKAGE_EXCLUDE can only exclude packages that is in the initial installation list.
But what normal users would expect from PACKAGE_EXCLUDE is that as long as a package is listed in this variable and there's no dependency problem, it could be correctly excluded.

This requirement is from Yocto mailing list.
Some user wants to exclude acl-ptest package when has has enabled *-ptest complementary packages.
I realize that current implementation of PACKAGE_EXCLUDE cannot handle such situation.
I think we need to change this situation.
Comment 1 Jagadeesh Krishnanjanappa 2018-08-14 05:18:03 UTC
We have PACKAGE_EXCLUDE_COMPLEMENTARY to exclude complementary package from installing.

Adding below line in local.conf, prevents acl-ptest to be excluded.

PACKAGE_EXCLUDE_COMPLEMENTARY = "acl"


Also, If PACKAGE_EXCLUDE = "acl" is used, then the installed packages list does not have acl entry (provided there is no package pulls in acl via RDEPENDS), and most likely the acl complementary packages are not installed without PACKAGE_EXCLUDE_COMPLEMENTARY setting.

Regards,
Jagadeesh
Comment 2 Chen Qi 2018-08-15 02:07:53 UTC
PACKAGE_EXCLUDE_COMPLEMENTARY can only exclude all complementary pkgs. What the user wants is exclude one of them. For example, the user wants acl-ptest to be excluded while all other complementary packages are installed.

In fact PACKAGE_EXCLUDE_COMPLEMENTARY works by 'not installing' instead 'removing'. That is, it takes effect at install time. In contrast, PACKAGE_EXCLUDE works as 'removing'. That is, it takes effect by trying to remove some package.

I'd like to copy the original email from yocto mailing list below. Please check.

I just checked the codes. I think the ref manual might be a little misleading.
"Prevents specific packages from being installed when you are installing complementary packages. "
might better be changed to:
"Prevents specific packages to install their complementary packages. Items specified by this variable are considered as regular expression."
Maybe an example should follow.

So specifying value 'acl' for PACKAGE_EXCLUDE_COMPLEMENTARY should be valid and its complementary packages should not be installed. Specifying 'acl.*' should have the same effect, but 'acl.' should not.

Anyway, I think you should file a bug with steps to reproduce the problem.

Also, I found that we currently don't have a mechanism to exclude some specific complementary package (e.g. acl-ptest in your case). We might need to reconsider what PACKAGE_EXCLUDE should mean.
More specifically, if a user requires *-ptest packages in general, but wants to exclude acl-ptest package, there's no easy way to do so.

Best Regards,
Chen Qi

On 05/09/2018 11:30 PM, Erik Nellessen wrote:
> I would like to exclude some ptest packages from an image build.
>
> To include ptest packages in general, my image recipe contains the following line:
> IMAGE_FEATURES_append = " ptest-pkgs"
>
> As a first step, I tried to exclude the acl-ptest package. To do so, I added the following to my image recipe:
> PACKAGE_EXCLUDE_COMPLEMENTARY = "acl-ptest"
>
> I thought that this would exclude the acl-ptest package as described in the project reference: https://www.yoctoproject.org/docs/2.4.2/ref-manual/ref-manual.html#var-PACKAGE_EXCLUDE_COMPLEMENTARY
> "Prevents specific packages from being installed when you are installing complementary packages.
>
> You might find that you want to prevent installing certain packages when you are installing complementary packages. For example, if you are using IMAGE_FEATURES to install dev-pkgs, you might not want to install all packages from a particular multilib. If you find yourself in this situation, you can use the PACKAGE_EXCLUDE_COMPLEMENTARY variable to specify regular expressions to match the packages you want to exclude."
>
> Anyhow this did not result in an image without the acl-ptest package, as I could validate by having a look at the image's manifest file.
>
> When I changed the image recipe to contain the regular expression "acl", i.e
> PACKAGE_EXCLUDE_COMPLEMENTARY = "acl"
> all three acl packages (acl, acl-lic, acl-ptest) were excluded from the image.
>
> And to really start the confusion, when changed the regular expression to "acl.", i.e.
> PACKAGE_EXCLUDE_COMPLEMENTARY = "acl."
> all three packages were also excluded.
>
> Now I am really confused. My expectation was that "acl-ptest" and "acl" would match the acl-ptest package name and "acl." would not match the acl package name.
>
> Does anybody see what I am missing here?
>
> Thanks in advance,
> Erik
Comment 3 Jagadeesh Krishnanjanappa 2018-08-15 15:19:06 UTC
(In reply to comment #2)
> PACKAGE_EXCLUDE_COMPLEMENTARY can only exclude all complementary pkgs. What
> the user wants is exclude one of them. For example, the user wants acl-ptest
> to be excluded while all other complementary packages are installed.
> 
> In fact PACKAGE_EXCLUDE_COMPLEMENTARY works by 'not installing' instead
> 'removing'. That is, it takes effect at install time. In contrast,
> PACKAGE_EXCLUDE works as 'removing'. That is, it takes effect by trying to
> remove some package.
> 
> I'd like to copy the original email from yocto mailing list below. Please
> check.
> 
> I just checked the codes. I think the ref manual might be a little
> misleading.
> "Prevents specific packages from being installed when you are installing
> complementary packages. "
> might better be changed to:
> "Prevents specific packages to install their complementary packages. Items
> specified by this variable are considered as regular expression."
> Maybe an example should follow.
> 
> So specifying value 'acl' for PACKAGE_EXCLUDE_COMPLEMENTARY should be valid
> and its complementary packages should not be installed. Specifying 'acl.*'
> should have the same effect, but 'acl.' should not.
> 

Basically, when "acl" is added into PACKAGE_EXCLUDE_COMPLEMENTARY, it gets added into skipval variable of scripts/oe-pkgdata-util. So the resulting skipval would be '-locale-|^locale-base-|-dev$|-doc$|-dbg$|-staticdev$|^kernel-module-|acl'.
This value is used as a regular expression pattern to match/search complementary packages. That means any package which contains "acl" string, their corresponding complementary packages are excluded i.e acl and libacl1 complementary packages are excluded. 

That's the reason why acl,acl-dev,acl-dbg,acl-doc,acl-ptest etc.. are excluded when PACKAGE_EXCLUDE_COMPLEMENTARY = "acl".

This can be verified using below commands,

With acl in exclude:
oe-pkgdata-util -d -p /opt/work_jagadeesh/poky_master/build/tmp/pkgdata/qemux86 glob /tmp/inst *-ptest *-dev *-dbg --exclude="acl" 2>&1 | grep acl
DEBUG: acl -> !!
DEBUG: libacl1 -> !!

In the above scenario, none of the acl complementary gets installed.

Without acl in exclude:
oe-pkgdata-util -d -p /opt/work_jagadeesh/poky_master/build/tmp/pkgdata/qemux86 glob /tmp/inst *-ptest *-dev *-dbg --exclude="acl-ptest" 2>&1 | grep acl
DEBUG: acl (*-ptest) -> acl-ptest
DEBUG: acl (*-dev) -> acl-dev
DEBUG: acl (*-dbg) -> acl-dbg
DEBUG: libacl1 (*-ptest) -> acl-ptest
DEBUG: libacl1 (*-dev) -> acl-dev
DEBUG: libacl1 (*-dbg) -> acl-dbg

In the above scenario, acl-ptest, acl-dev, and acl-dbg gets installed.

> Anyway, I think you should file a bug with steps to reproduce the problem.
>
So the logic of regular expression in complementary package is correct.
 
> Also, I found that we currently don't have a mechanism to exclude some
> specific complementary package (e.g. acl-ptest in your case). We might need
> to reconsider what PACKAGE_EXCLUDE should mean.
> More specifically, if a user requires *-ptest packages in general, but wants
> to exclude acl-ptest package, there's no easy way to do so.
>
Actually if we want to exclude "acl-ptest", then PACKAGE_EXCLUDE = "acl-ptest" will do the job.


Please let me know if queries.

Regards,
Jagadeesh 

> Best Regards,
> Chen Qi
> 
> On 05/09/2018 11:30 PM, Erik Nellessen wrote:
> > I would like to exclude some ptest packages from an image build.
> >
> > To include ptest packages in general, my image recipe contains the following line:
> > IMAGE_FEATURES_append = " ptest-pkgs"
> >
> > As a first step, I tried to exclude the acl-ptest package. To do so, I added the following to my image recipe:
> > PACKAGE_EXCLUDE_COMPLEMENTARY = "acl-ptest"
> >
> > I thought that this would exclude the acl-ptest package as described in the project reference: https://www.yoctoproject.org/docs/2.4.2/ref-manual/ref-manual.html#var-PACKAGE_EXCLUDE_COMPLEMENTARY
> > "Prevents specific packages from being installed when you are installing complementary packages.
> >
> > You might find that you want to prevent installing certain packages when you are installing complementary packages. For example, if you are using IMAGE_FEATURES to install dev-pkgs, you might not want to install all packages from a particular multilib. If you find yourself in this situation, you can use the PACKAGE_EXCLUDE_COMPLEMENTARY variable to specify regular expressions to match the packages you want to exclude."
> >
> > Anyhow this did not result in an image without the acl-ptest package, as I could validate by having a look at the image's manifest file.
> >
> > When I changed the image recipe to contain the regular expression "acl", i.e
> > PACKAGE_EXCLUDE_COMPLEMENTARY = "acl"
> > all three acl packages (acl, acl-lic, acl-ptest) were excluded from the image.
> >
> > And to really start the confusion, when changed the regular expression to "acl.", i.e.
> > PACKAGE_EXCLUDE_COMPLEMENTARY = "acl."
> > all three packages were also excluded.
> >
> > Now I am really confused. My expectation was that "acl-ptest" and "acl" would match the acl-ptest package name and "acl." would not match the acl package name.
> >
> > Does anybody see what I am missing here?
> >
> > Thanks in advance,
> > Erik
Comment 4 Jagadeesh Krishnanjanappa 2018-08-15 15:26:53 UTC
Further to comment 3:

Without acl:
oe-pkgdata-util -d -p /opt/work_jagadeesh/poky_master/build/tmp/pkgdata/qemux86 glob /tmp/inst *-ptest *-dev *-dbg 2>&1 | grep acl
DEBUG: acl (*-ptest) -> acl-ptest
DEBUG: acl (*-dev) -> acl-dev
DEBUG: acl (*-dbg) -> acl-dbg
DEBUG: libacl1 (*-ptest) -> acl-ptest
DEBUG: libacl1 (*-dev) -> acl-dev
DEBUG: libacl1 (*-dbg) -> acl-dbg
acl-dev
acl-dbg
acl-ptest

In the above scenario, acl-ptest, acl-dev, and acl-dbg gets installed.

Regards,
Jagadeesh
Comment 5 Chen Qi 2018-08-16 01:59:27 UTC
You are right.
I think we can close this bug.
Comment 6 Manish Das 2018-10-17 01:34:52 UTC
As a test, below is observation for connman  package with PACKAGE_EXCLUDE and PACKAGE_EXCLUDE_COMPLEMENTARY:

1. It is removing all connman package (connman, connman-dev, connman-dbg etc.) as per mentione below  in local.conf

EXTRA_IMAGE_FEATURES_append = " dev-pkgs dbg-pkgs"
IMAGE_INSTALL_append = " connman"

PACKAGE_EXCLUDE = "connman"

2. It removing only mentioned connman package e.i. connman-dev as per mentioned below in local.conf

EXTRA_IMAGE_FEATURES_append = " dev-pkgs dbg-pkgs"
IMAGE_INSTALL_append = " connman"

PACKAGE_EXCLUDE = "connman-dev"


3. It is removing complementary packages of connman (connman-dev, connman-dbg). It will not remove main connman package (connman) ,as per mentioned below in local.conf

EXTRA_IMAGE_FEATURES_append = " dev-pkgs dbg-pkgs"
IMAGE_INSTALL_append = " connman"

PACKAGE_EXCLUDE_COMPLEMENTARY = "connman"


4. It is removing complementary packages of connman (connman-dev, connman-dbg). It will not remove main connman package (connman) ,as per mentioned below in local.conf

EXTRA_IMAGE_FEATURES_append = " dev-pkgs dbg-pkgs"
IMAGE_INSTALL_append = " connman"

PACKAGE_EXCLUDE_COMPLEMENTARY = "connman-*"


5. It is installing all packages of connman (connman, connman-dev, connman-dbg) as per mentioned below in local.conf
EXTRA_IMAGE_FEATURES_append = " dev-pkgs dbg-pkgs"
IMAGE_INSTALL_append = " connman"

PACKAGE_EXCLUDE_COMPLEMENTARY = "connman-dbg"
OR
PACKAGE_EXCLUDE_COMPLEMENTARY = "connman-dev"


Conclusion as per above testing:

PACKAGE_EXCLUDE_COMPLEMENTARY: It will remove  only all complementary packages like (-dbg,-dev etc..) of a sepcified package name but it will not remove the main package('conman' as per above example).
And it works with regular expression also i.e  PACKAGE_EXCLUDE_COMPLEMENTARY = "connman-*" By this regular expression it removes only complementray package. 

PACKAGE_EXCLUDE: It removes all packages of specified package with PACKAGE_EXCLUDE if we want to exclude specific complimentary package(-dbg,-dev etc..) we need to mention  the name of complementary  package 
For  example to remove complementary  "connman-dev" package of connman write like this ( PACKAGE_EXCLUDE = "connman-dev")
if we mention  as PACKAGE_EXCLUDE = "connnman" then it will remove  all main package(connman as per this example) as well as complementary package also (like connnman-dev,connnman-dbg etc)

In the light of the observations, Chen Qi may proceed to close the bug.
Comment 7 Chen Qi 2018-10-17 01:46:34 UTC
Close this bug as there's no problem in current product.