Bug 11427

Nothing stands out as being clearly of...but I am suspecting the root cause may be a behavior change between internalsolv and libsolv: Provides dependencies that have version constrains have to be real packages. The libsolv behavior is more restrictive, but is the correct one by Debian policy [1].

Seems like opkg is failing when installing task-core-boot. A way to get more info would be to try to manually run the opkg install command on task-core-boot [2]. I would assume that will fail on keymaps. Then the idea would be to try to install keymaps manually, so we can get down the tree chain (and so on).


[1] https://www.debian.org/doc/debian-policy/ch-relationships.html (Provides section)
[2] /home/hfreyther/images/poky-resurrection/new/tmp/work/bsc_v2-poky-linux/image-rauc-rescue-initramfs/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /home/hfreyther/images/poky-resurrection/new/tmp/work/bsc_v2-poky-linux/image-rauc-rescue-initramfs/1.0-r0/opkg.conf -t /home/hfreyther/images/poky-resurrection/new/tmp/work/bsc_v2-poky-linux/image-rauc-rescue-initramfs/1.0-r0/temp/ipktemp/ -o /home/hfreyther/images/poky-resurrection/new/tmp/work/bsc_v2-poky-linux/image-rauc-rescue-initramfs/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   install task-core-boot

so you would like to have "opkg -V4 install keymaps"? What are you looking for?

yes, if you can go down the chain would be great. So, try to install task-core-boot first. If that fails because keymaps can't be installed, then try to install keymaps. If keymaps fail because X , then try X. I am basically trying to see what's the dependency that can't be satisified.

yes, if you can go down the chain would be great. So, try to install task-core-boot first. If that fails because keymaps can't be installed, then try to install keymaps. If keymaps fail because X , then try X. I am basically trying to see what's the dependency that can't be satisified.

Am I missing something but isn't it printed?

task-core-boot -> keymaps -> kbd (kbd can't as it seems to have been eliminated, merged with itself?)?

Nope, you are not missing anything. I mainly wanted to see at which node the dependency fails, but I guess it's clear it's on keymaps. Can you paste the control file of keymaps then?

also, instructions on how to reproduce the problem can go a long way. If you could provide me with those, it will be simpler for me to root cause what's going on.

Sorry. I didn't have the time to try the rootfs by hand.

1.) Get your pyro poky
2.) Get the BSP
git clone git://git.sysmocom.de/poky/meta-sysmocom-bsp/
3.) Use "sysmocom-bsc-v2" as MACHINE type
4.) build "image-rauc-rescue-initramfs"

(there are other layers like meta-telephony but they should not be required here

I was able to reproduce the problem using both dnf and opkg. After digging a bit more:

adelcast@delcastillo2 ~/upstream/poky/meta-sysmocom-bsp $ grep -R BAD_RECOMMENDATIONS | grep kbd
recipes-core/images/image-rauc-rescue-initramfs.bb:BAD_RECOMMENDATIONS_append = " busybox-syslog kbd kbd-consolefonts kbd-keymaps"

removing the BAD_RECOMMENDATIONS fixes the issue. According to the yocto manual [1]:

"It is important to realize that if you choose to not install packages using this variable and some other packages are dependent on them (i.e. listed in a recipe's RDEPENDS variable), the OpenEmbedded build system ignores your request and will install the packages to avoid dependency errors."

which confused me a bit, since packagegroup-core-boot depends on keymaps, which depends on kbd. However, that seems to be a documentation/bug issue with OE, not with opkg, so I am closing this ticket as not a bug. Feel free to re-open if you experience something different that points to opkg directly.


[1] http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html

(In reply to comment #9)
> I was able to reproduce the problem using both dnf and opkg. After digging a
> bit more:
> 
> adelcast@delcastillo2 ~/upstream/poky/meta-sysmocom-bsp $ grep -R
> BAD_RECOMMENDATIONS | grep kbd
> recipes-core/images/image-rauc-rescue-initramfs.bb:
> BAD_RECOMMENDATIONS_append = " busybox-syslog kbd kbd-consolefonts
> kbd-keymaps"


Oh! How did you debug that? Maybe this is something for opkg to ease? E.g. I didn't notice anything related to BAD_RECOMMENDATIONS in the output.


> "It is important to realize that if you choose to not install packages using
> this variable and some other packages are dependent on them (i.e. listed in
> a recipe's RDEPENDS variable), the OpenEmbedded build system ignores your
> request and will install the packages to avoid dependency errors."
> 
> which confused me a bit, since packagegroup-core-boot depends on keymaps,
> which depends on kbd. However, that seems to be a documentation/bug issue
> with OE, not with opkg, so I am closing this ticket as not a bug. Feel free
> to re-open if you experience something different that points to opkg
> directly.

I disagree here. Te above says "ignores your request" but with libresolv it will fail. I don't mind it failing (it is probably an improvement) but it should lead to an error that is easier to understand?

(In reply to comment #10)
> (In reply to comment #9)
> > I was able to reproduce the problem using both dnf and opkg. After digging a
> > bit more:
> > 
> > adelcast@delcastillo2 ~/upstream/poky/meta-sysmocom-bsp $ grep -R
> > BAD_RECOMMENDATIONS | grep kbd
> > recipes-core/images/image-rauc-rescue-initramfs.bb:
> > BAD_RECOMMENDATIONS_append = " busybox-syslog kbd kbd-consolefonts
> > kbd-keymaps"
> 
> 
> Oh! How did you debug that? Maybe this is something for opkg to ease? E.g. I
> didn't notice anything related to BAD_RECOMMENDATIONS in the output.

The installation succeeded if I manually deleted the file that keep package status (/var/lib/opkg/status) and manually tried to install keymaps. Looking at that file, I noticed that kbd status was set to "deinstall hold not-installed", which is not a common status. Grepping on oe-core found that in oe-core at lib/oe/package_manager.py OE is manually modifying the status file in the case of BAD_RECOMMENDATIONS. 

Opkg (and dnf) are correctly identifying the conflict, so I don't think there is anything for opkg to do. As I mentioned, if this worked on the internal solver, then this probably means OE needs to change it's BAD_RECOMMENDATIONS code to have it worked as before.

> > "It is important to realize that if you choose to not install packages using
> > this variable and some other packages are dependent on them (i.e. listed in
> > a recipe's RDEPENDS variable), the OpenEmbedded build system ignores your
> > request and will install the packages to avoid dependency errors."
> > 
> > which confused me a bit, since packagegroup-core-boot depends on keymaps,
> > which depends on kbd. However, that seems to be a documentation/bug issue
> > with OE, not with opkg, so I am closing this ticket as not a bug. Feel free
> > to re-open if you experience something different that points to opkg
> > directly.
> 
> I disagree here. Te above says "ignores your request" but with libresolv it
> will fail. I don't mind it failing (it is probably an improvement) but it
> should lead to an error that is easier to understand?

I do believe BAD_RECOMMENDS is not working as advertised, but that is not an opkg issue (it also fails with dnf), but an OE issue. I am not saying that the request should be ignored, what I am saying is that it shouldn't be tagged as an opkg issue but as an OE issue.

Reopen against OE-Core as there is likely an issue here.

(In reply to comment #12)
> Reopen against OE-Core as there is likely an issue here.

FWIW, we're seeing this too.

A package is listed in BAD_RECOMMENDATIONS and when I add it in a packagegroup (the packagegroup RDEPENDS on that package), I get the same failure.

I took a look at the code again, and looks to me like the current implementation for opkg is a bit hackish (it manually pins the bad recommends package to be deinstalled). Looking at dnf, it uses an --exclude flag to signal BAD_RECOMMENDS packages. I haven't tried this myself, but I am guessing those excludes will be ignored if there is a hard dependency.

I would be open to implement an --excludes flag in opkg (or at least give it a shot, if libsolv provides a way to signal that a package should be excluded, unless it's a hard dependency). If that pans out, we could keep the solution that changes the status file directly for opkg internalsolver, but have the --exclude flag solution for libsolv.

Ross, let me know your thoughts to see if I show move ahead and explore adding the --exclude option.

--exclude sounds like a much better solution than hacking the status file.  Did you look at what dpkg/apt does?  I believe you can do something like:

$ apt-get install foo thing-that-foo-recommends- 

To install foo but not thing-that-foo-recommends (the trailing hyphen being the trick).

Today, I finally decided to take another stab at this.

Looking into opkg, I realized that it already have a -add-exclude flag that does exactly the same thing as dnf --excludes. However, neither of those flags solve the BAD_RECOMMENDATIONS issue as they unconditionally exclude the listed packages. I ran a test where I included in an image package A and B, where A recommends C and B depends on C. I added a BAD_RECOMMENDATIONS=C. The expected behavior was to have package A, B and C installed (BAD_RECOMMENDATIONS would be ignored, since C depends on B). Instead, I got a solver "nothing provides C" error on both dnf and opkg.

At this point I am not sure how to proceed forward as it looks like we may need a libsolv way to implement something similar to smart's ignore-recommends per-package flag. Maybe we can loop in the dnf mantainer to decide on the best way forward for both opkg and dnf, since this is a feature request for both package managers? any other ideas?

I prototyped a solution that adds a new --add-ignore-recommends option to opkg. That option is based on the libsolv solver flag SOLVER_DISFAVOR. The problem is that SOLVER_DISFAVOR doesn't work in conjunction with SOLVER_FLAG_STRONG_RECOMMENDS, which opkg uses to enforce recommended packages upgrades. I opened a bug ticket to libsolv [1]. This ticket is blocked until the libsolv ticket is solved.

[1] https://github.com/openSUSE/libsolv/issues/254

The libsolv fix just landed on master. I tested with my prototype of --add-ignore-recommends and it worked, so we have a path forward. I am going to wait until libsolv releases (0.7.3) to submit the new opkg feature (--add-ignore-recommends flag). Once that is in, we need to put the patches on OE, then modify OE BAD_RECOMMENDATIONS code to use the new flag.

Commit 0d11e813ba9b4e8de9e6e5099ff85f5d914243bc replaces the current BAD_RECOMMENDATIONS implementation with the one based on the new opkg flag --add-ignore-recommends. This should fix the issue on this ticket.