Bug 5204 - meta/recipes-support/curl/curl_7.29.0.bb install dependency fails
Summary: meta/recipes-support/curl/curl_7.29.0.bb install dependency fails
Status: RESOLVED WORKSFORME
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: core (show other bugs)
Version: 1.4.2
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.5.1
Assignee: Paul Eggleton
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-09-16 09:54 UTC by Javi Roman
Modified: 2014-01-16 16:32 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Javi Roman 2013-09-16 09:54:40 UTC
Curl bitbake package builds with success:

bitbake -v curl


deploy-rpms/x86_64/curl-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/libcurl4-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/libcurl-dev-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/libcurl-staticdev-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/libcurl-doc-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/curl-dbg-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/curl-dev-7.29.0-r1.x86_64.rpm
deploy-rpms/x86_64/curl-doc-7.29.0-r1.x86_64.rpm

However the installation in the BSP is failing due to a dependency issue:

root@romley:~# smart query *curl*
Loading cache...
Updating cache...                                       ############################################################################# [100%]

curl-7.29.0-r1@x86_64
curl-dbg-7.29.0-r1@x86_64
curl-dev-7.29.0-r1@x86_64
curl-doc-7.29.0-r1@x86_64
libcurl-dev-7.29.0-r1@x86_64
libcurl-doc-7.29.0-r1@x86_64
libcurl-staticdev-7.29.0-r1@x86_64
libcurl4-7.29.0-r1@x86_64


root@romley:~# smart install curl
Loading cache...
Updating cache...                                       ############################################################################# [100%]

Computing transaction...                                                                                                                     error: Can't install libcurl4-7.29.0-r1@x86_64: no package provides curl-certs

The package curl-certs is not built.

The branch affected is:

		git clone -b dylan git://git.yoctoproject.org/poky.git
		git reset --hard 789b2b7e0cca404ff8ba8ed5ba823d215e6ba5b4
Comment 1 Jay See 2013-09-18 14:37:47 UTC
I see two problems here:

- curl-certs is an empty package. there's nothing out of FILES_${PN}-certs = "${datadir}/curl/curl-*"

- there's NO dependency between curl and curl-certs. libcurl _recommends_ curl-certs:

RRECOMMENDS_lib${BPN} += "${PN}-certs"

but I can't find any place where curl is declared to _depend_ on curl-certs.

Are there two different bugs in one here?
Comment 2 Javi Roman 2013-09-19 08:10:11 UTC
It looks like a problem with the "smart" package manager.

If you download curl, and install one using rpm directly, the "Suggest" tag in the SPEC file is working properly.

root@romley:~# smart download curl
Loading cache...
Updating cache...                                       ############################################################################# [100%]

root@romley:~# smart download libcurl4
Loading cache...
Updating cache...                                       ############################################################################# [100%]

root@romley:~# rpm -ih libcurl4-7.29.0-r1.x86_64.rpm 
########################################### [100%]
########################################### [100%]

root@romley:~# rpm -ih curl-7.29.0-r1.x86_64.rpm 
########################################### [100%]
########################################### [100%]

The dependency of curl-certs is avoid in this case, so "smart" the responsible of this problem.
Comment 3 Mark Hatle 2013-09-19 17:24:26 UTC
might be worth trying to reproduce this using either BAD_RECOMMENDATIONS = "curl-certs" or NO_RECOMMENDATIONS = "1"  to see if the issue goes away.
Comment 4 Javi Roman 2013-09-20 09:35:07 UTC
Those variables didn't work for me in a curl bbappend recipe.

(In reply to comment #3)
> might be worth trying to reproduce this using either BAD_RECOMMENDATIONS =
> "curl-certs" or NO_RECOMMENDATIONS = "1"  to see if the issue goes away.
Comment 5 Paul Eggleton 2013-09-25 16:41:54 UTC
I've tried on qemux86 and nuc machines on both dylan and master and have not been able to reproduce this issue - curl installs without error here. I'm not sure what I'm doing differently but clearly I'm not replicating your setup (aside from not being able to test on a romley machine).

Mark, this bug is against dylan so those variables wouldn't be applicable.
Comment 6 Jay See 2013-09-25 19:38:12 UTC
Paul which curl version comes when you try to "smart install" it? May be there were an update somewhere in the mean time ?
Comment 7 Paul Eggleton 2013-09-26 10:08:25 UTC
It's the same version here. The problem here is definitely to do with smart, rpm, or the data being fed into them, rather than curl.

Can you tell me what output you get with the following commands?

 smart query --show-recommends libcurl4

 smart query --show-requires libcurl4

Actually I suspect I know what might be going on. Did you manually run your host distro's version of createrepo on the host rpm repositories? I know we advise people to do that but it occurs to me now that that might not be a good idea because it probably won't index the recommends properly.
Comment 8 Jay See 2013-09-27 07:31:04 UTC
Going to a new build I now have libcurl5 instead of 4? Dunno what I might have done wrong here (I don't see significant update in meta/recipes-support?). 

Anyway, your intuition is right, I'm running createrepo on the rpm directory. Should this be the problem, what are the alternatives?

Here the requested logs:

# smart query --show-recommends libcurl5

libcurl5-7.29.0-r1@armv5te
  Recommends:
    curl-certs

# smart query --show-requires libcurl5

libcurl5-7.29.0-r1@armv5te
  Requires:
    /bin/sh [pre]
    libc.so.6(GLIBC_2.17)
    libc6 >= 2.17 [pre]
    libc6 >= 2.17
    libcap.so.2 [pre]
    libcap.so.2
    libcap2 >= 2.22 [pre]
    libcap2 >= 2.22
    libgcc1 >= 4.7.2 [pre]
    libgcc1 >= 4.7.2
    libgcc_s.so.1 [pre]
    libgcc_s.so.1
    libgcc_s.so.1(GCC_3.5) [pre]
    libgcc_s.so.1(GCC_3.5)
    libgcrypt.so.11 [pre]
    libgcrypt.so.11
    libgcrypt.so.11(GCRYPT_1.2) [pre]
    libgcrypt.so.11(GCRYPT_1.2)
    libgcrypt11 >= 1.5.0 [pre]
    libgcrypt11 >= 1.5.0
    libgnutls.so.26 [pre]
    libgnutls.so.26
    libgnutls.so.26(GNUTLS_1_4) [pre]
    libgnutls.so.26(GNUTLS_1_4)
    libgnutls26 >= 2.12.23 [pre]
    libgnutls26 >= 2.12.23
    libgpg-error.so.0 [pre]
    libgpg-error.so.0
    libgpg-error0 >= 1.11 [pre]
    libgpg-error0 >= 1.11
    libpthread.so.0 [pre]
    libpthread.so.0
    libpthread.so.0(GLIBC_2.4) [pre]
    libpthread.so.0(GLIBC_2.4)
    libtasn1-3 >= 2.14 [pre]
    libtasn1-3 >= 2.14
    libtasn1.so.3 [pre]
    libtasn1.so.3
    libz.so.1 [pre]
    libz.so.1
    libz1 >= 1.2.7 [pre]
    libz1 >= 1.2.7
    rtld(GNU_HASH) [pre]
    rtld(GNU_HASH)
Comment 9 Paul Eggleton 2013-09-27 08:47:00 UTC
So the fix really is to use the version of createrepo we build within the build system; this has been patched to support recommends. FYI this is the version we use to create the indexes when an image is built; so if you built an image and then used the feeds from the target immediately afterwards you shouldn't see the problem (and I'm guessing that's what you've done just now with your most recent build?)

Of course, having to rebuild an image every time to update the indexes would be a pain. The good news is there is "bitbake package-index" which will do this; the bad news is I just found yesterday that it's broken. However, the fix is now in master, is fairly straightforward and applies cleanly on top of dylan:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2f64fae60b3f61a2c45dc9e5314d2d16b1e64605

Apply that and then you should be able to run "bitbake package-index" to update the indexes instead of running your host's createrepo. However note that this does mean you need to add channels for each package architecture rather than just one for all, since that's how we index them.
Comment 10 Jay See 2013-09-27 14:17:38 UTC
Wow...

- bitbake package-index does not fail at home without your patch :(
- smart cannot find a new package, even after a package-index :( ??

Is there something in the workflow I should have missed ? 

What I've done:
- create image
- add repo to image
- smart search cmake (i used cmake as a test as it's missing from my image)
- from my host, bitbake cmake, bitbake package-index
- smart update
- smart search cmake : still nothing :(
Comment 11 Paul Eggleton 2013-09-27 14:24:43 UTC
It won't always actually error out without my patch - but it'll always fail to update the package index properly (it misses out the important package architectures). See bug #5278 for details.

So the steps you performed were with my patch or still without?
Comment 12 Jay See 2013-09-27 21:53:28 UTC
Well I did it with your patch but had missed the comment about entering one repo per directory. I added them and now the flow is ok. 

I need to rebuild an image without curl to check if the dependency issue is solved or not...
Comment 13 Jay See 2013-10-01 22:40:11 UTC
Things are getting better, but now I face a new issue. In the following install (with the use of bitbake package-index) :
root@qemuarm:~# smart install URI-Escape-XS
Loading cache...
Updating cache...                                        ############################################################################## [100%]

Computing transaction...

Installing packages (9):
  URI-Escape-XS-0.10-r0@armv5te                                         libgcc1-4.7.2-r20@armv5te                                             
  busybox-1.20.2-r11@armv5te                                            libperl5-5.14.3-r1@armv5te                                            
  busybox-syslog-1.20.2-r11@armv5te                                     perl-5.14.3-r1@armv5te                                                
  busybox-udhcpc-1.20.2-r11@armv5te                                     update-alternatives-cworth-1:0.1.8+svnr649-r13.0@armv5te              
  libc6-2.17-r3@armv5te                                                 

358.9kB of package files are needed. 5.0MB will be used.

Confirm changes? (Y/n): Y

Fetching packages...                                                                                                                          
-> http://192.168.7.1:81/rpm/armv5te/busybox-syslog-1.20.2-r11.armv5te.rpm                                                                    
busybox-syslog-1.20.2-r11.armv5te.rpm                    ############################################################################## [ 66%]
-> http://192.168.7.1:81/rpm/armv5te/busybox-udhcpc-1.20.2-r11.armv5te.rpm                                                                    
busybox-udhcpc-1.20.2-r11.armv5te.rpm                    ############################################################################## [ 77%]
-> http://192.168.7.1:81/rpm/armv5te/URI-Escape-XS-0.10-r0.armv5te.rpm                                                                        
URI-Escape-XS-0.10-r0.armv5te.rpm                        ############################################################################## [ 88%]
-> http://192.168.7.1:81/rpm/armv5te/busybox-1.20.2-r11.armv5te.rpm                                                                           
busybox-1.20.2-r11.armv5te.rpm                           ############################################################################## [100%]

                                                                                                                                              
Committing transaction...
error: busybox-syslog-1.20.2-r11 requires /dev/null                                                                                           



Note that if I "smart download" the package and "rpm -i" it, I don't face the same issue....
Comment 14 Paul Eggleton 2013-10-03 11:17:36 UTC
This is very strange. What do you get if you do:

smart query --show-all busybox-syslog
Comment 15 Paul Eggleton 2013-10-07 09:42:17 UTC
Changing to NEEDINFO as per above.
Comment 16 Jay See 2013-10-08 18:51:13 UTC
Hi Paul, 
Sorry for the lag in the reply. Here's the command result:

busybox-syslog-1.20.2-r11@armv5te {0} [local3] - Tiny versions of many common UNIX utilities in a single small executable.
  Provides:
    busybox-syslog = 1.20.2-r11@armv5te
    busybox-syslog-systemd
  Requires:
    /bin/sh
      Provided By:
        bash-4.2-r6@armv5te (/bin/sh)
        busybox-1.20.2-r11@armv5te (/bin/sh)
    update-alternatives-cworth
        update-alternatives-cworth-1:0.1.8+svnr649-r13.0@armv5te (update-alternatives-cworth = 1:0.1.8+svnr649-r13.0@armv5te)
  Upgrades:
    busybox-syslog < 1.20.2-r11@armv5te
    busybox-syslog-systemd
  Conflicts:
    busybox-syslog-systemd
Comment 17 Paul Eggleton 2013-10-18 10:54:58 UTC
Hmm, no mention of /dev/null in there.

Right now I'm at a bit of a loss unless I can reproduce this here. Can you provide me with a bit more information as to how you have set up your build so I can try to do the same here?
Comment 18 Paul Eggleton 2013-12-13 17:07:35 UTC
Setting as NEEDINFO again as per above - can you help me to reproduce this?
Comment 19 Paul Eggleton 2014-01-16 16:32:00 UTC
I'm marking this as resolved - I can't reproduce the problem. If you can still reproduce the issue and you're able to help me to do so I'm more than happy to investigate it further; in which case please reopen the bug.