6329 2014-05-14 18:32:14 +0000 libcurl bug when compiled against gnutls 2014-07-01 17:46:25 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 1.5.1 x86 Multiple RESOLVED FIXED Medium normal 1.7 1 lists.volker sgw meta.mr.watcher meta.watcher poky.bs.watcher poky.watcher --- --- No (bug/feature does not impact docs) oldest_to_newest 43338 0 1957 lists.volker 2014-05-14 18:32:14 +0000 Created attachment 1957 script that exposes the libcurl bug There is a bug in libcurl that raises execeptions when compiled against gnutls. If libcurl is compiled against ssl it works without any problems. How to reproduce: git clone -b dora git://git.yoctoproject.org/poky.git cd poky # get version for bugzilla git rev-parse HEAD 0a6f0dbf9476dcb4a17a210af90e2dd1a43b61aa # initialize env source oe-init-build-env # Add packages to reproduce it. python-pycurl is a must # some of the others are needed for the test script # that exposes the bug, others might not be required. # installing all of them doesn't matter too much echo "IMAGE_INSTALL_append = ' python-pycurl python-ctypes python-datetime python-xmlrpc python-threading python-subprocess python-shell python-compression'" >> conf/local.conf # Parallel build echo 'BB_NUMBER_THREADS ?= "8"' >> conf/local.conf echo 'PARALLEL_MAKE ?= "-j 4"' >> conf/local.conf # build & wait bitbake core-image-minimal # start test environment runqemu qemux86 Get the attached script that exposes the bug into the image (e.g. via wget from another system you have access to) and execute it inside the qemu environment. The attached script could also be used for ptest after some smaller modifications in the log output. A workaround is creating a recipe with the following file and content filename: curl_7.32.0.bbappend content: # Workaround for libcurl bug when compiled against gnutls CURLGNUTLS = "--without-gnutls --with-ssl" 43411 1 sgw 2014-05-19 20:53:33 +0000 Have you tried to build/test curl upstream for this issue, ie is this an issue in the recipe configuration or in the actual code, in which case we should also report it to the upstream maintainer to get a proper fix. 43412 2 sgw 2014-05-19 21:01:20 +0000 Is this the issue you are seeing: http://curl.haxx.se/mail/lib-2014-04/0145.html? 43745 3 lists.volker 2014-06-03 19:09:15 +0000 As requested I send an email to the libcurl mailing list: http://curl.haxx.se/mail/lib-2014-06/0028.html 43828 4 lists.volker 2014-06-09 18:26:30 +0000 I am not in the mood to discuss with the libcurl developers the issue nor do I want to do a step by step explanation of the testing script one of my co-workers wrote and I can't explain well. I send the report to the libcurl mailinglist as requested. Find the entire thread on http://curl.haxx.se/mail/lib-2014-06/index.html (search for Yocto). I would suggest changing the default GNUTLS settings as described in my first entry to --- CURLGNUTLS = "--without-gnutls --with-ssl" --- 43962 5 sgw 2014-06-16 04:09:37 +0000 An update on this issue, I build and booted our current master which contains the new 7.36 version of curl and the newer version of GnuTLS. root@qemux86-64:~# curl --version curl 7.36.0 (x86_64-poky-linux-gnu) libcurl/7.36.0 GnuTLS/3.2.15 zlib/1.2.8 Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smtp smtps telnet tftp Features: IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP I was able to run your script with no issue, I am doing a build with 1.5.1 HEAD to reproduce this with the older version of libcurl, maybe there is a patch that can be back ported. I was also able to verify that Daisy (1.6) does NOT have your issue, so that limits it to a difference between libcurl 7.32 and 7.35, since with daisy GnuTLS did not change. If you have time to test 7.33 and 7.34 that would help narrow this down also. I am not sure I have much more time, since this is fixed in the Upstream, this might help them narrow the issue down in 7.32. 44271 6 sgw 2014-07-01 17:46:25 +0000 http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=97294829cde141c8b6b50418de27f53507f5764e 1957 2014-05-14 18:32:14 +0000 2014-05-14 18:32:14 +0000 script that exposes the libcurl bug test-bad-libcurl.py text/x-python 6689 lists.volker IyEvdXNyL2Jpbi9weXRob24KCiMgVGVzdCBpcyBmcmVlIG9mIHVzZSBmb3IgdGVzdGluZyB0aGUg YSBidWcgZm91bmQgaW4gWW9jdG8KCiIiInRlc3QgY2FzZSBmb3IgcHJvYmxlbSB3aXRoIEdudVRM UyBsaWJjdXJsLiIiIgoKaW1wb3J0IGxvZ2dpbmcKaW1wb3J0IHRpbWUKaW1wb3J0IGNTdHJpbmdJ TyBhcyBTdHJpbmdJTwppbXBvcnQgY3R5cGVzCmltcG9ydCBlcnJubwppbXBvcnQgb3MKCmltcG9y dCBweWN1cmwKCmNsYXNzIEZldGNoRXhjZXB0aW9uKEV4Y2VwdGlvbik6CiAgICAiIiJDb21tb24g YmFzZSBjbGFzcyBmb3IgYWxsIHRoaXMgbW9kdWxlJ3MgZXhjZXB0aW9ucy4iIiIKICAgIGRlZiBf X2luaXRfXyhzZWxmLCBhcmdzPU5vbmUpOgogICAgICAgIGlmIGlzaW5zdGFuY2UoYXJncywgRXhj ZXB0aW9uKToKICAgICAgICAgICAgc2VsZi5hcmdzID0gKGFyZ3MuX19jbGFzc19fLl9fbmFtZV9f LCBzdHIoYXJncykpCiAgICAgICAgZWxzZToKICAgICAgICAgICAgc2VsZi5hcmdzID0gKGFyZ3Ms KQoKCmNsYXNzIFRpbWVkT3V0KEZldGNoRXhjZXB0aW9uKToKICAgIHBhc3MKCgpjbGFzcyBSZXNv bHZlRXJyb3IoRmV0Y2hFeGNlcHRpb24pOgogICAgcGFzcwoKCmNsYXNzIE90aGVyRXJyb3IoRmV0 Y2hFeGNlcHRpb24pOgogICAgcGFzcwoKCmNsYXNzIEhlYWRlcnMob2JqZWN0KToKICAgICIiIkNv bGxlY3QgdGhlIGhlYWRlcnMgYXMgcHljdXJsIGdpdmVzIHRoZW0gdG8gdXMuIiIiCiAgICBkZWYg X19pbml0X18oc2VsZik6CiAgICAgICAgc2VsZi5oZWFkZXJzID0ge30KCiAgICBkZWYgd3JpdGUo c2VsZiwgaCk6CiAgICAgICAgaCA9IGguc3RyaXAoKQogICAgICAgIGlmIGg6CiAgICAgICAgICAg IGxvZ2dpbmcuZGVidWcoJ2hlYWRlcjogIiVzIicsIGgpCiAgICAgICAgICAgIGlmIGguc3RhcnRz d2l0aCgnSFRUUC8nKTogICAgICAgIyByZXN0YXJ0IG9uIHJlZGlyZWN0aW9uCiAgICAgICAgICAg ICAgICBzZWxmLmhlYWRlcnMgPSB7fQogICAgICAgICAgICBlbGlmIGg6CiAgICAgICAgICAgICAg ICBzID0gaC5zcGxpdCgnOicsIDEpCiAgICAgICAgICAgICAgICBpZiBsZW4ocykgPT0gMjoKICAg ICAgICAgICAgICAgICAgICBzZWxmLmhlYWRlcnNbc1swXS5zdHJpcCgpXSA9IHNbMV0uc3RyaXAo KQoKCmRlZiBfcmVhZF91cmxfaGVscGVyKHVybCwgbnVtX3RyaWVzLCB0aW1lb3V0LCBkYXRhPU5v bmUpOgogICAgc3RhcnRfdGltZSA9IHRpbWUudGltZSgpCiAgICBoZWFkZXJzID0gSGVhZGVycygp CiAgICBmb3IgX3RyeSBpbiByYW5nZShudW1fdHJpZXMpOgogICAgICAgIHRyeToKICAgICAgICAg ICAgIwogICAgICAgICAgICAjIGJhaWwgaWYgd2UgaGF2ZSBubyB0aW1lIGxlZnQKICAgICAgICAg ICAgIwogICAgICAgICAgICB0aW1lX3JlbWFpbmluZyA9IHRpbWVvdXQgLSAodGltZS50aW1lKCkg LSBzdGFydF90aW1lKQogICAgICAgICAgICBpZiB0aW1lX3JlbWFpbmluZyA8PSAwLjA6CiAgICAg ICAgICAgICAgICBsb2dnaW5nLndhcm5pbmcoJ3RpbWVvdXQgZXhjZWVkZWQgYmVmb3JlIGNvbm5l Y3Rpb24sIHRyeTogJWQnLCBfdHJ5KQogICAgICAgICAgICAgICAgcmFpc2UgVGltZWRPdXQoKQoK ICAgICAgICAgICAgY3VybCA9IHB5Y3VybC5DdXJsKCkKICAgICAgICAgICAgY3VybC5zZXRvcHQo cHljdXJsLlVSTCwgdXJsKQogICAgICAgICAgICBjdXJsLnNldG9wdChweWN1cmwuRk9MTE9XTE9D QVRJT04sIDEpCiAgICAgICAgICAgIGN1cmwuc2V0b3B0KHB5Y3VybC5NQVhSRURJUlMsIDUpCiAg ICAgICAgICAgIGN1cmwuc2V0b3B0KHB5Y3VybC5OT1NJR05BTCwgMSkKICAgICAgICAgICAgY3Vy bC5zZXRvcHQocHljdXJsLkZBSUxPTkVSUk9SLCAxKQoKICAgICAgICAgICAgIwogICAgICAgICAg ICAjIHdlIGRvbid0IGNhcmUgYWJvdXQgY29ycmVjdCBzc2wgY2VydCBoZXJlCiAgICAgICAgICAg ICMKICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLlNTTF9WRVJJRllQRUVSLCAwKQogICAg ICAgICAgICBjdXJsLnNldG9wdChweWN1cmwuU1NMX1ZFUklGWUhPU1QsIDApCgogICAgICAgICAg ICAjCiAgICAgICAgICAgICMgY2FuIGdldCBtb3JlIHZlcmJvc2UgZGVidWdnaW5nIGZyb20gY3Vy bCBpdHNlbGYgaWYgd2Ugd2FudAogICAgICAgICAgICAjCiAgICAgICAgICAgIGlmIEZhbHNlOiAg ICAgICAgICAgICAgICAgICAjIHByYWdtYTogbm8gY292ZXIKCiAgICAgICAgICAgICAgICBkZWYg ZGVidWcodHlwZV8sIG1zZyk6CiAgICAgICAgICAgICAgICAgICAgdG1hcCA9IHtweWN1cmwuSU5G T1RZUEVfVEVYVDogJ1RFWFQnLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgcHljdXJsLklO Rk9UWVBFX0hFQURFUl9JTjogJ0hFQURFUl9JTicsCiAgICAgICAgICAgICAgICAgICAgICAgICAg ICBweWN1cmwuSU5GT1RZUEVfSEVBREVSX09VVDogJ0hFQURFUl9PVVQnLAogICAgICAgICAgICAg ICAgICAgICAgICAgICAgcHljdXJsLklORk9UWVBFX0RBVEFfSU46ICdEQVRBX0lOJywKICAgICAg ICAgICAgICAgICAgICAgICAgICAgIHB5Y3VybC5JTkZPVFlQRV9EQVRBX09VVDogJ0RBVEFfT1VU J30KICAgICAgICAgICAgICAgICAgICBpZiB0eXBlXyBpbiB0bWFwOgogICAgICAgICAgICAgICAg ICAgICAgICBsb2dnaW5nLmRlYnVnKCdweWN1cmw6ICUxMHM6ICVzJywgdG1hcFt0eXBlX10sIG1z Zy5zdHJpcCgpKQogICAgICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLlZFUkJPU0UsIDEp CiAgICAgICAgICAgICAgICBjdXJsLnNldG9wdChweWN1cmwuREVCVUdGVU5DVElPTiwgZGVidWcp CgogICAgICAgICAgICAjCiAgICAgICAgICAgICMgc2V0IHVwIHRoZSBjb25uZWN0aW9uCiAgICAg ICAgICAgICMKICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLkNPTk5FQ1RfT05MWSwgMSkK ICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLkNPTk5FQ1RUSU1FT1VUX01TLCBpbnQodGlt ZV9yZW1haW5pbmcgKiAxMDAwKSkKICAgICAgICAgICAgY3VybC5wZXJmb3JtKCkKCiAgICAgICAg ICAgICMKICAgICAgICAgICAgIyBiYWlsIGlmIHdlIGhhdmUgbm8gdGltZSBsZWZ0CiAgICAgICAg ICAgICMKICAgICAgICAgICAgdGltZV9yZW1haW5pbmcgPSB0aW1lb3V0IC0gKHRpbWUudGltZSgp IC0gc3RhcnRfdGltZSkKICAgICAgICAgICAgaWYgdGltZV9yZW1haW5pbmcgPD0gMC4wOgogICAg ICAgICAgICAgICAgbG9nZ2luZy53YXJuaW5nKCd0aW1lb3V0IGV4Y2VlZGVkIGFmdGVyIGNvbm5l Y3Rpb24sIHRyeTogJWQnLCBfdHJ5KQogICAgICAgICAgICAgICAgcmFpc2UgVGltZWRPdXQoKQoK ICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLkhFQURFUkZVTkNUSU9OLCBoZWFkZXJzLndy aXRlKQoKICAgICAgICAgICAgIwogICAgICAgICAgICAjIGFjdHVhbGx5IHJlYWQgdGhlIGRhdGEs IHVzaW5nIHRoZSBjb25uZWN0aW9uIGFscmVhZHkgZXN0YWJsaXNoZWQKICAgICAgICAgICAgIwog ICAgICAgICAgICBpZiBkYXRhIGlzIE5vbmU6CiAgICAgICAgICAgICAgICBkYXRhID0gU3RyaW5n SU8uU3RyaW5nSU8oKQogICAgICAgICAgICBjdXJsLnNldG9wdChweWN1cmwuV1JJVEVGVU5DVElP TiwgZGF0YS53cml0ZSkKICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLkNPTk5FQ1RfT05M WSwgMCkKICAgICAgICAgICAgY3VybC5zZXRvcHQocHljdXJsLlRJTUVPVVRfTVMsIGludCh0aW1l X3JlbWFpbmluZyAqIDEwMDApKQogICAgICAgICAgICBjdXJsLnBlcmZvcm0oKQogICAgICAgICAg ICBsb2dnaW5nLmRlYnVnKCdlZmZlY3RpdmUgVVJMOiAlcycsIGN1cmwuZ2V0aW5mbyhweWN1cmwu RUZGRUNUSVZFX1VSTCkpCiAgICAgICAgICAgIGxvZ2dpbmcuZGVidWcoJ2FsbCBoZWFkZXJzOiAl cycsIGhlYWRlcnMuaGVhZGVycykKICAgICAgICAgICAgcmV0dXJuIGRhdGEsIGN1cmwuZ2V0aW5m byhweWN1cmwuSFRUUF9DT0RFKSwgaGVhZGVycy5oZWFkZXJzLCBjdXJsLmdldGluZm8ocHljdXJs LkVGRkVDVElWRV9VUkwpCiAgICAgICAgZXhjZXB0IHB5Y3VybC5lcnJvciwgZToKICAgICAgICAg ICAgaWYgY3VybC5nZXRpbmZvKHB5Y3VybC5IVFRQX0NPREUpOgogICAgICAgICAgICAgICAgcmV0 dXJuIGRhdGEsIGN1cmwuZ2V0aW5mbyhweWN1cmwuSFRUUF9DT0RFKSwgaGVhZGVycy5oZWFkZXJz LCBjdXJsLmdldGluZm8ocHljdXJsLkVGRkVDVElWRV9VUkwpCiAgICAgICAgICAgIGxvZ2dpbmcu d2FybmluZygncHljdXJsIGVycm9yOiAlcywgdXJsOiAlcywgdHJ5OiAlZCcsIGUsIHVybCwgX3Ry eSkKICAgICAgICAgICAgaWYgZS5hcmdzWzBdID09IHB5Y3VybC5FX0NPVUxETlRfUkVTT0xWRV9I T1NUOgogICAgICAgICAgICAgICAgcmFpc2UgUmVzb2x2ZUVycm9yCiAgICAgICAgICAgIGlmIGUu YXJnc1swXSA9PSBweWN1cmwuRV9PUEVSQVRJT05fVElNRU9VVEVEOgogICAgICAgICAgICAgICAg cmFpc2UgVGltZWRPdXQKICAgICAgICAgICAgcmFpc2UgT3RoZXJFcnJvcgogICAgICAgIGV4Y2Vw dCBGZXRjaEV4Y2VwdGlvbjoKICAgICAgICAgICAgcmFpc2UKICAgICAgICBleGNlcHQgRXhjZXB0 aW9uOgogICAgICAgICAgICBsb2dnaW5nLmV4Y2VwdGlvbigndW5oYW5kbGVkIGV4Y2VwdGlvbiwg dHJ5OiAlZCcsIF90cnkpCgogICAgbG9nZ2luZy53YXJuaW5nKCdmYWlsaW5nIGFmdGVyICVkIHRy aWVzJywgbnVtX3RyaWVzKQogICAgcmFpc2UgVGltZWRPdXQoKQoKCmRlZiByZWFkX3VybF9hc19z dHJpbmdfZXh0cmFzKHVybCwgbnVtX3RyaWVzPTEsIHRpbWVvdXQ9MTAuMCk6CiAgICByZXN1bHQs IGNvZGUsIGhlYWRlcnMsIGVmZmVjdGl2ZV91cmwgPSBfcmVhZF91cmxfaGVscGVyKHVybCwgbnVt X3RyaWVzLCB0aW1lb3V0KQogICAgaWYgcmVzdWx0IGlzIE5vbmU6CiAgICAgICAgcmV0dXJuIE5v bmUsIGNvZGUsIGhlYWRlcnMsIGVmZmVjdGl2ZV91cmwKICAgIGVsc2U6CiAgICAgICAgcmV0dXJu IHJlc3VsdC5nZXR2YWx1ZSgpLCBjb2RlLCBoZWFkZXJzLCBlZmZlY3RpdmVfdXJsCgoKdHJ5Ogog ICAgX25hbWUgPSAnbGliYy5zby42JwogICAgX2xpYmMgPSBjdHlwZXMuQ0RMTChfbmFtZSwgdXNl X2Vycm5vPVRydWUpCmV4Y2VwdCBFeGNlcHRpb24sIF9lOgogICAgcHJpbnQgPj4gc3lzLnN0ZGVy ciwgJ0xvYWRpbmcgc2hhcmVkIGxpYnJhcnkgIiVzIicgJSBfbmFtZQogICAgcHJpbnQgPj4gc3lz LnN0ZGVyciwgX2UKICAgIHJhaXNlCmRlbCBfbmFtZQoKCmNsYXNzIF90aW1ldmFsKGN0eXBlcy5T dHJ1Y3R1cmUpOgogICAgIiIiZnJvbSA8c3lzL3RpbWUuaD4iIiIKICAgIF9maWVsZHNfID0gWygn dHZfc2VjJywgY3R5cGVzLmNfbG9uZyksCiAgICAgICAgICAgICAgICAoJ3R2X3VzZWMnLCBjdHlw ZXMuY19sb25nKV0KCgpkZWYgX2VycmNoZWNrKHJlc3VsdCwgZnVuYywgYXJndW1lbnRzKToKICAg ICIiImNoZWNrIHN5c3RlbSBjYWxsIGZvciBlcnJvcnMsIG1hcCB0byBleGNlcHRpb24iIiIKICAg IGlmIHJlc3VsdCA9PSAtMToKICAgICAgICBlcnIgPSBjdHlwZXMuZ2V0X2Vycm5vKCkKICAgICAg ICByYWlzZSBPU0Vycm9yKChlcnIsICclczogJXMnICUgKGVycm5vLmVycm9yY29kZVtlcnJdLCBv cy5zdHJlcnJvcihlcnIpKSkpCiAgICByZXR1cm4gYXJndW1lbnRzCgpfbGliYy5hZGp0aW1lLmFy Z3R5cGVzID0gW2N0eXBlcy5jX3ZvaWRfcCwgY3R5cGVzLmNfdm9pZF9wXQpfbGliYy5hZGp0aW1l LnJlc3R5cGUgPSBjdHlwZXMuY19pbnQKX2xpYmMuYWRqdGltZS5lcnJjaGVjayA9IF9lcnJjaGVj awoKCmRlZiBhZGp0aW1lKHQpOgogICAgIiIiY2FsbCB0aGUgYWRqdGltZSgyKSBzeXN0ZW0gY2Fs bCB0byBhZGp1c3QgdGhlIHN5c3RlbSB0aW1lCiAgICBncmFkdWFsbHkgYnkgc29tZSBkZWx0YS4i IiIKICAgIHRpbWV2YWwgPSBfdGltZXZhbCgpCiAgICB0aW1ldmFsLnR2X3NlYywgdGltZXZhbC50 dl91c2VjID0gaW50KHQpLCBpbnQoKHQgLSBpbnQodCkpICogMWU2KQogICAgX2xpYmMuYWRqdGlt ZShjdHlwZXMuYnlyZWYodGltZXZhbCksIE5vbmUpCgoKaWYgX19uYW1lX18gPT0gJ19fbWFpbl9f JzoKICAgIGxvZ2dpbmcuYmFzaWNDb25maWcobGV2ZWw9bG9nZ2luZy5JTkZPLCBmb3JtYXQ9JyUo YXNjdGltZSlzICUobGV2ZWxuYW1lKS04cyAlKGxpbmVubyk0ZCAlKG1lc3NhZ2UpcycpCgogICAg aHR0cHNfdXJsID0gJ2h0dHBzOi8vd3d3Lmdvb2dsZS5jb20nCgogICAgdHJ5OgogICAgICAgIGxv Z2dpbmcuaW5mbygnY2FsbGluZyBhZGp0aW1lIGJlZm9yZSBIVFRQUyBHRVQgdG8gJXMnLCBodHRw c191cmwpCiAgICAgICAgYWRqdGltZSgwLjAxKQoKICAgICAgICByZXN1bHQsIGNvZGUsIGhlYWRl cnMsIGVmZmVjdGl2ZV91cmwgPSByZWFkX3VybF9hc19zdHJpbmdfZXh0cmFzKGh0dHBzX3VybCkK CiAgICAgICAgbG9nZ2luZy5pbmZvKCdjYWxsaW5nIGFkanRpbWUgYWZ0ZXIgSFRUUFMgR0VUIHRv ICVzJywgaHR0cHNfdXJsKQogICAgICAgIGFkanRpbWUoMC4wMSkKICAgIGV4Y2VwdCBFeGNlcHRp b246CiAgICAgICAgI2xvZ2dpbmcuZXhjZXB0aW9uKCdleGNlcHRpb24nKQogICAgICAgIHByaW50 ICJGQUlMOiBodHRwcy1hZGp0aW1lLWxpYmN1cmwgaXNzdWUiCiAgICAgICAgcmFpc2UgU3lzdGVt RXhpdCgxKQoKICAgIHByaW50ICJPSzogaHR0cHMtYWRqdGltZS1saWJjdXJsIGlzc3VlIgogICAg cmFpc2UgU3lzdGVtRXhpdCgwKQo=