| Summary: | SState: wget improvements | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Benjamin Esquivel <benjamin.esquivel> |
| Component: | core | Assignee: | Aníbal Limón <anibal.limon> |
| Status: | VERIFIED FIXED | QA Contact: | Daniel Istrate <daniel.alexandrux.istrate> |
| Severity: | enhancement | ||
| Priority: | Medium+ | CC: | alexandru.c.georgescu, benjamin.esquivel, bogdanx.a.voiculescu, meta.mr.watcher, meta.watcher, richard.purdie |
| Version: | unspecified | ||
| Target Milestone: | 1.9 M2 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | Don't know | |
|
Description
Benjamin Esquivel
2015-05-20 16:34:48 UTC
There are two stages for this. One is parallelising the "present in sstate" checks. The patches for this have been separated out, cleaned up and merged. There are also improvements to the wget fetcher, particularly so it can reuse conections rather than making a connection attempt per url. There is improved code for this in the branch which now uses TLS to hold the connection cache. To work correctly, the code had to be switched from wget to urllib2. We have to hack the class to make it reuse connections though (and also to support HEAD requests). We need to use the class to ensure proxy information is handled correctly. My rpurdie/t222 branch has the improved code but more cleanup needs to be done, along with better tests before it can be submitted. In review at, http://lists.openembedded.org/pipermail/openembedded-core/2015-June/106430.html http://lists.openembedded.org/pipermail/openembedded-core/2015-June/106439.html http://lists.openembedded.org/pipermail/bitbake-devel/2015-June/005912.html Integration branch at, http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=alimon/fetch2-wget-checkstatus Merged at, http://cgit.openembedded.org/bitbake/commit/?id=9fa6407e6cefe66c77467419a8040d6957a6bb01 http://cgit.openembedded.org/openembedded-core/commit/?id=e6f66370c6ce15aca18ef64491bab3dc92b80c57 Verified on master: 505a82673ac2487df5ea343a6422c2fc47018831 The tests from bitbake/lib/bb/tests/fetch.py --> FetchCheckStatusTest validate this feature. [daniel@fedora-ws poky-build]$ bitbake-selftest -v bb.tests.fetch.FetchCheckStatusTest test_wget_checkstatus (bb.tests.fetch.FetchCheckStatusTest) ... ok test_wget_checkstatus_connection_cache (bb.tests.fetch.FetchCheckStatusTest) ... ok ---------------------------------------------------------------------- Ran 2 tests in 23.763s OK [daniel@fedora-ws poky-build]$ |