| Summary: | HASHSERVER_UPSTREAM discards any gains from the batch stream API | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] BitBake | Reporter: | Michal Sieron <michalwsieron> | ||||
| Component: | bitbake | Assignee: | Joshua Watt <JPEWhacker> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | Medium | CC: | JPEWhacker, poky.bs.watcher, poky.watcher, randy.macleod | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 6.1 | ||||||
| Hardware: | All | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | |||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||
| Attachments: |
|
||||||
The idea is sound, but I'm not a big fan of the way this patch works. I do have an idea for how to do this, so I can try to work it up in the comming weeks Wow, that was quick. Thanks a lot. Will test on Monday when in office Initially it seemed to work just fine and indeed the speedup was there, but then on a rerun I had some issues where it seemed that bitbake couldn't communicate with the local hashserv (leftover .sock?). After cleaning up the builddir, now I get the same issue as on the Autobuilder with `hashserv.client.AsyncQueue.Shutdown`. |
Created attachment 5236 [details] AI generated fix This is something I noticed quite some time ago as our builds' startup was taking a lot of time during the `Initialising tasks: 44%` step, which is when Bitbake queries the hashserver. Basically, when `BB_HASHSERVE_UPSTREAM` is configured, a local hash equivalence server awaits a blocking `upstream_client.get_unihash()` for every miss, which discards any gains from the batch stream API. Because I am not confident with my understanding of this part of Bitbake's code or async in general, I had to use LLM for a fix. It seems to work, but as it is 100% generated, I didn't want to send it straight to the mailing list. You can check the attached patch and decide what to do with it.