| Summary: | Git clones to NAS are very slow and have significant impact on build times | ||
|---|---|---|---|
| Product: | [Infrastructure] AutoBuilder | Reporter: | Joshua Lock <joshuagloe> |
| Component: | autobuilder | Assignee: | Michael Halstead <mhalstead> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | major | ||
| Priority: | Medium | CC: | infras.ab.watcher, Infras.watcher, pidge |
| Version: | 2.4 | ||
| Target Milestone: | 4.99 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Joshua Lock
2017-03-22 14:29:28 UTC
What restrictions are there on clones made by the build-system itself? They have to be done in the SOURCE_PUBLISH_DIR or SOURCE_DL_DIR specifically? We need attribute caching turned off to keep our downloads from racing but it looks like we will need attribute caching on for large git clones. Lowering the urgency of this and pushing out to 2.4 as we have mitigated the impact of this by removing the need to re-clone the repository on each oe-selftest run. The NAS storage is performing very well in testing. All clients are now using nfs 4.2. Does this resolve the issue? If not my research indicates that git will always be very slow when working over nfs unless we cache stat data longer. We have seen intermittent build failures in the past when allowing caching of stat data. I don't recall exactly why but I think some processes use that data to avoid race conditions. I resolving this bug because our workaround is functional and the direct solutions cause critical issues. |