We are observing *very* slow git clones of linux-yocto during oe-selftest on both clusters. In #10612 we moved cloning of poky.git off the NAS, however that workaround isn't tenable for clones made by the build-system itself. This is easily reproduced by logging into one of the servers and comparing the speed of a clone on the local filesystem vs one on the NAS: git -c core.fsyncobjectfiles=0 clone --bare --mirror git://git.yoctoproject.org/linux-yocto-4.10.git /home/pokybuild/git.yoctoproject.org.linux-yocto-4.10.git --progress Receiving objects: 100% (5195778/5195778), 1.04 GiB | 11.62 MiB/s, done. git -c core.fsyncobjectfiles=0 clone --bare --mirror git://git.yoctoproject.org/linux-yocto-4.10.git /srv/autobuilder/autobuilder.yoctoproject.org/current_sources/rptest/git.yoctoproject.org.linux-yocto-4.10.git --progress Receiving objects: 0% (45737/5195778), 20.53 MiB | 325.00 KiB/s Speeds are significantly slower and have a negative impact on the overall build times.
What restrictions are there on clones made by the build-system itself? They have to be done in the SOURCE_PUBLISH_DIR or SOURCE_DL_DIR specifically? We need attribute caching turned off to keep our downloads from racing but it looks like we will need attribute caching on for large git clones.
Lowering the urgency of this and pushing out to 2.4 as we have mitigated the impact of this by removing the need to re-clone the repository on each oe-selftest run.
The NAS storage is performing very well in testing. All clients are now using nfs 4.2. Does this resolve the issue? If not my research indicates that git will always be very slow when working over nfs unless we cache stat data longer. We have seen intermittent build failures in the past when allowing caching of stat data. I don't recall exactly why but I think some processes use that data to avoid race conditions.
I resolving this bug because our workaround is functional and the direct solutions cause critical issues.