12460 2018-01-02 02:03:23 +0000 Change in push.yoctoproject.org fingerprint causing connection timeout and autobuilder hang 2018-01-31 22:07:05 +0000 1 1 5 Infrastructure AutoBuilder autobuilder unspecified x86 Multiple RESOLVED FIXED Medium+ normal 2.5 M2 1 rebecca.swee.fun.chang mhalstead infras.ab.watcher Infras.watcher joshuagloe pidge richard.purdie --- --- No (bug/feature does not impact docs) oldest_to_newest 78859 0 rebecca.swee.fun.chang 2018-01-02 02:03:23 +0000 We need to investigate and root cause the issue in push.yoctoproject.org that causes autobuilder hang with command timed out: 1200 seconds without output. RP has done a diagnostic on the failure manually: ----- snip ----- [rpurdie@fedora26 ~]$ sudo -iu pokybuild [pokybuild@fedora26 ~]$ cd /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build [pokybuild@fedora26 build]$ pwd=`pwd`;if [ ! -d /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb ];then mkdir -p /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb;git init /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb; fi;rm -rf /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb;mkdir -p /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb;git init /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb;if git ls-remote --exit-code ssh://git@push.yoctoproject.org/poky-buildhistory refs/heads/poky/master-next/nightly-x86-lsb> /dev/null;then git push -q ssh://git@push.yoctoproject.org/poky-buildhistory :poky/master-next/nightly-x86-lsb;fi;if ! git ls-remote --exit-code ssh://git@push.yoctoproject.org/poky-buildhistory refs/heads/poky/master-next/nightly-x86-lsb > /dev/null;then cd /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb; echo 'Initializing Repo' >> README;git checkout -b poky/master-next/nightly-x86-lsb;git add README; git commit -s -m 'Initializing Repo';git push -q ssh://git@push.yoctoproject.org/poky-buildhistory poky/master-next/nightly-x86-lsb:poky/master-next/nightly-x86-lsb;fi Initialized empty Git repository in /srv/autobuilder/buildhistory/poky/master-next/nightly-x86-lsb/.git/ The authenticity of host 'push.yoctoproject.org (140.211.169.56)' can't be established. RSA key fingerprint is SHA256:UG8/tDXRT6y2kewUBO0lSQG6raErMJsqOU+p1wzol/I. RSA key fingerprint is MD5:66:93:3e:a3:f6:60:a6:9e:07:c6:1b:b5:7d:e6:b1:d6. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'push.yoctoproject.org' (RSA) to the list of known hosts. Switched to a new branch 'poky/master-next/nightly-x86-lsb' ----- snip ----- It appears that it is waiting for user's input to confirm about the change in fingerprint. However, we are seeing two more failures on MIPS and PPC. https://autobuilder.yocto.io/builders/nightly-mips-lsb/builds/656/steps/CreateAutoConf/logs/stdio https://autobuilder.yocto.io/builders/nightly-ppc-lsb/builds/669/steps/CreateAutoConf/logs/stdio 79283 1 mhalstead 2018-01-31 22:07:05 +0000 The fingerprint did not change but the hostname changed from git.yoctoproject.org (RO) to push.yoctoproject.org (RW) and needed to be accepted again. This is resolved on all the public workers. Some internal workers may still hit this issue. Add the following to ~/.ssh/known_hosts or run a git push command and accept the new hostname at the prompt. push.yoctoproject.org ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAux9kpB6l6UVdVBDhAWSkoH1VDuf/m9gxRU3g/h0a0BINoeIigev9pI8DRzwIm/dI5+2VRVZuzC0Pq8XZyHTqdKEcw92g2Xh84qfZZivh37XErx9cr//HmBr6/rQ30VvQ6EpRhMrJyG1qvrROUtpu7buRp7XxD/Soa0tiHY72afmXHfKulG2bxOOwPieoCiVWXa+r/21NHY9YfCQxvWoHNRV+g8XrAA8Go611Q2gtYgM8X5Fn71Ehy4JM3Je+FIlUOV0zkL9PkSdRA5og4Eb3rQdgEz1++p6dO82PmgTTzns/+SIgzJBxBHzCi94GbOOAzpwgVFlRFqWIcjE7RWnaNw==