This is most easily observed in the failures of an otherwise green build #59[1]. The nightly-checkuri contains several log[2] entries like: ERROR: alsa-utils-scripts-1.1.2-r0 do_checkuri: Function failed: Fetcher failure for URL: 'ftp://ftp.alsa-project.org/pub/utils/alsa-utils-1.1.2.tar.bz2'. URL ftp://ftp.alsa-project.org/pub/utils/alsa-utils-1.1.2.tar.bz2 doesn't work ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-checkuri/build/build/tmp/work/i586-poky-linux/alsa-utils-scripts/1.1.2-r0/temp/log.do_checkuri.39012 Log data follows: | DEBUG: Executing python function do_checkuri | DEBUG: Testing URL ftp://ftp.alsa-project.org/pub/utils/alsa-utils-1.1.2.tar.bz2 | DEBUG: checkstatus: trying again | DEBUG: checkstatus() urlopen failed: <urlopen error ftp error: TimeoutError(110, 'Connection timed out')> | DEBUG: Python function do_checkuri finished | ERROR: Function failed: Fetcher failure for URL: 'ftp://ftp.alsa-project.org/pub/utils/alsa-utils-1.1.2.tar.bz2'. URL ftp://ftp.alsa-project.org/pub/utils/alsa-utils-1.1.2.tar.bz2 doesn't work NOTE: recipe alsa-utils-scripts-1.1.2-r0: task do_checkuri: Failed 1. http://autobuilder.yocto.io:8010/builders/nightly/builds/59 2. http://autobuilder.yocto.io:8010/builders/nightly-checkuri/builds/37/steps/BuildImages/logs/stdio
Because of the way passive ftp works, opening up high ports at random, our security appliance is unhappy. Options: a) either turn off security appliance b) setup proxy server in DMZ and setup proxy settings on all builders c) recode recipes to use http or active ftp b is preference We may want to think about sun-setting FTP, if possible.
I've updated settings on the firewall and FTP failures have generally abated. While I still see failures when running wget in a loop the current state is good enough. The failures may be on the remote side based on the number of failures. Let's mark this working for now.
Sadly the last builds on the autobuilder showed this issue. Not sure why but we're going to need to look into this further.
I tried disabling the security appliance and the errors continued. I've started to suspect routing or mtu issues. For now I've set up an ftp proxy on another network and enabled it for all workers except ubuntu1510 which is currently busy. I'll update it next. This should solve the problem at least for now.