Bug 15556

Summary:	AB-INT: retrieving artifacts over SSH failed
Product:	[Infrastructure] AutoBuilder	Reporter:	Alexandre Belloni <alexandre.belloni>
Component:	autobuilder	Assignee:	Alexandre Belloni <alexandre.belloni>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	Medium+	CC:	infras.ab.watcher, Infras.watcher, pidge, randy.macleod
Version:	unspecified
Target Milestone:	5.2 M1
Hardware:	x86
OS:	Multiple
Whiteboard:	AB-INT
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)

Description Alexandre Belloni 2024-07-18 13:23:21 UTC

Following a ptest failure, retrieving artifacts failed and actually stuck the rest of the build:

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6806/steps/13/logs/stdio

log.do_testimage.311128 ended up being 4.3G

DEBUG: [Running]$ ssh -l root -o ServerAliveCountMax=2 -o ServerAliveInterval=30 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.20 export PATH=/usr/sbin:/sbin:/usr/bin:/bin; tar zcf - /var/log /etc
DEBUG: Waiting for process output: time: 1721087387.1578567, endtime: 1721087687.1568744
tar: removing leading '/' from member names
DEBUG: Partial data from SSH call:
<garbage>

Comment 1 Randy MacLeod 2024-07-18 14:50:19 UTC

There are some artifact collection code changes in master-next that Alex should be aware of.

Comment 2 Alexis Lothoré 2024-09-26 07:08:26 UTC

I discussed a bit the issue with Alexandre, and despite "hung SSH session" likely being the suspect, I do not see how it could happen, considering the parameters passed to ssh commandline in oeqa scripts (connection keepalive and timeout parameters).
I did not manage neither to reproduce locally the issue.
As Randy mentioned, there has been some changes in the mean time which have reduced the number of ssh connections involved in artifacts retrieval (https://git.yoctoproject.org/poky/commit/?id=6e80b2ab660e60135b52bd7358f0ce7612226f06).
Has the issue been observed recently (ie after those changes) ? If not, I suggest to close the issue.

Comment 3 Alexandre Belloni 2024-10-03 10:35:30 UTC

Not seen again, closing for now