Bug 15556 - AB-INT: retrieving artifacts over SSH failed
Summary: AB-INT: retrieving artifacts over SSH failed
Status: RESOLVED FIXED
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 5.2 M1
Assignee: Alexandre Belloni
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2024-07-18 13:23 UTC by Alexandre Belloni
Modified: 2024-10-03 10:35 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandre Belloni 2024-07-18 13:23:21 UTC
Following a ptest failure, retrieving artifacts failed and actually stuck the rest of the build:

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6806/steps/13/logs/stdio

log.do_testimage.311128 ended up being 4.3G

DEBUG: [Running]$ ssh -l root -o ServerAliveCountMax=2 -o ServerAliveInterval=30 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.20 export PATH=/usr/sbin:/sbin:/usr/bin:/bin; tar zcf - /var/log /etc
DEBUG: Waiting for process output: time: 1721087387.1578567, endtime: 1721087687.1568744
tar: removing leading '/' from member names
DEBUG: Partial data from SSH call:
<garbage>
Comment 1 Randy MacLeod 2024-07-18 14:50:19 UTC
There are some artifact collection code changes in master-next that Alex should be aware of.
Comment 2 Alexis Lothoré 2024-09-26 07:08:26 UTC
I discussed a bit the issue with Alexandre, and despite "hung SSH session" likely being the suspect, I do not see how it could happen, considering the parameters passed to ssh commandline in oeqa scripts (connection keepalive and timeout parameters).
I did not manage neither to reproduce locally the issue.
As Randy mentioned, there has been some changes in the mean time which have reduced the number of ssh connections involved in artifacts retrieval (https://git.yoctoproject.org/poky/commit/?id=6e80b2ab660e60135b52bd7358f0ce7612226f06).
Has the issue been observed recently (ie after those changes) ? If not, I suggest to close the issue.
Comment 3 Alexandre Belloni 2024-10-03 10:35:30 UTC
Not seen again, closing for now