Bug 14767 - buildbot timeout of build for unknown reasons (fedora35-ty-2, genericx86-alt)
Summary: buildbot timeout of build for unknown reasons (fedora35-ty-2, genericx86-alt)
Status: RESOLVED FIXED
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: 4.1
Hardware: x86 Multiple
: Medium normal
Target Milestone: 4.1
Assignee: Unassigned
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2022-03-23 22:25 UTC by Richard Purdie
Modified: 2022-06-16 15:05 UTC (History)
5 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Purdie 2022-03-23 22:25:11 UTC
The buildnot UI reports the build timed out with no output but the build looks like it completed.
 
Worker: fedora35-ty-2
genericx86-alt target

https://autobuilder.yoctoproject.org/typhoon/#/builders/103/builds/3818

We can see the step timing information here:

https://autobuilder.yoctoproject.org/typhoon/api/v2/builders/103/builds/3818/steps/11

Translating those epoch values gives:

22 March 2022 17:21:42
22 March 2022 23:01:16

Logs saved here:

https://autobuilder.yocto.io/pub/failed-builds-data/fedora35-genericx86-alt-20220322/

With the endtime on https://autobuilder.yocto.io/pub/failed-builds-data/fedora35-genericx86-alt-20220322/bitbake-cookerdaemon.log showing a time about 10 mins after the start and a clear server exit.

This suggests the server really did complete and the UI hung at exit for some reason. Why? Don't know :(
Comment 1 Richard Purdie 2022-03-23 23:30:47 UTC
I am suspicious of 1785558 14:31:16.523180 More than one thread left?: [<_MainThread(MainThread, started 139817764251456)>, <Thread(asyncio_0, started 139817658435136)>]
Comment 2 Richard Purdie 2022-03-26 17:58:21 UTC
https://git.yoctoproject.org/poky/commit/?id=88fcf8422dd04372d73aacfef0ee50de09ef41bd and https://git.yoctoproject.org/poky/commit/?id=cff6c1a18d56e20d99d2331704dd490940c386e3 should handle the asyncio thread issue which is python 3.10 specific. Not sure if this will help the hang.
Comment 3 Richard Purdie 2022-06-16 15:05:52 UTC
Possibly fixed by https://git.yoctoproject.org/poky/commit/bitbake?id=5940020cfb721d79111072b3cd5d581fff59561b (there was a similar bug I can't find right now)