| Summary: | AB-INT: test_list_pkg_files: ResourceWarning: unclosed transport | ||
|---|---|---|---|
| Product: | [QA/Testing] Functional (self) Testing | Reporter: | Mathieu Dubois-Briand <mathieu.dubois-briand> |
| Component: | oe-selftest | Assignee: | Francisco Pedraza <pacopedraza87> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | Medium | CC: | anibal, JPEWhacker, pacopedraza87, randy.macleod, richard.purdie, tim.orling, yoann.congal |
| Version: | unspecified | ||
| Target Milestone: | 6.1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | AB-INT | ||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
oe-selftest-debian debian11-vk-1 mathieu/master-next completed at 2026-04-07 12:22:14+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3618/steps/15/logs/stdio scarthgap oe-selftest-debian ubuntu2204-vk-1 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/3612/steps/15/logs/stdio 2026-04-07 00:03:57,628 - oe-selftest - INFO - oelib.buildhistory.TestBlobParsing.test_compare_dict_blobs_default (subunit.RemotedTestCase) 2026-04-07 00:03:57,628 - oe-selftest - INFO - ... FAIL Stderr: 2026-04-06 22:32:36,881 - oe-selftest - INFO - Adding: "include selftest.inc" in /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3927156/conf/local.conf 2026-04-06 22:32:36,881 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf Loading cache...WARNING: /usr/lib/python3.10/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f9063ee7460> _warn(f"unclosed transport {self!r}", ResourceWarning, source=self) Might be due to some recent denial of service attacks. The two builds were about 12 hours apart. oe-selftest-debian ubuntu2204-vk-2 master completed at 2026-04-15 02:44:52+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3677/steps/15/logs/stdio Since this is reporting an unclosed SSL connection, this is almost certainly an unclosed websocket used by the hash equivalence server. There are only a few places where hash equivalence clients are created. Almost all of them use a `with` block to ensure that the client is properly cleaned up, except for one spot. In the SignatureGeneratorUniHashMixIn class in bb/lib/siggen.py, it has:
```
@contextmanager
def client(self):
with self._client_env():
if getattr(self, '_client', None) is None:
self._client = hashserv.create_client(self.server, **self.get_hashserv_creds())
yield self._client
def reset(self, data):
self.__close_clients()
return super().reset(data)
def exit(self):
self.__close_clients()
return super().exit()
def __close_clients(self):
with self._client_env():
if getattr(self, '_client', None) is not None:
self._client.close()
self._client = None
if getattr(self, '_client_pool', None) is not None:
self._client_pool.close()
self._client_pool = None
```
In this case, siggen will create a client (if one doesn't already exist) in the `client()` call, but it will not clean it up unless `reset()` or `exit()` is called; this seems like the most likely culprit as to the error.
(In reply to Joshua Watt from comment #5) > Since this is reporting an unclosed SSL connection, this is almost certainly > an unclosed websocket used by the hash equivalence server. There are only a > few places where hash equivalence clients are created. Almost all of them > use a `with` block to ensure that the client is properly cleaned up, except > for one spot. In the SignatureGeneratorUniHashMixIn class in > bb/lib/siggen.py, it has: > > ``` > @contextmanager > def client(self): > with self._client_env(): > if getattr(self, '_client', None) is None: > self._client = hashserv.create_client(self.server, > **self.get_hashserv_creds()) > yield self._client > > def reset(self, data): > self.__close_clients() > return super().reset(data) > > def exit(self): > self.__close_clients() > return super().exit() > > def __close_clients(self): > with self._client_env(): > if getattr(self, '_client', None) is not None: > self._client.close() > self._client = None > if getattr(self, '_client_pool', None) is not None: > self._client_pool.close() > self._client_pool = None > ``` > > In this case, siggen will create a client (if one doesn't already exist) in > the `client()` call, but it will not clean it up unless `reset()` or > `exit()` is called; this seems like the most likely culprit as to the error. Thanks Joshua, I'll try to reproduce it and if succeed do a patch for it, may be a test case. On a debian 11 system with python 3.9, the bitbake-cookerdaemon.log is full of these messages. The oe-selftest failure is obviously bad timing of the garbage collection but if we can remove them from the log generally, it would mean we likely fix the oe-selftest issue even if we can't direction reproduce it. oe-selftest-debian debian11-vk-1 master completed at 2026-05-07 04:37:57+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3804/steps/15/logs/stdio Is this one another synpto of the same issue? In recipetool. oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-05-14 03:09:43+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3954/steps/15/logs/stdio oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-05-12 05:20:32+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3935/steps/15/logs/stdio (In reply to Richard Purdie from comment #7) > On a debian 11 system with python 3.9, the bitbake-cookerdaemon.log is full > of these messages. > > The oe-selftest failure is obviously bad timing of the garbage collection > but if we can remove them from the log generally, it would mean we likely > fix the oe-selftest issue even if we can't direction reproduce it. Reviewing the Python3 issues there was two tickets related to this behavior one found in 3.4/3.5 [1] and other in 3.7 - 3.11 [2]. Reading [2], I found Python 3.11 changed the whole implementation of asyncio/ssl [3]. Have this error been seen on newer Linux distros, we could add a warning suppress based on the Python version as workaround while we investigate more. [1] https://bugs.python.org/issue23243 [2] https://bugs.python.org/issue46318 [3] https://bugs.python.org/issue44011 oe-selftest-debian debian11-vk-3 master completed at 2026-05-22 03:59:11+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3893/steps/15/logs/stdio oe-selftest-debian debian11-vk-1 master completed at 2026-06-06 00:52:49+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3982/steps/15/logs/stdio oe-selftest-armhost ubuntu2204-vk-arm2 master-next completed at 2026-06-18 09:36:50+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4143/steps/15/logs/stdio oe-selftest-debian debian11-vk-1 contrib/ycongal/wrynose-nut completed at 2026-06-18 19:20:45+00:00 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/4070 oe-selftest-debian ubuntu2204-vk-1 master completed at 2026-06-22 02:41:46+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4116/steps/15/logs/stdio oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-06-21 03:09:16+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4168/steps/15/logs/stdio oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-06-28 03:09:51+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4217/steps/15/logs/stdio oe-selftest-armhost ubuntu2204-vk-arm2 wrynose completed at 2026-07-05 01:42:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/23/builds/4277 Not the same test failing but same bug: the "ResourceWarning: unclosed transport" message is injected into a path. Log: 2026-07-05 01:04:57,250 - oe-selftest - INFO - 1: 42/48 567/676 (2.45s) (0 failed) (recipetool.RecipetoolTests.test_recipetool_load_plugin) 2026-07-05 01:04:57,250 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/recipetool.py", line 892, in test_recipetool_load_plugin with open(srcfile) as fh: FileNotFoundError: [Errno 2] No such file or directory: 'WARNING: /usr/lib/python3.10/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0xffff080d6bc0>\n _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)\n\n/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1137552/meta-selftest/lib/recipetool/bbpath.py' oe-selftest-armhost ubuntu2204-vk-arm2 stable/2.18-nut&stable/wrynose-nut completed at 2026-07-05 01:42:36+00:00 https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4277/steps/15/logs/stdio I spoke with Anibal, he is okay if I take this one. (In reply to Francisco Pedraza from comment #19) > I spoke with Anibal, he is okay if I take this one. https://lists.openembedded.org/g/openembedded-core/message/242917 Patch merged: https://git.openembedded.org/bitbake/commit/?id=7a3cb0d55e3698e883a9ff8491febc9d155482ca The issue seems to be gone. |
We had this warning, just once so far. 2026-04-07 11:25:15,946 - oe-selftest - INFO - pkgdata.OePkgdataUtilTests.test_list_pkg_files (subunit.RemotedTestCase) 2026-04-07 11:25:15,946 - oe-selftest - INFO - ... FAIL ... Loading cache...WARNING: /usr/lib/python3.9/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f1421b1a940> _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)