Bug 16236

Summary: AB-INT: test_list_pkg_files: ResourceWarning: unclosed transport
Product: [QA/Testing] Functional (self) Testing Reporter: Mathieu Dubois-Briand <mathieu.dubois-briand>
Component: oe-selftestAssignee: Francisco Pedraza <pacopedraza87>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: anibal, JPEWhacker, pacopedraza87, randy.macleod, richard.purdie, tim.orling, yoann.congal
Version: unspecified   
Target Milestone: 6.1   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Mathieu Dubois-Briand 2026-04-09 08:31:51 UTC
We had this warning, just once so far.

2026-04-07 11:25:15,946 - oe-selftest - INFO - pkgdata.OePkgdataUtilTests.test_list_pkg_files (subunit.RemotedTestCase)
2026-04-07 11:25:15,946 - oe-selftest - INFO -  ... FAIL
...
Loading cache...WARNING: /usr/lib/python3.9/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f1421b1a940>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
Comment 1 Mathieu Dubois-Briand 2026-04-09 08:51:19 UTC
oe-selftest-debian debian11-vk-1 mathieu/master-next completed at 2026-04-07 12:22:14+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3618/steps/15/logs/stdio
Comment 2 Yoann Congal 2026-04-09 14:36:03 UTC
scarthgap oe-selftest-debian ubuntu2204-vk-1 https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/3612/steps/15/logs/stdio

2026-04-07 00:03:57,628 - oe-selftest - INFO - oelib.buildhistory.TestBlobParsing.test_compare_dict_blobs_default (subunit.RemotedTestCase)
2026-04-07 00:03:57,628 - oe-selftest - INFO -  ... FAIL

Stderr:
2026-04-06 22:32:36,881 - oe-selftest - INFO - Adding: "include selftest.inc" in /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3927156/conf/local.conf
2026-04-06 22:32:36,881 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
Loading cache...WARNING: /usr/lib/python3.10/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7f9063ee7460>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
Comment 3 Randy MacLeod 2026-04-09 14:46:22 UTC
Might be due to some recent denial of service attacks.
The two builds were about 12 hours apart.
Comment 4 Mathieu Dubois-Briand 2026-04-15 13:29:08 UTC
oe-selftest-debian ubuntu2204-vk-2 master completed at 2026-04-15 02:44:52+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3677/steps/15/logs/stdio
Comment 5 Joshua Watt 2026-04-23 15:21:51 UTC
Since this is reporting an unclosed SSL connection, this is almost certainly an unclosed websocket used by the hash equivalence server. There are only a few places where hash equivalence clients are created. Almost all of them use a `with` block to ensure that the client is properly cleaned up, except for one spot. In the SignatureGeneratorUniHashMixIn class in bb/lib/siggen.py, it has:

```
    @contextmanager
    def client(self):
        with self._client_env():
            if getattr(self, '_client', None) is None:
                self._client = hashserv.create_client(self.server, **self.get_hashserv_creds())
            yield self._client

    def reset(self, data):
        self.__close_clients()
        return super().reset(data)

    def exit(self):
        self.__close_clients()
        return super().exit()

    def __close_clients(self):
        with self._client_env():
            if getattr(self, '_client', None) is not None:
                self._client.close()
                self._client = None
            if getattr(self, '_client_pool', None) is not None:
                self._client_pool.close()
                self._client_pool = None
```

In this case, siggen will create a client (if one doesn't already exist) in the `client()` call, but it will not clean it up unless `reset()` or `exit()` is called; this seems like the most likely culprit as to the error.
Comment 6 Aníbal Limón 2026-04-23 17:21:40 UTC
(In reply to Joshua Watt from comment #5)
> Since this is reporting an unclosed SSL connection, this is almost certainly
> an unclosed websocket used by the hash equivalence server. There are only a
> few places where hash equivalence clients are created. Almost all of them
> use a `with` block to ensure that the client is properly cleaned up, except
> for one spot. In the SignatureGeneratorUniHashMixIn class in
> bb/lib/siggen.py, it has:
> 
> ```
>     @contextmanager
>     def client(self):
>         with self._client_env():
>             if getattr(self, '_client', None) is None:
>                 self._client = hashserv.create_client(self.server,
> **self.get_hashserv_creds())
>             yield self._client
> 
>     def reset(self, data):
>         self.__close_clients()
>         return super().reset(data)
> 
>     def exit(self):
>         self.__close_clients()
>         return super().exit()
> 
>     def __close_clients(self):
>         with self._client_env():
>             if getattr(self, '_client', None) is not None:
>                 self._client.close()
>                 self._client = None
>             if getattr(self, '_client_pool', None) is not None:
>                 self._client_pool.close()
>                 self._client_pool = None
> ```
> 
> In this case, siggen will create a client (if one doesn't already exist) in
> the `client()` call, but it will not clean it up unless `reset()` or
> `exit()` is called; this seems like the most likely culprit as to the error.

Thanks Joshua, I'll try to reproduce it and if succeed do a patch for it, may be a test case.
Comment 7 Richard Purdie 2026-04-29 09:13:44 UTC
On a debian 11 system with python 3.9, the bitbake-cookerdaemon.log is full of these messages.

The oe-selftest failure is obviously bad timing of the garbage collection but if we can remove them from the log generally, it would mean we likely fix the oe-selftest issue even if we can't direction reproduce it.
Comment 8 Mathieu Dubois-Briand 2026-05-12 08:22:46 UTC
oe-selftest-debian debian11-vk-1 master completed at 2026-05-07 04:37:57+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3804/steps/15/logs/stdio
Comment 9 Mathieu Dubois-Briand 2026-05-15 08:53:27 UTC
Is this one another synpto of the same issue? In recipetool.

oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-05-14 03:09:43+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3954/steps/15/logs/stdio

oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-05-12 05:20:32+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3935/steps/15/logs/stdio
Comment 10 Aníbal Limón 2026-05-15 16:05:43 UTC
(In reply to Richard Purdie from comment #7)
> On a debian 11 system with python 3.9, the bitbake-cookerdaemon.log is full
> of these messages.
> 
> The oe-selftest failure is obviously bad timing of the garbage collection
> but if we can remove them from the log generally, it would mean we likely
> fix the oe-selftest issue even if we can't direction reproduce it.

Reviewing the Python3 issues there was two tickets related to this behavior one found in 3.4/3.5 [1] and other in 3.7 - 3.11 [2].

Reading [2], I found Python 3.11 changed the whole implementation of asyncio/ssl [3].

Have this error been seen on newer Linux distros, we could add a warning suppress based on the Python version as workaround while we investigate more.

[1] https://bugs.python.org/issue23243 
[2] https://bugs.python.org/issue46318
[3] https://bugs.python.org/issue44011
Comment 11 Mathieu Dubois-Briand 2026-05-29 14:40:13 UTC
oe-selftest-debian debian11-vk-3 master completed at 2026-05-22 03:59:11+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3893/steps/15/logs/stdio
Comment 12 Mathieu Dubois-Briand 2026-06-09 09:10:48 UTC
oe-selftest-debian debian11-vk-1 master completed at 2026-06-06 00:52:49+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3982/steps/15/logs/stdio
Comment 13 Mathieu Dubois-Briand 2026-06-18 10:32:29 UTC
oe-selftest-armhost ubuntu2204-vk-arm2 master-next completed at 2026-06-18 09:36:50+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4143/steps/15/logs/stdio
Comment 14 Yoann Congal 2026-06-19 07:29:18 UTC
oe-selftest-debian debian11-vk-1 contrib/ycongal/wrynose-nut completed at 2026-06-18 19:20:45+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/35/builds/4070
Comment 15 Mathieu Dubois-Briand 2026-06-22 09:06:15 UTC
oe-selftest-debian ubuntu2204-vk-1 master completed at 2026-06-22 02:41:46+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4116/steps/15/logs/stdio

oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-06-21 03:09:16+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4168/steps/15/logs/stdio
Comment 16 Mathieu Dubois-Briand 2026-06-29 08:30:02 UTC
oe-selftest-armhost ubuntu2204-vk-arm2 master completed at 2026-06-28 03:09:51+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4217/steps/15/logs/stdio
Comment 17 Yoann Congal 2026-07-05 07:36:06 UTC
oe-selftest-armhost ubuntu2204-vk-arm2 wrynose completed at 2026-07-05 01:42:36+00:00
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/23/builds/4277

Not the same test failing but same bug: the "ResourceWarning: unclosed transport" message is injected into a path.

Log:
2026-07-05 01:04:57,250 - oe-selftest - INFO - 1: 42/48 567/676 (2.45s) (0 failed) (recipetool.RecipetoolTests.test_recipetool_load_plugin)
2026-07-05 01:04:57,250 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/recipetool.py", line 892, in test_recipetool_load_plugin
    with open(srcfile) as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'WARNING: /usr/lib/python3.10/asyncio/sslproto.py:320: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0xffff080d6bc0>\n  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)\n\n/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/build-st-1137552/meta-selftest/lib/recipetool/bbpath.py'
Comment 18 Mathieu Dubois-Briand 2026-07-21 09:09:02 UTC
oe-selftest-armhost ubuntu2204-vk-arm2 stable/2.18-nut&stable/wrynose-nut completed at 2026-07-05 01:42:36+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4277/steps/15/logs/stdio
Comment 19 Francisco Pedraza 2026-08-06 16:39:16 UTC
I spoke with Anibal, he is okay if I take this one.
Comment 20 Francisco Pedraza 2026-08-06 16:41:00 UTC
(In reply to Francisco Pedraza from comment #19)
> I spoke with Anibal, he is okay if I take this one.

https://lists.openembedded.org/g/openembedded-core/message/242917
Comment 21 Mathieu Dubois-Briand 2026-08-20 13:05:19 UTC
Patch merged: https://git.openembedded.org/bitbake/commit/?id=7a3cb0d55e3698e883a9ff8491febc9d155482ca

The issue seems to be gone.