Bug 16403 - AB-INT PTEST: python3-packaging ptest failure in test_compatible_release_matches_expected_compatible
Summary: AB-INT PTEST: python3-packaging ptest failure in test_compatible_release_matc...
Status: RESOLVED FIXED
Alias: None
Product: Package Testing (ptest)
Classification: QA/Testing
Component: ptest (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 6.1 M4
Assignee: Ross Burton
QA Contact:
URL:
Whiteboard: AB-INT
Depends on:
Blocks:
 
Reported: 2026-08-20 09:15 UTC by João Marcos Costa
Modified: 2026-10-20 16:12 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description João Marcos Costa 2026-08-20 09:15:15 UTC
It happened twice, this month:

Failed ptests:
{'python3-packaging': ['tests/property/test_specifier_matching.py:TestCompatibleReleaseDefinition.test_compatible_release_matches_expected_compatible']}

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/4250/steps/16/logs/stdio
qemux86-64-ptest opensuse156-vk-1


https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4263/steps/15/logs/stdio
qemuarm64-ptest ubuntu2404-vk-arm2

As per the logs:

---
self = <tests.property.test_specifier_matching.TestCompatibleReleaseDefinition object at 0xffff7c744550>

    @given(spec_ver=multi_segment_versions(), candidate=pep440_versions())
>   @SETTINGS
              
E   hypothesis.errors.FailedHealthCheck: It looks like this test is filtering out a lot of inputs. 8 inputs were generated successfully, while 50 inputs were filtered out. 
E   
E   An input might be filtered out by calls to assume(), strategy.filter(...), or occasionally by Hypothesis internals.
E   
E   Applying this much filtering makes input generation slow, since Hypothesis must discard inputs which are filtered out and try generating it again. It is also possible that applying this much filtering will distort the domain and/or distribution of the test, leaving your testing less rigorous than expected.
E   
E   If you expect this many inputs to be filtered out during generation, you can disable this health check with @settings(suppress_health_check=[HealthCheck.filter_too_much]). See https://hypothesis.readthedocs.io/en/latest/reference/api.html#hypothesis.HealthCheck for details.

tests/property/test_specifier_matching.py:202: FailedHealthCheck
FAIL: tests/property/test_specifier_matching.py:TestCompatibleReleaseDefinition.test_compatible_release_matches_expected_compatible
---

This could be related to the recent updates in python3-hypothesis as well, but this ptest failure didn't reproduce since the very last bump: Sat Aug 15 13:46:21 2026 +0100 (2ff674cb6167: "python3-hypothesis: upgrade 6.165.2 -> 6.165.9")

Full logs at: https://valkyrie.yocto.io/pub/non-release/20260812-79/testresults/qemuarm64-ptest/python3-packaging.log
Comment 1 Randy MacLeod 2026-08-20 14:48:18 UTC
Transient upgrade issue, likely.
Re-open if seen again.
Comment 2 João Marcos Costa 2026-08-24 09:48:01 UTC
Some other occurrences from August 20th:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/4356/steps/16/logs/stdio

qemux86-64-ptest debian11-vk-1


https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1022/steps/15/logs/stdio

qemuarm64-musl-ptest debian13-vk-arm1


They are from the same day I filed the bug, so will not reopen right away.
Comment 3 João Marcos Costa 2026-09-10 08:41:22 UTC
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1121/steps/15/logs/stdio

Failed ptests:
{'python3-packaging': ['tests/property/test_specifier_matching.py:TestCompatibleReleaseDefinition.test_compatible_release_matches_expected_compatible']}

qemuarm64-musl-ptest stream10-vk-arm1
Comment 4 Ross Burton 2026-09-10 16:19:17 UTC
Looks like an upstream issue with hypothesis, filed https://github.com/pypa/packaging/issues/1406.

I'll give them a few days to respond, it appears to be limited to a single test so we can skip that temporarily if needed.
Comment 5 Ross Burton 2026-09-16 13:58:09 UTC
Patch to fix this on the list.