Bug 15959 - Exclude rust failing tests in OE Self-Tests
Summary: Exclude rust failing tests in OE Self-Tests
Status: NEW
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium+ normal
Target Milestone: 6.1
Assignee: Deepesh Varatharajan
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2025-08-28 12:50 UTC by Jayasurya Maganuru
Modified: 2026-05-28 13:40 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
this attachments has the error log related to doc-tests, test-float-parse from rust v1.89.0 (34.17 KB, text/plain)
2025-08-28 12:50 UTC, Jayasurya Maganuru
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jayasurya Maganuru 2025-08-28 12:50:49 UTC
Created attachment 5136 [details]
this attachments has the error log related to doc-tests, test-float-parse from rust v1.89.0

Several OE self-tests fail despite a successful build:

 (tidy, coverage-dump, doc-tests, test-float-parse)
In rust v1.88.0
tools/coverage-dump
tools/tidy

In rust v1.89.0
tools/test-float-parse
doc-tests fail in approximately 55 crates.

To allow OE self-tests to run successfully, these tests need to be temporarily excluded from the test suite until the root causes are resolved.(https://github.com/yoctoproject/poky/blob/917430ed349f03869c9ca064f117bd74484090ee/meta/lib/oeqa/selftest/cases/rust.py#L62)


See the attachment for complete failure log from v1.89.0
Comment 1 Randy MacLeod 2025-08-28 14:35:49 UTC
We'll fix this in the next upgrade, hopefully.
Parking at 5.3-M4 until the 6.0 milestones are created.
Comment 2 Randy MacLeod 2026-02-10 16:23:42 UTC
Move back to 6.0-M4. 
We should work on minimizing the skipped tests for that milestone.
Also, medium+ bugs should be given an M* target not a 6.0, 6.1 milestone.

It would be great if we could eliminate all skipped tests but I understand that
this might take some time. Deepesh, can you comment on the 1.93 status and maybe attach the same output as was used for the v1.89.0 attachment so people can compare them and see where we are.
Comment 3 Randy MacLeod 2026-03-31 14:43:35 UTC
Deepesh, update the numbers please.
Comment 4 Deepesh Varatharajan 2026-03-31 15:00:26 UTC
With Rust-1.94, the following testcases are currently excluded:

Testcases - 37,
Unit-tests - 23,

Arch specific unit-test exclusion :

arm32 - 1,
ppc - 1,
riscv64 - 1
Comment 5 Deepesh Varatharajan 2026-04-01 09:41:54 UTC
Below is a summary of testcase exclusions across previous and current Rust versions. Notably, the number (and percentage) of excluded tests has decreased over time:
 
+----------+----------+----------------+------------+---------------+--------+
| Rust Ver |  Tests*  | Excluded Tests | Unit Tests | Combined Excl | Excl % |
| -------- | ---------| -------------- | ---------- | ------------- | ------ |
| 1.94     | ~20,500  | 37             | 23         | 60            | 0.29%  |
| 1.90     | ~19,200  | 35             | 26         | 61            | 0.32%  |
| 1.84.1   | ~18,600  | 31             | 50         | 81            | 0.44%  |
| 1.79     | ~18,300  | 34             | 43         | 77            | 0.42%  |
| 1.75.0   | ~18,000  | (mixed list)   | —          | ~139          | 0.77%  |
+----------+----------+----------------+------------+---------------+--------+

More importantly, no new exclusions/ignores have been introduced in the last two Rust releases.

That said, we have not recently revalidated whether previously excluded/ignored testcases now pass with newer versions of rustc. This is something we plan to revisit and clean up going forward.