Bug 15959

Summary: Exclude rust failing tests in OE Self-Tests
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Jayasurya Maganuru <Maganuru.Jayasurya>
Component: devtools / tool chainAssignee: Deepesh Varatharajan <deepesh.varatharajan>
Status: NEW --- QA Contact:
Severity: normal    
Priority: Medium+ CC: meta.mr.watcher, meta.watcher, randy.macleod, sundeep.kokkonda
Version: unspecified   
Target Milestone: 6.1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
this attachments has the error log related to doc-tests, test-float-parse from rust v1.89.0 none

Description Jayasurya Maganuru 2025-08-28 12:50:49 UTC 
Created attachment 5136 [details]
this attachments has the error log related to doc-tests, test-float-parse from rust v1.89.0

Several OE self-tests fail despite a successful build:

 (tidy, coverage-dump, doc-tests, test-float-parse)
In rust v1.88.0
tools/coverage-dump
tools/tidy

In rust v1.89.0
tools/test-float-parse
doc-tests fail in approximately 55 crates.

To allow OE self-tests to run successfully, these tests need to be temporarily excluded from the test suite until the root causes are resolved.(https://github.com/yoctoproject/poky/blob/917430ed349f03869c9ca064f117bd74484090ee/meta/lib/oeqa/selftest/cases/rust.py#L62)


See the attachment for complete failure log from v1.89.0
Comment 1 Randy MacLeod 2025-08-28 14:35:49 UTC 
We'll fix this in the next upgrade, hopefully.
Parking at 5.3-M4 until the 6.0 milestones are created.
Comment 2 Randy MacLeod 2026-02-10 16:23:42 UTC 
Move back to 6.0-M4. 
We should work on minimizing the skipped tests for that milestone.
Also, medium+ bugs should be given an M* target not a 6.0, 6.1 milestone.

It would be great if we could eliminate all skipped tests but I understand that
this might take some time. Deepesh, can you comment on the 1.93 status and maybe attach the same output as was used for the v1.89.0 attachment so people can compare them and see where we are.
Comment 3 Randy MacLeod 2026-03-31 14:43:35 UTC 
Deepesh, update the numbers please.
Comment 4 Deepesh Varatharajan 2026-03-31 15:00:26 UTC 
With Rust-1.94, the following testcases are currently excluded:

Testcases - 37,
Unit-tests - 23,

Arch specific unit-test exclusion :

arm32 - 1,
ppc - 1,
riscv64 - 1
Comment 5 Deepesh Varatharajan 2026-04-01 09:41:54 UTC 
Below is a summary of testcase exclusions across previous and current Rust versions. Notably, the number (and percentage) of excluded tests has decreased over time:
 
+----------+----------+----------------+------------+---------------+--------+
| Rust Ver |  Tests*  | Excluded Tests | Unit Tests | Combined Excl | Excl % |
| -------- | ---------| -------------- | ---------- | ------------- | ------ |
| 1.94     | ~20,500  | 37             | 23         | 60            | 0.29%  |
| 1.90     | ~19,200  | 35             | 26         | 61            | 0.32%  |
| 1.84.1   | ~18,600  | 31             | 50         | 81            | 0.44%  |
| 1.79     | ~18,300  | 34             | 43         | 77            | 0.42%  |
| 1.75.0   | ~18,000  | (mixed list)   | —          | ~139          | 0.77%  |
+----------+----------+----------------+------------+---------------+--------+

More importantly, no new exclusions/ignores have been introduced in the last two Rust releases.

That said, we have not recently revalidated whether previously excluded/ignored testcases now pass with newer versions of rustc. This is something we plan to revisit and clean up going forward.