Bug 16094

Summary: Generically improve cargo-based tests package reproducibility
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Yoann Congal <yoann.congal>
Component: devtools / tool chainAssignee: Unassigned <unassigned>
Status: IN PROGRESS DESIGN --- QA Contact:
Severity: enhancement    
Priority: Medium CC: ccasciato, elmehdi.younes, meta.mr.watcher, meta.watcher, randy.macleod, sundeep.kokkonda, SunilKumar.Dora, yoann.congal
Version: unspecified   
Target Milestone: 6.2   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Yoann Congal 2025-12-08 17:09:55 UTC
When compiling tests, cargo set environment variables to allow test binaries to find tests binaries, vectors, ...

For example:
CARGO_BIN_EXE_<name> — Absolute path to a binary target’s executable.
CARGO_MANIFEST_PATH  — The path to the manifest of your package.
OUT_DIR — [...] set to the folder where the build script should place its output

If tests binaries use this through the env!() macro (ie putting the var value into the source code), they are not reproducible.

Current cargo does not allow to override these variables.

A solution is to patch upstream packages to override cargo-defined variables (like https://github.com/rpm-software-management/rpm-sequoia/pull/86) but it is not generic and will have to be repeated on every Rust package that has integration tests.

The proper idea is to work with the cargo community to find a generic solution. Something like path remapping but for these variables?
Comment 1 Randy MacLeod 2025-12-11 16:01:43 UTC
CC Sundeep as FYI. 
Yoann will work on this bug.
Comment 2 Randy MacLeod 2026-03-31 14:37:56 UTC
Yoann, Do you still plan to work on this bug ?
Comment 3 Yoann Congal 2026-03-31 14:43:18 UTC
(In reply to Randy MacLeod from comment #2)
> Yoann, Do you still plan to work on this bug ?

Not myself but someone on my team (Mehdi) is working on it right now. He has a working PoC, I'll start reviewing code soon.
That will most likely NOT be done for 6.0 though (as it needs an upstream cargo approval)
Comment 4 Randy MacLeod 2026-03-31 21:52:21 UTC
Yoann, Thanks, if Medhi has a YP account they could take ownership of this issue.

Let us know if you want help for upstream or patch review, etc.

Bumping to 6.1.
Comment 5 Yoann Congal 2026-04-01 09:56:31 UTC
First piece of the puzzle:
Elyou/cargo ptest by elyouG · Pull Request #16816 · rust-lang/cargo
https://github.com/rust-lang/cargo/pull/16816
Comment 6 El Mehdi Younes 2026-04-03 07:19:00 UTC
Hi All, 
About the problem of OUT_DIR, I only saw it at rpm-sequoia for now. Cargo maintainers confirm that it is a Cargo variable which has been used by projects in ways that were not originally intended. Rpm-sequoia for example uses it to find the out_dir and then they keep going back until they find the lib, which is not too clean, while Cargo is instead trying to move towards a new solution with a new layout that separates build-dir and target-dir. 

CARGO_BIN_EXE_<name>, Cargo is now providing it at runtime, so maybe we can try to convince the others to use an std::env::var_os("CARGO_BIN_EXE_*") instead of the env!, so that we can specify this at runtime. 

For CARGO_MANIFEST_PATH, in my solution, I tried to override this variable when we are in test cases, which works with yocto-ptest, since for each test we have just one manifest-dir. The cargo maintainer is concerned about having this in officially in Cargo, since it may break some use cases of Manifest-dir that we don't specially know. We have an ongoing discussion about possible solutions. 

See: https://github.com/rust-lang/cargo/issues/16817
https://blog.rust-lang.org/2026/03/13/call-for-testing-build-dir-layout-v2/
Comment 7 El Mehdi Younes 2026-05-05 13:10:19 UTC
I am stepping down from this task due to time constraints, but I have done
some work: since Cargo 1.94, CARGO_BIN_EXE_<name> variables are 
provided at runtime. We must pivot to this approach instead of using 
compile-time hardcoded paths. 

To maintain compatibility with Cargo's recommendation of using a fallback 
while satisfying our Yocto reproducibility requirements, I propose 
implementing a "strict-paths" (feel free to change the name) feature 
in upstream projects. 

The implementation would look like this (ripgrep):

    #[cfg(feature = "strict-paths")]
    let rg = std::env::var_os("CARGO_BIN_EXE_rg")
        .map(PathBuf::from)
        .expect("CARGO_BIN_EXE_rg must be set at runtime");

    #[cfg(not(feature = "strict-paths"))]
    let rg = std::env::var_os("CARGO_BIN_EXE_rg")
        .map(PathBuf::from)
        .unwrap_or_else(|| PathBuf::from(env!("CARGO_BIN_EXE_rg")));

This allows projects to keep the Cargo-recommended `env!` fallback for 
standard usage, while we can enforce clean, runtime-only path resolution in 
our Yocto recipes by simply adding:

    CARGO_BUILD_FLAGS:append:task-compile-ptest-cargo = " --features strict-paths"

This approach prevents any hardcoded paths from being written into the 
resulting binaries during our build process.
Comment 8 El Mehdi Younes 2026-05-05 13:10:47 UTC
Regarding project resources, libraries, or test data, many projects rely on
`OUT_DIR` or `CARGO_MANIFEST_DIR` via `env!` macros to derive paths. 

My proposal is to shift entirely to runtime-provided paths, where the project
itself defines the environment variable needed, rather than relying on
cargo-injected variables.

To facilitate upstream adoption, I suggest using the same "strict-paths"
feature pattern. This allows us to present the change not as a radical
replacement, but as a robust compatibility layer:

    #[cfg(feature = "strict-paths")]
    let data_path = std::env::var_os("MY_PROJECT_DATA_DIR")
        .map(PathBuf::from)
        .expect("Path must be provided at runtime");

    #[cfg(not(feature = "strict-paths"))]
    let data_path = std::env::var_os("MY_PROJECT_DATA_DIR")
        .map(PathBuf::from)
        .unwrap_or_else(|| PathBuf::from(env!("OUT_DIR")).join("data"));

I am still awaiting a clear recommendation from the Cargo community to
validate this design before discussing it with other upstream projects. 
see https://github.com/rust-lang/cargo/issues/16817