16094 2025-12-08 17:09:55 +0000 Generically improve cargo-based tests package reproducibility 2026-10-29 17:52:56 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain unspecified x86 Multiple IN PROGRESS DESIGN Medium enhancement 6.2 1 yoann.congal unassigned ccasciato elmehdi.younes meta.mr.watcher meta.watcher randy.macleod sundeep.kokkonda SunilKumar.Dora yoann.congal --- --- No (bug/feature does not impact docs) oldest_to_newest 103509 0 yoann.congal 2025-12-08 17:09:55 +0000 When compiling tests, cargo set environment variables to allow test binaries to find tests binaries, vectors, ... For example: CARGO_BIN_EXE_<name> — Absolute path to a binary target’s executable. CARGO_MANIFEST_PATH — The path to the manifest of your package. OUT_DIR — [...] set to the folder where the build script should place its output If tests binaries use this through the env!() macro (ie putting the var value into the source code), they are not reproducible. Current cargo does not allow to override these variables. A solution is to patch upstream packages to override cargo-defined variables (like https://github.com/rpm-software-management/rpm-sequoia/pull/86) but it is not generic and will have to be repeated on every Rust package that has integration tests. The proper idea is to work with the cargo community to find a generic solution. Something like path remapping but for these variables? 103532 1 randy.macleod 2025-12-11 16:01:43 +0000 CC Sundeep as FYI. Yoann will work on this bug. 104872 2 randy.macleod 2026-03-31 14:37:56 +0000 Yoann, Do you still plan to work on this bug ? 104875 3 yoann.congal 2026-03-31 14:43:18 +0000 (In reply to Randy MacLeod from comment #2) > Yoann, Do you still plan to work on this bug ? Not myself but someone on my team (Mehdi) is working on it right now. He has a working PoC, I'll start reviewing code soon. That will most likely NOT be done for 6.0 though (as it needs an upstream cargo approval) 104887 4 randy.macleod 2026-03-31 21:52:21 +0000 Yoann, Thanks, if Medhi has a YP account they could take ownership of this issue. Let us know if you want help for upstream or patch review, etc. Bumping to 6.1. 104892 5 yoann.congal 2026-04-01 09:56:31 +0000 First piece of the puzzle: Elyou/cargo ptest by elyouG · Pull Request #16816 · rust-lang/cargo https://github.com/rust-lang/cargo/pull/16816 104948 6 elmehdi.younes 2026-04-03 07:19:00 +0000 Hi All, About the problem of OUT_DIR, I only saw it at rpm-sequoia for now. Cargo maintainers confirm that it is a Cargo variable which has been used by projects in ways that were not originally intended. Rpm-sequoia for example uses it to find the out_dir and then they keep going back until they find the lib, which is not too clean, while Cargo is instead trying to move towards a new solution with a new layout that separates build-dir and target-dir. CARGO_BIN_EXE_<name>, Cargo is now providing it at runtime, so maybe we can try to convince the others to use an std::env::var_os("CARGO_BIN_EXE_*") instead of the env!, so that we can specify this at runtime. For CARGO_MANIFEST_PATH, in my solution, I tried to override this variable when we are in test cases, which works with yocto-ptest, since for each test we have just one manifest-dir. The cargo maintainer is concerned about having this in officially in Cargo, since it may break some use cases of Manifest-dir that we don't specially know. We have an ongoing discussion about possible solutions. See: https://github.com/rust-lang/cargo/issues/16817 https://blog.rust-lang.org/2026/03/13/call-for-testing-build-dir-layout-v2/ 105357 7 elmehdi.younes 2026-05-05 13:10:19 +0000 I am stepping down from this task due to time constraints, but I have done some work: since Cargo 1.94, CARGO_BIN_EXE_<name> variables are provided at runtime. We must pivot to this approach instead of using compile-time hardcoded paths. To maintain compatibility with Cargo's recommendation of using a fallback while satisfying our Yocto reproducibility requirements, I propose implementing a "strict-paths" (feel free to change the name) feature in upstream projects. The implementation would look like this (ripgrep): #[cfg(feature = "strict-paths")] let rg = std::env::var_os("CARGO_BIN_EXE_rg") .map(PathBuf::from) .expect("CARGO_BIN_EXE_rg must be set at runtime"); #[cfg(not(feature = "strict-paths"))] let rg = std::env::var_os("CARGO_BIN_EXE_rg") .map(PathBuf::from) .unwrap_or_else(|| PathBuf::from(env!("CARGO_BIN_EXE_rg"))); This allows projects to keep the Cargo-recommended `env!` fallback for standard usage, while we can enforce clean, runtime-only path resolution in our Yocto recipes by simply adding: CARGO_BUILD_FLAGS:append:task-compile-ptest-cargo = " --features strict-paths" This approach prevents any hardcoded paths from being written into the resulting binaries during our build process. 105358 8 elmehdi.younes 2026-05-05 13:10:47 +0000 Regarding project resources, libraries, or test data, many projects rely on `OUT_DIR` or `CARGO_MANIFEST_DIR` via `env!` macros to derive paths. My proposal is to shift entirely to runtime-provided paths, where the project itself defines the environment variable needed, rather than relying on cargo-injected variables. To facilitate upstream adoption, I suggest using the same "strict-paths" feature pattern. This allows us to present the change not as a radical replacement, but as a robust compatibility layer: #[cfg(feature = "strict-paths")] let data_path = std::env::var_os("MY_PROJECT_DATA_DIR") .map(PathBuf::from) .expect("Path must be provided at runtime"); #[cfg(not(feature = "strict-paths"))] let data_path = std::env::var_os("MY_PROJECT_DATA_DIR") .map(PathBuf::from) .unwrap_or_else(|| PathBuf::from(env!("OUT_DIR")).join("data")); I am still awaiting a clear recommendation from the Cargo community to validate this design before discussing it with other upstream projects. see https://github.com/rust-lang/cargo/issues/16817