15808 2025-03-24 17:44:08 +0000 Investigate implementing shared-source for Rust 2026-07-30 12:21:28 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 5.3 x86 Multiple RESOLVED FIXED Medium enhancement 6.1 M3 1 randy.macleod deepesh.varatharajan deepesh.varatharajan meta.mr.watcher meta.watcher ross.burton sundeep.kokkonda yoann.congal --- --- Don't know oldest_to_newest 101475 0 randy.macleod 2025-03-24 17:44:08 +0000 Consider implementing shared-source for Rust along the lines but hopefully in a better way than the gcc shared source implementation: meta/recipes-devtools/gcc/gcc-shared-source.inc This only makes sense if rust do an out of tree build. Background: In reviewing the patch: rust: Fix bloat issue in recent rust updates https://lore.kernel.org/openembedded-core/20250321132502.2607941-1-Deepesh.Varatharajan@windriver.com/ people objected since that the clean-up would only apply to the rust source unpack for the target whereas adding the unpack rule to rust-source.inc would save space for all the recipes that unpack rust:0 --- ❯ rg rust-source meta/recipes-devtools/rust/ meta/recipes-devtools/rust/cargo_1.83.0.bb 14:require rust-source.inc meta/recipes-devtools/rust/rust_1.83.0.bb 230:require rust-source.inc meta/recipes-devtools/rust/rust-llvm_1.83.0.bb 9:require rust-source.inc meta/recipes-devtools/rust/libstd-rs_1.83.0.bb 7:require rust-source.inc --- and the idea that we wouldn't have to worry about that and would save more FS space if these rust recipes used a shared rust source tree. 101566 1 randy.macleod 2025-04-05 00:54:55 +0000 Sundeep I decided to check if the rust build script (x.py) can be run from outside the SRCDIR to see if a shared dir is even possible. It is. You can build rust, out of tree: ❯ tar xf /.../downloads/rustc-1.85.1-src.tar.xz ❯ mkdir b1 ❯ ls b1 rustc-1.85.1-src ❯ cd b1/ ❯ ../rustc-1.85.1-src/x.py INFO: Downloading and building bootstrap before processing --help command. See src/bootstrap/README.md for help with common commands. downloading https://static.rust-lang.org/dist/2025-01-09/rust-std-1.84.0-x86_64-unknown-linux-gnu.tar.xz ####################################################################################################################################################################################### 100.0% downloading https://static.rust-lang.org/dist/2025-01-09/rustc-1.84.0-x86_64-unknown-linux-gnu.tar.xz ####################################################################################################################################################################################### 100.0% downloading https://static.rust-lang.org/dist/2025-01-09/cargo-1.84.0-x86_64-unknown-linux-gnu.tar.xz ####################################################################################################################################################################################### 100.0% extracting /media/rmacleod/gitter/rmacleod/src/lang/rust/1.85.1/b1/build/cache/2025-01-09/rust-std-1.84.0-x86_64-unknown-linux-gnu.tar.xz extracting /media/rmacleod/gitter/rmacleod/src/lang/rust/1.85.1/b1/build/cache/2025-01-09/cargo-1.84.0-x86_64-unknown-linux-gnu.tar.xz extracting /media/rmacleod/gitter/rmacleod/src/lang/rust/1.85.1/b1/build/cache/2025-01-09/rustc-1.84.0-x86_64-unknown-linux-gnu.tar.xz Building bootstrap Compiling proc-macro2 v1.0.89 Compiling unicode-ident v1.0.13 Compiling memchr v2.7.4 Compiling version_check v0.9.5 ... < eventually prints help text> ❯ ../rustc-1.85.1-src/x.py build ... Compiling rustc_transmute v0.0.0 (/.../rust/1.85.1/rustc-1.85.1-src/compiler/rustc_transmute) ... Finished `release` profile [optimized] target(s) in 32.75s WARNING: you have not made a `config.toml` HELP: consider running `./x.py setup` or copying `config.example.toml` by running `cp config.example.toml config.toml` NOTE: this message was printed twice to make it more likely to be seen Build completed successfully in 0:04:21 It seems to be working. Right Sundeep? Note this is the first time in a while that I'm building rust src outside of bitbake so please let me know if there's a more sensible check. So, next steps are: 1. determine what stages need the bootstrap and can it be shared if needed. 2. figure out how do we structure the recipes to do the src sharing (see gcc-shared for what RP says is a bad example) 3. Calculate the download time/space savings and decide if it's worth it. ../Randy 102614 2 randy.macleod 2025-08-15 18:07:26 +0000 The rust release tarball is very large, so as part of this work, we should look at alternatives to using the main tarball. For 1.88 (rust-native unpack): the rustc-1.88.0-src.tar.xz file is 271MB and the unpacked source is ~4 GB. 2 GB is under the vendor dir, with 750 MB being windows related and not useful. some of the vendor directory source may already be in oe-core as a recipe. The github rust-1.88 tarball: https://github.com/rust-lang/rust/archive/refs/tags/1.88.0.tar.gz is only 31 MB compressed, and 332 MB unpacked. Of course it doesn't contain the llvm-clone which we are hoping to get rid of, tools, docs, and other mostly useful stuff from the src dir. The llvm-clone includes llvm, clang, lldb, compiler-rt, and more. We need to explain this problem to upstream and see if other people are concerned about the large default distribution Some ncdu screens: --- /...nux/rust-native/1.88.0/sources --- /.. 3.9 GiB [##########] /rustc-1.88.0-src 548.7 MiB [# ] /rust-snapshot-components 128.0 KiB [ ] 0001-Define-more-...unknown-linux-g.patch 44.0 KiB [ ] 0001-riscv32-Defi...r-time64-varian.patch 12.0 KiB [ ] rust-oe-selftest.patch 12.0 KiB [ ] revert-link-std-s..._driver-feature.patch 12.0 KiB [ ] repro-issue-fix-w...c-crate-hashmap.patch 4.0 KiB [ ] 0001-src-core-bui...ff-lto-for-rust.patch 4.0 KiB [ ] 0001-libunwind-Us...arget-attribute.patch 4.0 KiB [ ] 0001-Disable-libu...cture-unwinding.patch 4.0 KiB [ ] oeqa-selftest-Inc...cess-sigpipe-ru.patch 4.0 KiB [ ] triagebot.patch --- /...e/1.88.0/sources/rustc-1.88.0-src --- /.. 2.0 GiB [##########] /vendor 1.7 GiB [######## ] /src 174.6 MiB [ ] /tests 45.9 MiB [ ] /library 37.7 MiB [ ] /compiler 804.0 KiB [ ] RELEASES.md 144.0 KiB [ ] Cargo.lock 124.0 KiB [ ] /LICENSES 44.0 KiB [ ] bootstrap.example.toml 12.0 KiB [ ] LICENSE-APACHE 12.0 KiB [ ] license-metadata.json 8.0 KiB [ ] REUSE.toml --- /...0/sources/rustc-1.88.0-src/vendor --- 121.9 MiB [##########] /windows-0.58.0 121.7 MiB [######### ] /windows-0.57.0 115.4 MiB [######### ] /windows-0.59.0 113.4 MiB [######### ] /windows-0.61.1 53.7 MiB [#### ] /winapi-x86_64-pc-windows-gnu-0.4.0 51.5 MiB [#### ] /winapi-i686-pc-windows-gnu-0.4.0 50.9 MiB [#### ] /openssl-src-300.4.2+3.4.1 41.6 MiB [### ] /plotters-0.3.1 31.4 MiB [## ] /windows-bindgen-0.61.0 30.5 MiB [## ] /windows-sys-0.48.0 27.0 MiB [## ] /openssl-src-111.17.0+1.1.1m ... 2.1 MiB [ ] /syn-1.0.85 2.1 MiB [ ] /syn-1.0.86 2.1 MiB [ ] /syn-1.0.89 2.0 MiB [ ] /idna-0.1.5 2.0 MiB [ ] /object-0.36.7 ... $ ls /.../rustc-1.88.0-src/vendor | wc -l 1703 --- /...88.0/sources/rustc-1.88.0-src/src --- /.. 1.0 GiB [##########] /llvm-project 685.1 MiB [###### ] /tools 46.3 MiB [ ] /doc 4.0 MiB [ ] /librustdoc 3.0 MiB [ ] /etc 1.6 MiB [ ] /bootstrap 1.3 MiB [ ] /ci 84.0 KiB [ ] /build_helper 68.0 KiB [ ] /rustdoc-json-types 56.0 KiB [ ] /rustc-std-workspace 48.0 KiB [ ] stage0 8.0 KiB [ ] /gcc 4.0 KiB [ ] README.md 4.0 KiB [ ] version --- /...rustc-1.88.0-src/src/llvm-project --- /.. 409.1 MiB [##########] /clang 390.4 MiB [######### ] /llvm 86.2 MiB [## ] /lldb 50.8 MiB [# ] /compiler-rt 50.2 MiB [# ] /clang-tools-extra 22.8 MiB [ ] /lld 1.0 MiB [ ] /libunwind 72.0 KiB [ ] /runtimes 72.0 KiB [ ] /cmake 40.0 KiB [ ] /llvm-libgcc --- /...ources/rustc-1.88.0-src/src/tools --- /.. 391.1 MiB [##########] /enzyme 194.7 MiB [#### ] /rustc-perf 28.6 MiB [ ] /cargo 24.0 MiB [ ] /clippy 23.9 MiB [ ] /rust-analyzer 10.0 MiB [ ] /rustfmt 10.0 MiB [ ] /miri 624.0 KiB [ ] /compiletest ... $ ls .../rustc-1.88.0-src/src/tools/ | wc -l 48 106379 3 deepesh.varatharajan 2026-07-30 12:21:28 +0000 Fixed https://git.openembedded.org/openembedded-core/commit/?id=395fe161362c5b6d39a9b01e938d99567c6273c3