15493 2024-05-24 13:59:50 +0000 sstate reuse not as ideal as it should be 2024-05-30 14:34:58 +0000 1 1 7 Build System, Metadata & Runtime OE-Core core 0.0.0 x86 Multiple RESOLVED FIXED Undecided normal 0.0.0 1 richard.purdie unassigned meta.mr.watcher meta.watcher ross.burton tgamblin --- --- No (bug/feature does not impact docs) oldest_to_newest 99077 0 richard.purdie 2024-05-24 13:59:50 +0000 I had a world build complete then came back and ran another build the next day having only made bitbake changes: https://valkyrie.yoctoproject.org/#/builders/17/builds/18/steps/11/logs/stdio Sstate summary: Wanted 19990 Local 17552 Mirrors 0 Missed 2438 Current 0 (87% match, 0% complete) There is a lot of sstate matching but there were also rebuilds of base-files, connman, bluez, linux-yocto and more which was unexpected. Theories: * the revision changed, changing base-files and things have a dependency on base-files, maybe useradd related which ripple causing more rebuilds than they should * somehow DATE/TIME may also come into play We should invesitgate the dependencies and fix them so we can see better sstate reuse. 99078 1 ross.burton 2024-05-24 14:39:43 +0000 I suspect I've seen this before but not reliably. My hunch was the DISTRO_VERSION gets into the dependency tree somewhow. 99079 2 richard.purdie 2024-05-24 14:49:35 +0000 https://valkyrie.yoctoproject.org/#/builders/17/builds/19/steps/12/logs/stdio was a following build without changing anything. sstate reuse is better but some things were still rebuilding (why?) I then changed the master-next revision and started: https://valkyrie.yoctoproject.org/#/builders/17/builds/20 99080 3 richard.purdie 2024-05-24 14:55:30 +0000 Changing the master-next revision dropped things back to: Sstate summary: Wanted 19990 Local 17552 Mirrors 0 Missed 2438 Current 0 (87% match, 0% complete) when the repeat build was 98% 99081 4 richard.purdie 2024-05-24 15:20:01 +0000 This is probably systemd depending on os-release. I'm testing a tweak for that but there may be further issues. 99083 5 ross.burton 2024-05-24 15:39:02 +0000 I did a build of systemd. Touched and committed a file in poky, and built systemd: NOTE: recipe base-files-3.0.14-r0: task do_install: Started NOTE: recipe os-release-1.0-r0: task do_compile: Started NOTE: recipe os-release-1.0-r0: task do_install: Started NOTE: recipe base-files-3.0.14-r0: task do_populate_sysroot: Started NOTE: recipe base-files-3.0.14-r0: task do_package: Started NOTE: recipe os-release-1.0-r0: task do_package: Started NOTE: recipe os-release-1.0-r0: task do_packagedata: Started NOTE: recipe base-files-3.0.14-r0: task do_packagedata: Started NOTE: recipe os-release-1.0-r0: task do_create_spdx: Started NOTE: recipe os-release-1.0-r0: task do_package_write_ipk: Started NOTE: recipe systemd-1_255.4-r0: task do_package_write_ipk_setscene: Started NOTE: recipe systemd-1_255.4-r0: task do_package_qa_setscene: Started NOTE: recipe systemd-1_255.4-r0: task do_create_runtime_spdx: Started So nobody _really_ notices because systemd only repackages, but this is definitely a path where SHAs changing causes rebuilds. 99097 6 richard.purdie 2024-05-30 07:54:27 +0000 (In reply to Ross Burton from comment #5) > I did a build of systemd. Touched and committed a file in poky, and built > systemd: > > NOTE: recipe base-files-3.0.14-r0: task do_install: Started > NOTE: recipe os-release-1.0-r0: task do_compile: Started > NOTE: recipe os-release-1.0-r0: task do_install: Started > NOTE: recipe base-files-3.0.14-r0: task do_populate_sysroot: Started > NOTE: recipe base-files-3.0.14-r0: task do_package: Started > NOTE: recipe os-release-1.0-r0: task do_package: Started > NOTE: recipe os-release-1.0-r0: task do_packagedata: Started > NOTE: recipe base-files-3.0.14-r0: task do_packagedata: Started > NOTE: recipe os-release-1.0-r0: task do_create_spdx: Started > NOTE: recipe os-release-1.0-r0: task do_package_write_ipk: Started > NOTE: recipe systemd-1_255.4-r0: task do_package_write_ipk_setscene: Started > NOTE: recipe systemd-1_255.4-r0: task do_package_qa_setscene: Started > NOTE: recipe systemd-1_255.4-r0: task do_create_runtime_spdx: Started > > So nobody _really_ notices because systemd only repackages, but this is > definitely a path where SHAs changing causes rebuilds. Debian renaming dependencies will mean anything with a runtime dependency on systemd would also repackage so this does cause a lot of rebuilding of packages so it is best fixed! 99100 7 tgamblin 2024-05-30 14:34:58 +0000 Fixed here: https://git.yoctoproject.org/poky/commit/?id=7ec8dcfcef5b169c66c81d018551c96d9730e1aa