When the kernel is rebuild the modules is not rebuild as well and will comes from the sstate-cache. The following procedure is made using the LmP from foundries.io but I think the same works with if we use the core-image-minimal: #---------------------------------------------------------------------------------- # 1- install some kernel module in your image CORE_IMAGE_BASE_INSTALL += "cryptodev-module" #---------------------------------------------------------------------------------- # 2- rebuild and generate the siggen hashes as well as populate the sstate-cache bitbake core-image-minimal #---------------------------------------------------------------------------------- # 3 - change something in do_shared_workdir and check diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index 8299b394a7..29a147e2a5 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -586,6 +586,11 @@ do_shared_workdir () { fi } +do_shared_workdir:append () { + rm -vf $kerneldir/tmp.* + mktemp -p $kerneldir +} + # We don't need to stage anything, not the modules/firmware since those would clash with linux-firmware sysroot_stage_all () { : #---------------------------------------------------------------------------------- # 4 - check what needs to rebuild bitbake core-image-minimal -S printdiff Task linux-lmp-fslc-imx:do_shared_workdir couldn't be used from the cache because: We need hash 85c99187b5e563213439668b6724998f4126f034314fc34082d9daf1696a54d2, closest matching task was 0e96468fb9118db739fa78ae1a4ba120f2b1d23994721f7f6146436b1e572254 basehash changed from 11e552d3eef656d703d5ef3aaf8a39b6f0fcb0c3995f5596dad4d15c3b3b0efd to 5032428c730a2dc1a3eed61748ca751b5d1567666f7ca3f46d6d04772a80c149 Variable do_shared_workdir value changed: @@ -60,4 +60,6 @@ cp tools/objtool/objtool ${kerneldir}/tools/objtool/ fi fi + rm -vf $kerneldir/tmp.* + mktemp -p $kerneldir cp modsign_key.pem $kerneldir/ #---------------------------------------------------------------------------------- # 5 - the kernel change (what is right and was expected) so rebuild and generate a new siggen bitbake virtual/kernel #---------------------------------------------------------------------------------- # 6 - and finally check the image again bitbake core-image-minimal -S printdiff The cryptodev-module in my image seems not exist for bitbake, because it won't be rebuilt and will come from the sstate-cache which was what I wanted to demonstrate. Adding in module.bbclass virtual/kernel as dependency solves the issue and the module is rebuild
Did you see any undesired side effects?
So far we don't have any side effects and we've been using it in some kernel modules for some time
So you could prepare a formal patch and send it to OE-Core so it can go over CI tests.
Is good ideia if it can be tested on the CI, Patch send and add you in cc as well https://lists.openembedded.org/g/openembedded-core/message/169253
I have reproduce the same issue without changing the do_shared_workdir and confirm this using the OE-core with the core-image-minimal. #---------------------------------------------------------------------------------- # 1 - local.conf will have two kernel modules where one of them depends on virtual/kernel: IMAGE_INSTALL:append = " lttng-modules" IMAGE_INSTALL:append = " cryptodev-module" DEPENDS:append:pn-cryptodev-module = " virtual/kernel" #---------------------------------------------------------------------------------- # 2 - prepare rebuilding and generate the siggen hashes as well as populate the sstate-cache bitbake virtual/kernel lttng-modules cryptodev-module core-image-minimal -c cleansstate && bitbake core-image-minimal -c do_rootfs #---------------------------------------------------------------------------------- # 3 - change a kernel task in local.conf (it affects kernel_do_compile and do_compile_kernelmodules) KERNEL_EXTRA_ARGS:append = " THIS_IS_ONLY_TO_INVALDATE_THE_BITBAKE_TASKS=0 " #---------------------------------------------------------------------------------- # 4 - and finally check the image again bitbake core-image-minimal -c do_rootfs -S printdiff Task cryptodev-module:do_prepare_recipe_sysroot couldn't be used from the cache because: We need hash 0e5c8044bde34d808dc93235d21476e23d11ae3386d18fe3054f3d7782ff0882, closest matching task was 5a0d090d05ed4cdcf2cd3169b3ebad32edba188d8dcc93d8dfdcdbe7b6fdbb12 Hash for task dependency linux/linux-lmp_git.bb:do_populate_sysroot changed from cdad89a22827db08348bb6f941ba77b35eb84f954ab8a813763f7ec4e0601e7d to 724402a0e0774d5f7486dcfff04b193b32e15d80f31e53ec0dd1bed7fd9ddba1 Hash for task dependency linux/linux-lmp_git.bb:do_install changed from 95676d5a45ae3f6d1cce1541161646f75f7371e01af85035f4932b8ac1bdfaf7 to a6babf8ff0e8a7542bd0a00c02db2a5dc42868c16f52df7d8e85e6a4e1350d25 Hash for task dependency linux/linux-lmp_git.bb:do_transform_kernel changed from 2e35142a949ba2900e808df567ecb5ce96078974045c259a0c11dc1ed91a2aa9 to a2001bf9d04202e45e374cdae9c39c72c7be719391f896b22ea9f567075aaa9d Hash for task dependency linux/linux-lmp_git.bb:do_compile changed from f9e0d5cf1e243d90c751811bc0617f7ee364a72b9ca8dfd91895ea2ee665e8b0 to 9da921c93660f3e2457db2e31deddb3da0aff10d2ab9697afaf956cda57dd9b7 basehash changed from a0fe87e699eb3cbb89a6114f85541b47ad178db9bec38f841bfb6920260380f2 to 542e16d6f70dfdd9f224add286c4a87f9dd291ea6cdf8396daf7c31cd749e639 Variable KERNEL_EXTRA_ARGS value changed: @@ -1,2 +1,2 @@ - ${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)} + THIS_IS_ONLY_TO_INVALDATE_THE_BITBAKE_TASKS=0 ${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)} KERNEL_DEBUG_OPTIONS{stack} = Set Task make-mod-scripts:do_configure couldn't be used from the cache because: We need hash 279a5c90b981e7a4360feb28633d0511ed0ce32986e7be033aa20e690a87d550, closest matching task was 816f3c9b74d7a2ebc4136d4fee6fa977aa2219e855d3a33e4b7fc2501e245b78 Hash for task dependency linux/linux-lmp_git.bb:do_shared_workdir changed from 82a2f548515312ccd4abb0579e7c7925ece186cd661472fc5af4f46533dd1249 to 097b22a5aad87054ecff9b7d675514beb1d0a0458020b933f1c99e93058f9157 Hash for task dependency linux/linux-lmp_git.bb:do_compile changed from f9e0d5cf1e243d90c751811bc0617f7ee364a72b9ca8dfd91895ea2ee665e8b0 to 9da921c93660f3e2457db2e31deddb3da0aff10d2ab9697afaf956cda57dd9b7 basehash changed from a0fe87e699eb3cbb89a6114f85541b47ad178db9bec38f841bfb6920260380f2 to 542e16d6f70dfdd9f224add286c4a87f9dd291ea6cdf8396daf7c31cd749e639 Variable KERNEL_EXTRA_ARGS value changed: @@ -1,2 +1,2 @@ - ${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)} + THIS_IS_ONLY_TO_INVALDATE_THE_BITBAKE_TASKS=0 ${@bb.utils.contains('KERNEL_DEBUG_OPTIONS', 'stack', 'HOST_LIBELF_LIBS="-L${RECIPE_SYSROOT_NATIVE}/usr/lib/pkgconfig/../../../usr/lib/ -lelf"', '', d)} KERNEL_DEBUG_OPTIONS{stack} = Set bitbake will rebuild only the kernel module cryptodev-module, the lttng-modules will comes from the sstate cache
When repeating the steps verified that to get the same output of step 4 we need to rebuild the kernel at the end of step 3 with: bitbake virtual/kernel
I have also encountered this bug using a different procedure after upgrading from Langdale to Mickledore (Yocto 4.2.1) on x86-64. Below are my notes. I am using multiple customized copies of linux-yocto-custom.bb, one for each of several different kernel versions. I also have a number of external kernel modules. When I change PREFERRED_VERSION_linux-yocto-custom back and forth between two different versions and build the firmware image, bitbake doesn't rebuild the external kernel modules, resulting in do_rootfs failing due to the version mismatch between the kernel and modules. This used to work in Langdale but broke in Mickledore. It seems the virtual/kernel:do_shared_workdir task is not being re-run. To reproduce, first get both versions into the sstate cache: PREFERRED_VERSION_linux-yocto-custom := "5.15.119" bitbake virtual/kernel PREFERRED_VERSION_linux-yocto-custom := "6.1.36" bitbake virtual/kernel Then switch versions: PREFERRED_VERSION_linux-yocto-custom := "5.15.119" bitbake virtual/kernel (uses cached version instead of rebuilding) bitbake external-kernel-module cat tmp/work-shared/genericx86-64/kernel-build-artifacts/kernel-abiversion 6.1.36-yocto-base-cyber ls tmp/work/genericx86_64-poky-linux/external-kernel-module/version/image/lib/modules 6.1.36-yocto-base-cyber (the shared kernel source and the external module are still for the previous kernel version) bitbake -c cleansstate virtual/kernel bitbake virtual/kernel bitbake external-kernel-module cat tmp/work-shared/genericx86-64/kernel-build-artifacts/kernel-abiversion 5.15.119-yocto-base-cyber ls tmp/work/genericx86_64-poky-linux/external-kernel-module/version/image/lib/modules 6.1.36-yocto-base-cyber (the shared kernel source is now for the correct kernel version, but the external module is still built for the previous kernel version) bitbake -c cleansstate external-kernel-module bitbake external-kernel-module ls tmp/work/genericx86_64-poky-linux/external-kernel-module/version/image/lib/modules 5.15.119-yocto-base-cyber (now everything has been switched over) Same thing happens every time switching back and forth between the two versions. The best workaround is to "bitbake -c cleansstate virtual/kernel external-kernel-module" immediately after changing PREFERRED_VERSION_linux-yocto-custom. Poking around in the source it looks like this might have something to do with setscene_depvalid(), but I don't know too much about it. I tried this patch but it didn't fix my problem: https://lists.openembedded.org/g/openembedded-core/message/169253
This still seems to be an issue, at least in Scarthgap. Does anyone have any decent workarounds for it?
I have just been using bitbake -c cleansstate virtual/kernel name-of-external-kernel-module.
I still use something like this in my distro and it work for us https://lists.openembedded.org/g/openembedded-core/message/169253 A global class with: python __anonymous() { if bb.data.inherits_class('module', d): d.appendVar('DEPENDS', ' virtual/kernel') }
The workaround unfortunately did not work for me. I came across this thread today which seems relevant: https://lists.openembedded.org/g/openembedded-core/message/215446. I'm going to try disabling LOCALVERSION_AUTO to see if it helps.
Relevant commit in master: https://git.yoctoproject.org/poky/commit/?id=2d2d17e13665f767de8b2b0b33f6b2debb174019 I haven't tested it though.
I'm wondering if a rust-module error in master-next that has the symptom that the kernel module was not present is related. Sundeep? Please CC others as needed.
Thanks Randy. So far we see 'kernel module not present' issue on one specific architecture (arm) and we suspect sstate is one possibility but not confirmed yet. The info in this bug is useful. We will continue watch this issue while debugging. For others: We see similar module not present issue on a new feature 'rust-in-kernel' which is under development.
Moving off old milestone per bug cleanup (to 6.2 M2).