14317 2021-03-23 03:28:37 +0000 kernel.bbclass cleans files written by make-mod-scripts recipe 2021-05-15 23:04:12 +0000 1 1 7 Build System, Metadata & Runtime OE-Core kernel unspecified All Multiple RESOLVED FIXED Medium+ normal 3.4 M1 1 kroylar+yoctobugzilla bruce.ashfield randy.macleod tom.zanussi --- --- Don't know oldest_to_newest 89690 0 kroylar+yoctobugzilla 2021-03-23 03:28:37 +0000 Any kernel module recipe will contain `inherit module`. This references meta/classes/module.bbclass which contains `inherit module-base`. This references meta/classes/module-base.bbclass which contains: # We do the dependency this way because the output is not preserved # in sstate, so we must force do_compile to run (once). do_configure[depends] += "make-mod-scripts:do_compile" The make-mod-scripts recipe (at meta/recipes-kernel/make-mod-scripts/make-mod-scripts.bb) adds files to the {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts directory. (This is referenced as STAGING_KERNEL_BUILDDIR which is set in conf/bitbake.conf.) Unfortunately, the kernel recipe will remove everything in the STAGING_KERNEL_BUILDDIR directory, since that directory is added to the `do_shared_workdir[cleandirs]` variable in meta/classes/kernel.bbclass. This ends up removing files that make-mod-scripts put there as well. 89703 1 bruce.ashfield 2021-03-23 12:25:51 +0000 (In reply to comment #0) > Any kernel module recipe will contain `inherit module`. This references > meta/classes/module.bbclass which contains `inherit module-base`. This > references meta/classes/module-base.bbclass which contains: > > # We do the dependency this way because the output is not preserved > # in sstate, so we must force do_compile to run (once). > do_configure[depends] += "make-mod-scripts:do_compile" > > The make-mod-scripts recipe (at > meta/recipes-kernel/make-mod-scripts/make-mod-scripts.bb) adds files to the > {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts directory. > (This is referenced as STAGING_KERNEL_BUILDDIR which is set in > conf/bitbake.conf.) > > Unfortunately, the kernel recipe will remove everything in the > STAGING_KERNEL_BUILDDIR directory, since that directory is added to the > `do_shared_workdir[cleandirs]` variable in meta/classes/kernel.bbclass. This > ends up removing files that make-mod-scripts put there as well. What's the actual issue though ? If you've cleaned the kernel, anything that uses what make-mod-scripts installs will trigger the dependency on the kernel and they are rebuild and reinstalled to the shared directory. If you've found a set of steps that get around this, share them here and we can have a look. 89712 2 kroylar+yoctobugzilla 2021-03-23 16:15:28 +0000 Oh sorry! I actually didn't explain that very well. Since cleaning the kernel removes files put in the STAGING_KERNEL_BUILDDIR by make-mod-scripts, anything that depends on make-mod-scripts (all external kernel modules inheriting the module bbclass) will fail to build with this error: ERROR: Kernel configuration is invalid. include/generated/autoconf.h or include/config/auto.conf are missing. Run 'make oldconfig && make prepare' on kernel src to fix it. So the situation is: 1. build an image 2. bitbake -c cleansstate <kernel> 3. bitbake -c cleansstate <external-module> 4. rebuild the image That last rebuild will correctly detect that both <kernel> and <external-module> need to be rebuilt. But it will not detect that make-mod-scripts also needs to be rebuilt. This causes the confusing error above. 89715 3 bruce.ashfield 2021-03-23 17:42:45 +0000 (In reply to comment #2) > Oh sorry! I actually didn't explain that very well. > > Since cleaning the kernel removes files put in the STAGING_KERNEL_BUILDDIR > by make-mod-scripts, anything that depends on make-mod-scripts (all external > kernel modules inheriting the module bbclass) will fail to build with this > error: > > ERROR: Kernel configuration is invalid. > include/generated/autoconf.h or include/config/auto.conf are missing. > Run 'make oldconfig && make prepare' on kernel src to fix it. > > > So the situation is: > > 1. build an image > 2. bitbake -c cleansstate <kernel> > 3. bitbake -c cleansstate <external-module> > 4. rebuild the image > > That last rebuild will correctly detect that both <kernel> and > <external-module> need to be rebuilt. But it will not detect that > make-mod-scripts also needs to be rebuilt. This causes the confusing error > above. Perfect! I'll see what's up with the dependencies. 89819 4 bruce.ashfield 2021-03-29 17:39:10 +0000 This didn't reproduce on one of my builders, trying on a second setup: I've added "hello-mod" from meta-skeleton to my test image, and have done the following: 2627 bitbake core-image-minimal 2628 bitbake -c cleansstate linux-yocto-dev 2629 bitbake -c cleansstate hello-mod 2630 bitbake core-image-minimal I'll update with my next test when available. 89821 5 kroylar+yoctobugzilla 2021-03-29 20:06:59 +0000 I may have mis-represented step 4. It should actually be: 4. bitbake <external-module> I think re-building the whole image will correctly rebuild the kernel and make-mod-scripts recipe before trying to build the external module. 89824 6 bruce.ashfield 2021-03-29 23:48:23 +0000 I switched to try this: 1708 bitbake core-image-minimal 1709 bitbake -c cleansstate linux-yocto 1710 bitbake -c cleansstate hello-mod 1711 bitbake hello-mod Still not triggering the error. Is this with master ? And which machine / kernel provider are you using ? 89913 7 kroylar+yoctobugzilla 2021-04-09 00:07:01 +0000 Sorry for the late reply. I was using the warrior branch, but I will try to reproduce with master myself now. I had assumed it would be easily reproducible because that code hasn't changed since warrior. I'll get back to you with my results. 89914 8 bruce.ashfield 2021-04-09 02:32:19 +0000 (In reply to comment #7) > Sorry for the late reply. I was using the warrior branch, but I will try to > reproduce with master myself now. I had assumed it would be easily > reproducible because that code hasn't changed since warrior. I'll get back > to you with my results. No worries. thanks for the follow up. I'll keep digging in the meantime, and see if I can see anything in the core code may have changed how the dependencies are processed between those two points. 90397 9 kroylar+yoctobugzilla 2021-05-15 21:49:49 +0000 I'm not yet able to reproduce this. I think my initial reproduction steps were incorrect and that the problem may arise with the use of devtool. But I've been experimenting with that as well to no avail. The problem is that I wasn't the one who originally ran into this, but it was my colleague, who doesn't have the history of what caused it. Perhaps we can close this ticket for now. If I ever see it happen again, I'll get solid reproducing steps before re-opening it or opening a new bug. 90398 10 kroylar+yoctobugzilla 2021-05-15 21:55:03 +0000 Wait! As I sent that I realized that I was able to reproduce it! I reproduced it in our custom environment, but I'll try again with these steps, which I think will work. bitbake core-image-minimal devtool modify linux-yocto-dev bitbake linux-yocto-dev bitbake -c cleansstate hello-mod devtool reset linux-yocto-dev bitbake hello-mod 90399 11 kroylar+yoctobugzilla 2021-05-15 23:04:12 +0000 Well, it actually seems to be fixed in master, or at least those same steps don't reproduce the problem. Sorry for the noise.