7441 2015-03-10 15:25:24 +0000 do_kernel_configme fails when patches create directories 2015-05-12 20:17:23 +0000 1 1 7 Build System, Metadata & Runtime OE-Core kernel 1.7.1 Other arm RESOLVED FIXED Medium enhancement 1.9 1 dev bruce.ashfield bruce.ashfield dev tom.zanussi --- --- No (bug/feature does not impact docs) oldest_to_newest 49341 0 dev 2015-03-10 15:25:24 +0000 Hardware: Zynq (via meta-xilinx) I'm using a Yocto flow to build for the Zynq with the meta-xilinx with some customizations: - I'm using a bbappend to replace the linux-xlnx 3.14 kernel with one that has had mainline changes from 3.14.17 merged in. - I added a task between patch and configure for applying the Xenomai kernel patches and running the Xenomai prepare_kernel script. All layers are on the dizzy branch. The linux-xlnx recipe inherits from linux-yocto but obtains the metadeta from an out of tree source using a SRC_URI append with type=kmeta. This ends up getting copied into the kernel tree with during the kernel_configme task. This is my first time working with a kernel recipe in the linux-yocto style so I'm still coming up to speed a bit. The problem arises because the Xenomai patch process creates additional directories in the Linux tree. As part of the kernel_configme task, the kgit-meta script will attempt to use the last unversioned directory returned by 'git ls-files -o --directory' as the metadata directory, which does not work. Maybe it is possible to update the directory detection logic in kgit-meta to look for another file within the candidate meta directory? I don't know enough about the general design to determine what should always be there. Thanks, Jeff Build output (note, I removed information about a proprietary layer): Build Configuration: BB_VERSION = "1.24.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "Ubuntu-14.04" TARGET_SYS = "arm-poky-linux-gnueabi" MACHINE = "zc706-zynq7" DISTRO = "poky" DISTRO_VERSION = "1.7.1" TUNE_FEATURES = " arm armv7a vfp neon zynq" TARGET_FPU = "vfp-neon" meta meta-yocto meta-yocto-bsp = "dizzy:6e9632e9798d78587c9317554c5547ab3b53837c" meta-oe meta-networking meta-python = "dizzy:853dcfa0d618dc26bd27b3a1b49494b98d6eee97" meta-xilinx = "dizzy:7f759048bb0aeef3c0b3938be81d2bcade7acb7e" meta-xenomai = "dizzy:8d070b7bb47e09616d07437cf0bd0ed0356b9d61" NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks ERROR: Function failed: do_kernel_configme (log file is located at /home/jmelville/yocto_dizzy/poky/build/tmp/work/zc706_zynq7-poky-linux-gnueabi/linux-xlnx/3.14.17-xilinx+gitAUTOINC+1bcd414b16_4f349cf055-r0/temp/log.do_kernel_configme.17708) ERROR: Logfile of failure stored in: /home/jmelville/yocto_dizzy/poky/build/tmp/work/zc706_zynq7-poky-linux-gnueabi/linux-xlnx/3.14.17-xilinx+gitAUTOINC+1bcd414b16_4f349cf055-r0/temp/log.do_kernel_configme.17708 Log data follows: | DEBUG: Executing shell function do_kernel_configme | NOTE: kernel configme | [INFO] Configuring target/machine combo: "standard/zynq" | cp: cannot create regular file 'kernelxenomai//meta-series': No such file or directory | [ERROR] cant copy ./.meta/meta-series to kernelxenomai//meta-series | Error running the meta series for collecting config data | config of "standard/zynq" failed | WARNING: exit code 1 from a shell command. | ERROR: Function failed: do_kernel_configme (log file is located at /home/jmelville/yocto_dizzy/poky/build/tmp/work/zc706_zynq7-poky-linux-gnueabi/linux-xlnx/3.14.17-xilinx+gitAUTOINC+1bcd414b16_4f349cf055-r0/temp/log.do_kernel_configme.17708) ERROR: Task 8 (/home/jmelville/yocto_dizzy/meta-xilinx/recipes-kernel/linux/linux-xlnx_3.14.bb, do_kernel_configme) failed with exit code '1' NOTE: Tasks Summary: Attempted 403 tasks of which 401 didn't need to be rerun and 1 failed. No currently running tasks (403 of 420) Summary: 1 task failed: /home/jmelville/yocto_dizzy/meta-xilinx/recipes-kernel/linux/linux-xlnx_3.14.bb, do_kernel_configme Summary: There was 1 ERROR message shown, returning a non-zero exit code. Output from git ls-files -o --directory in the Linux kernel dir: .meta/ arch/arm/boot/dts/omap3-igep0020-3430.dts arch/arm/boot/dts/omap3430-igep.dtsi arch/arm/include/asm/fcse.h arch/arm/include/asm/hardware/cache-l2x0.h.orig arch/arm/include/asm/ipipe.h arch/arm/include/asm/ipipe_base.h arch/arm/include/asm/ipipe_hwirq.h arch/arm/include/asm/resource.h arch/arm/include/asm/xenomai/ arch/arm/kernel/ipipe.c arch/arm/kernel/ipipe_tsc.c arch/arm/kernel/ipipe_tsc_asm.S arch/arm/mach-at91/at91_ipipe.c arch/arm/mach-at91/at91_ipipe.h arch/arm/mm/fcse.c arch/arm/xenomai/ drivers/xenomai/ include/asm-generic/xenomai/ include/ipipe/ include/linux/ipipe.h include/linux/ipipe_base.h include/linux/ipipe_compat.h include/linux/ipipe_debug.h include/linux/ipipe_domain.h include/linux/ipipe_lock.h include/linux/ipipe_tickdev.h include/linux/ipipe_trace.h include/xenomai/ init/Kconfig.orig kernel/ipipe/ kernel/xenomai/ 49342 1 bruce.ashfield 2015-03-10 16:08:21 +0000 For what it is worth, it is bad that *any* process leave the kernel directory dirty and not commit its changes. It means that future patching will be impossible and bisecting / debugging issues also not possible. I've had a change for making the meta directory detection more robust for a while, as part of a change unifying the detection across scripts. As you guessed, looking within the directory for a sentinel file is the easiest thing to do. I like the failure from the point of view that it is exposing some poor practices that are happening to the kernel directory .. and saving future pain, but we can still make things more robust. There's never hard in that. If you look at: git://git.yoctoproject.org/yocto-kernel-tools master-test You'll see the in progress patches. Feel free to take them for a test drive. 49344 2 dev 2015-03-10 16:21:13 +0000 Bruce, Thanks for taking a look at this so quickly. I'll consider having our patch process commit the changes. If nothing else, it will avoid shipping builds with "-dirty" kernels. I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting gets rusty from lack of use. It looks like you save the name of the metadir to the '.metadir' file. Subsequent tests use -d to check if .metadir is a directory before catting the file into a variable. If I read the code correctly, that path will never run. Cheers, Jeff 49345 3 bruce.ashfield 2015-03-10 16:27:49 +0000 (In reply to comment #2) > Bruce, > > Thanks for taking a look at this so quickly. I'll consider having our patch > process commit the changes. If nothing else, it will avoid shipping builds > with "-dirty" kernels. > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting > gets rusty from lack of use. > > It looks like you save the name of the metadir to the '.metadir' file. > Subsequent tests use -d to check if .metadir is a directory before catting > the file into a variable. If I read the code correctly, that path will never Yep. That's why this is under test :) That should be -f, not -d for the test. And I made the change here. Bruce > run. > > Cheers, > Jeff 49349 4 dev 2015-03-10 17:03:19 +0000 (In reply to comment #3) > (In reply to comment #2) > > Bruce, > > > > Thanks for taking a look at this so quickly. I'll consider having our patch > > process commit the changes. If nothing else, it will avoid shipping builds > > with "-dirty" kernels. > > > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting > > gets rusty from lack of use. > > > > It looks like you save the name of the metadir to the '.metadir' file. > > Subsequent tests use -d to check if .metadir is a directory before catting > > the file into a variable. If I read the code correctly, that path will never > > Yep. That's why this is under test :) That should be -f, not -d for the test. > And I made the change here. > > Bruce > > > run. > > > > Cheers, > > Jeff OK now the meta-xilinx layer is making this messy. They keep the Kernel metadata separately and add it to the build flow with SRC_URI and type=kmeta. As far as I can tell, those directories don't get copied into the kernel tree until 'updateme' gets run. When 'createme' runs, it can't detect the meta directory (because nothing exists yet) and meta_dir is empty going into metaize(). The script then tries to create /cfg and fails with permission errors. Jeff 49350 5 bruce.ashfield 2015-03-10 17:12:50 +0000 (In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > Bruce, > > > > > > Thanks for taking a look at this so quickly. I'll consider having our patch > > > process commit the changes. If nothing else, it will avoid shipping builds > > > with "-dirty" kernels. > > > > > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting > > > gets rusty from lack of use. > > > > > > It looks like you save the name of the metadir to the '.metadir' file. > > > Subsequent tests use -d to check if .metadir is a directory before catting > > > the file into a variable. If I read the code correctly, that path will never > > > > Yep. That's why this is under test :) That should be -f, not -d for the test. > > And I made the change here. > > > > Bruce > > > > > run. > > > > > > Cheers, > > > Jeff > > OK now the meta-xilinx layer is making this messy. They keep the Kernel > metadata separately and add it to the build flow with SRC_URI and > type=kmeta. As far as I can tell, those directories don't get copied into > the kernel tree until 'updateme' gets run. When 'createme' runs, it can't > detect the meta directory (because nothing exists yet) and meta_dir is empty > going into metaize(). The script then tries to create /cfg and fails with > permission errors. > > Jeff (In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > Bruce, > > > > > > Thanks for taking a look at this so quickly. I'll consider having our patch > > > process commit the changes. If nothing else, it will avoid shipping builds > > > with "-dirty" kernels. > > > > > > I looked over 0f1c9d2 quickly. Forgive me if I'm wrong, my bash scripting > > > gets rusty from lack of use. > > > > > > It looks like you save the name of the metadir to the '.metadir' file. > > > Subsequent tests use -d to check if .metadir is a directory before catting > > > the file into a variable. If I read the code correctly, that path will never > > > > Yep. That's why this is under test :) That should be -f, not -d for the test. > > And I made the change here. > > > > Bruce > > > > > run. > > > > > > Cheers, > > > Jeff > > OK now the meta-xilinx layer is making this messy. They keep the Kernel > metadata separately and add it to the build flow with SRC_URI and > type=kmeta. As far as I can tell, those directories don't get copied into > the kernel tree until 'updateme' gets run. When 'createme' runs, it can't > detect the meta directory (because nothing exists yet) and meta_dir is empty > going into metaize(). The script then tries to create /cfg and fails with > permission errors. Ah yes. I've run into this before as well. We won't be getting this change into 1.8, so I'll work on it more tomorrow, I'd rather not include any more complexity (I just removed a lot of it) so I need to come up with more than a one off solution. For now, the best thing to do is to make sure that some custom task runs before the kernel configme and cleans up, so everything works as expected. > > Jeff 49379 6 bruce.ashfield 2015-03-11 19:46:05 +0000 I pushed a new variant, with an exception case in the createme script. It will fallback to a default and then propagate it to all following tools. I haven't build tested this against the xilinx layers, but it does survive standard and linux-yocto-custom builds (which have no meta-data). 49458 7 dev 2015-03-13 19:01:54 +0000 (In reply to comment #6) > I pushed a new variant, with an exception case in the createme script. It > will fallback > to a default and then propagate it to all following tools. > > I haven't build tested this against the xilinx layers, but it does survive > standard and linux-yocto-custom > builds (which have no meta-data). Thanks I will test against my build flow on Monday. I have to roll back the workaround I made to work with the old scripts. 49825 8 dev 2015-03-26 20:36:21 +0000 (In reply to comment #7) > (In reply to comment #6) > > I pushed a new variant, with an exception case in the createme script. It > > will fallback > > to a default and then propagate it to all following tools. > > > > I haven't build tested this against the xilinx layers, but it does survive > > standard and linux-yocto-custom > > builds (which have no meta-data). > > Thanks I will test against my build flow on Monday. I have to roll back the > workaround I made to work with the old scripts. Sorry this took so long, I got around to testing it today and didn't see any issues. FWIW, I'm still going to stick with your original suggestion to not build a dirty kernel. Thanks! 49866 9 bruce.ashfield 2015-03-27 13:42:56 +0000 Great. Since I won't push this into 1.8 at this late point, I'll move this to 1.9 and send it for merging once the tree thaws.