Bug 11882

According to Saul, this works for him. We need to investigate why it fails in Refkit. Jussi, can you do that? If in doubt, build iwlwifi in Poky+meta-intel and compare.

meta-instal "works" if I build for example linux-yocto-rt 4.10 as kernel, because iwlwifi is not pulled in. But when kernel is linux-intel 4.9 it pulls in iwlwifi package as dependency and then build fails.

On poky, with "core-image-minimal" the build seems to work.

If it works in Poky but fails in refkit then that suggests that refkit is doing something, via a bbappend or distro config, to break it.

If there are no bbappends then my hunch is usrmerge interacting badly with the rm ${D}/usr in do_install.

I didn't find any bbappends or distro features that would be likely source of breaking it.

I don't know how to debug this further. I also find it strange that it doesn't fail on poky about the missing FILES_${PN} for the iwlwifi.conf from do_install_append(). I think it should fail there too. With my fix to the iwlwifi recipe to add the iwlwifi.conf to FILES_${PN} that error goes away and the package itself is built OK on both refkit and poky.

However, still the other error about missing kernel-module-cfg80211 remains on refkit, but doesn't appear on poky.

Based on what I checked, this is brought in through intel-common-drivers.scc. Manually pulling in that in the local.conf of both and get some warnings about unrelated config options that didn't end up in final config, refkit giving one warning more than poky while we shouldn't have any kernel config mods?


ERROR: refkit-image-common-1.0-r0 do_rootfs: Unable to install packages. Command '/scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/recipe-sysroot-native/usr/bin/opkg --volatile-cache -f /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/opkg.conf -t /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/temp/ipktemp/ -o /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   install connman-tools packagegroup-core-boot alsa-state lms8 rmc weston-init linux-firmware packagegroup-core-ssh-openssh pciutils kernel-modules run-postinsts packagegroup-common-test connman-client weston alsa-utils packagegroup-core-connectivity' returned 1:
Collected errors:
 * Solver encountered 1 problem(s):
 * Problem 1/1:
 *   - nothing provides kernel-module-cfg80211-4.9.30-yocto-standard needed by kernel-module-ath9k-common-4.9.30-yocto-standard-4.9.30+git0+299f12a06c_22a3e39758-r0.corei7-64-intel-common
 *
 * Solution 1:
 *   - do not ask to install a package providing kernel-modules

 * opkg_finalize_intercepts: Failed to open dir /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/temp/ipktemp//opkg-mSKt9Y/opkg-intercept-Bf14Wq: No such file or directory.
 * rm_r: Failed to open dir /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/temp/ipktemp//opkg-mSKt9Y/opkg-intercept-Bf14Wq: No such file or directory.
 * rm_r: Failed to open dir /scratch/jlaako/build/tmp-glibc/work/intel_corei7_64-refkit-linux/refkit-image-common/1.0-r0/temp/ipktemp//opkg-mSKt9Y: No such file or directory.

FWIW I tried building iwlwifi to look at this and can't even get past compile.

(In reply to comment #6)
> FWIW I tried building iwlwifi to look at this and can't even get past
> compile.

Were you using MACHINE=intel-corei7-64 and linux-intel?

The easiest way to reproduce this should be by checking out the intel-iot-refkit branch that I had mentioned in the description.

(In reply to comment #5)
> I don't know how to debug this further. I also find it strange that it
> doesn't fail on poky about the missing FILES_${PN} for the iwlwifi.conf from
> do_install_append(). I think it should fail there too.

The recipe uses modules.bbclass, which does some automatic packaging of the installed files. I find it likely that this packaging fails due to usrmerge in refkit, due to a /lib/modules vs. /usr/lib/modules mismatch somewhere.

Someone needs to check whether the class or do_install do it wrong.

So I can absolutely reproduce this in Poky/OE-Core with just enabling the usrmerge DISTRO_FEATURE, so there is something going on that I don't believe is correct, since it should not change that packaging behavior.

I also seems to loose the modules-load.d directory which MODULE_AUTOLOAD sets up, it's in the image, but does not get packaged either.  I see them getting installed into the image directory, but not packaged.

More digging is required.

Further refined the issue to the bitbake.conf root_prefix changing to base_prefix ("") from exec_prefix ("/usr") in usrmerge mode, so next digging is to what that breaks in the kernel-module class, I think that's the issue now.

Found the issue in iwlwifi is removing the /usr which is where usrmerge puts everything, so that was wrong.

http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/commit/?id=1a35dee120dde6c90af342974b528c5db1b838f9

Thank you! Now the build succeeds on RefKit too!

(In reply to comment #12)
> Thank you! Now the build succeeds on RefKit too!

I confirm it builds locally for me too but Refkit CI has this problem in do_install:

2017-08-09 10:24:17 | modinfo: ERROR: Module alias mac80211 not found.
2017-08-09 10:24:17 | depmod will prefer updates/ over kernel/ -- OK!
2017-08-09 10:24:17 | make[1]: /sbin/depmod: Command not found
2017-08-09 10:24:17 | Makefile.real:104: recipe for target 'install' failed
2017-08-09 10:24:17 | make[1]: *** [install] Error 127
2017-08-09 10:24:17 | Makefile:40: recipe for target 'install' failed
2017-08-09 10:24:17 | make: *** [install] Error 2
2017-08-09 10:24:17 | ERROR: oe_runmake failed

Ok, so I agree, my local build of refkit with all the images mentioned in the CI build set:

bitbake refkit-image-minimal refkit-image-computervision refkit-image-gateway refkit-image-industrial refkit-installer-image swtpm-wrappers-native ovmf 

and it succeeded for all of these, I tried to run the iot_test_export task, but something is missing in some setup, and it's not clear to me what.

I reviewed the log file from https://intel-iot-refkit.01.org/job/intel-iot-refkit_pull-requests/272/console

I can see the do_install() calls the make install and there are 6 modules installed, including both the mac80211 and cfg80211 modules, but some how it fails to find the mac80211 module and possibly depmod itself, altough I wonder if that is an artifact of not finding the kernel module.

Not sure what the CI does differently with setup in local.conf/auto.conf?  Since I see there is at least the iot_test task that does not run by default, I am missing some vital information.

(In reply to comment #15)


> 
> I can see the do_install() calls the make install and there are 6 modules
> installed, including both the mac80211 and cfg80211 modules, but some how it
> fails to find the mac80211 module and possibly depmod itself, altough I
> wonder if that is an artifact of not finding the kernel module.

I've root caused the problem and the reason is iwlwifi's make install leaks to host paths. The build works on hosts that have mac80211.ko available but fails in CI because it does not have it.

A closer look to the .sh scripts in 'install:' (Makefile.real) tells me they're relevant only if iwlwifi gets installed to your PC directly.

For our case we can probably skip them. Here's a trivial "patch" to, e.g., iwlwifi_git.bb's do_configure():

sed -i -e '/blacklist.sh/,/update-initramfs.sh/ d' ${S}/Makefile.real

(delete lines between blacklist.sh and update-initramfs.sh)

Merged Mikko's patch to the Makefile:
http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel/commit/?id=53004c75177b2b4b9eb4b8fb32f7ee9e4a9308f4