Bug 7095

So these packages serve different purposes.

kernel-dev allows for building out-of-tree modules by including headers, sources, and generated files necessary for that task. The vmlinux and such are also useful for other debugging tasks independent from module building.

kernel-devsrc provides the complete kernel source.

As they serve different purposes, we should be able to install them to different locations.

Debian uses a similar approach with:
linux-headers-*: this package aligns with kernel-dev and installs to /usr/src/linux-headers-<VERSION>-(common|<ARCH>)

linux-sourc-*: this package is analogous to kernel-devsrc, and includes a tarball of the sources as well as a set of configurations, installed to /usr/src/linux-source* and /usr/src/linux-config*.

This split makes sense to me and seems to mirror our requirements pretty well. Any objections to resolving this issue by having the two packages install to different /usr/src/* locations?

No objection here. I can live with that. We have almost no users of the new
recipes/names, so the switch should be straight forward from that point of view.

There is still some overlap in the packages, since you can also build modules
against kernel-devsrc. 

Does our debian example have the capability of building modules against both
the headers and the source ?

If we have the capability of building modules against both .. do we run the risk of
them getting out of sync ?

The Debian example installs a tarball of the source, so specifically not intended to build modules. I don't think devsrc should either.

But one of the goals of devsrc is that you can install it, and build the entire
kernel on the target. If you can do that, you can build modules. 

We need a package that has that capability. So either it is added to the 
headers (which seems wrong), or it is part of devsrc.

I believe that we do need to split these into two locations, but there still should not be an overlap of files between them.

kernel-devsrc is the kernel source code. kernel-dev should probably become a "kernel-buildconfig" type package containing the build artifacts like the defconfig, System.map file and other key components needed for the build of modules (other than the source).

Basically, kernel-devvsrc is work-shared/MACHINE/kernel-source
and kernel-dev would have the do_shared_workdir parts of work-shared/MACHINE/kernel-build (but not the make scripts ones). We do need to figure out the "make scripts" part at some later date, if may also become part of kernel-dev.

Fixing this bug is hard right now with the single usr/src/kernel dir, once we split it, this should become much easier.

Hrm.... so this packages things as they are on the host effectively, and I can see the appeal there. The typical distro considers these two packages to be very different things and they provide partially duplicated data (ascii ven diagram, without the circles):

linux-headers    both       linux-source
=============    ====       ============
                 headers
                 config
systemmap                   all source files
generated files


linux-headers is a minimal subset of the sources, plus any necessary generated files. This fills the "build out of tree modules against the running kernel" requirement. The full source is also available for rebuilding the full kernel, patching, etc. Debian splits out headers and generated files with the source and build symlinks to common and arch dirs respectively. The linux-source package provides config and a tarball. I can see this appealing to users as they only get as much as they require with each package.

In order to improve our build time, it sounds like the proposal from Richard is to forgo the manual pruning of only the required bits, and require the use of the complete source for the build external modules use case.

I suspect we'll have users that care about each use case, and some of those that just want to build modules really won't want to have to install all the sources.

How is the "minimal subset of the sources" built? It his the make install_headers or whatever linux-libc-headers does?

Created attachment 2642 [details]
kernel.bbclass patch to populate kernel-dev

A typical out-of-tree kernel module build process is described in
http://www.cyberciti.biz/tips/build-linux-kernel-module-against-installed-kernel-source-tree.html

The build relies on the existence of  
"/lib/modules/<kernel-version>/build", containing (or symlink to) a 
necessary set of kernel development files. 

This necessary set should be provided by kernel-dev. The subset should be located at /lib/modules/<kernel-version>/build directly or at a location specified by the symlink (which at present should be something else then /usr/src/<kernel-version>):

/lib/modules/<kernel-version>/build->/usr/src/<kernel-headers>

Typical content of the subset can be seen here:
https://www.archlinux.org/packages/core/i686/linux-headers/

Users usually install these "kernel-headers" or "linux-headers" via 
# apt-get install linux-headers-$(uname -r)

The easiest way to support both kernel-dev and kernel-devsrc at the same time  is to make sure they don't reside at the same place. 
Of course this could be optimized, as the bulk of all files for kernel-dev and kernel-devsrc is identical. 

(see the attached patch for kernel.bbclass as one possible way to implement kernel-dev)

This won't make 2.0, so I'm bumping it to 2.1 (even though this is assigned to
Darren). I have some preliminary work on this, so targeting M1 seems reasonable.

The implementation of this @ Wind River, showed some compatibility issues and isn't aligned with any defacto standard (or standard) at all.

I'm going to take the implementation of this myself, and move it to 2.2

I'm now able to install both at the same time, further testing required, but the basics are in place.

Packaging and contents aside, we can now install both:

root@qemux86-64:~# opkg list |grep kernel
kernel-4.4.3-yocto-standard - 4.4.3+git0+194a6c1c4b_3d2455f9da-r0
kernel-dev - 4.4.3+git0+194a6c1c4b_3d2455f9da-r0
kernel-devsrc - 1.0-r0
kernel-module-uvesafb - 4.4.3+git0+194a6c1c4b_3d2455f9da-r0

This is a core image minimal with::

CORE_IMAGE_EXTRA_INSTALL = " kernel-dev kernel-devsrc perf perf-python"
PERF_FEATURES_ENABLE += "perf-scripting perf-tui"
IMAGE_FEATURES += "package-management"

Works fine, both kernel-dev, kernel-devsrc co-exist peacefully.
Tested with a modified build-appliance-image 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8073
where I was able to build out-of-tree-kernel module.