kernel-devsrc is a curated / minimal set of kernel artifacts and build infrastructure to support out of tree module builds, and header files (similar to the RH packaging of the development artifacts). There are still cases where the full kernel source, or headers, can be useful as a reference against the running kernel (until the in kernel headers tgz is widely available). The issue with the multiple kernel source providers is really about test cycles. The smaller devsrc is for on-target module development and builds against the exported uapi headers, and that is what the nightly / automated tests will use. We had issues with both the amount of time it took to package the entire source, and the amount of space that it took up on the images. Hence the creation of devsrc. With new kernel-source and kernel-headers packages (the working names), they are provided as references to the running kernel, and will largely be an exercise left up to the developer to use them as they want.
current status: rebasing patches to master, and performing multi-arch tests. Some symlinks, etc, need to be moved to post install scripts as they conflict and prevent some packages from being installed at the same time.
FYI: The new reference headers and source packages are following the debian/ubuntu naming strategy of: linux-headers-<version> and linux-source-<version> The only real difference between linux-headers-<version> and kernel-devsrc is that there are extra headers. So it is essential a super-set of kernel-devsrc. This is due to the fact that to actually make the headers useful, you'd want to build against them. kernel-devsrc already has minimal headers + everything we need to do this. The request for linux-headers is to grab more "what if" header files. So it is a larger package and works similarly to the debian version. The details are: The linux-source-<version> package install to /usr/src/linux-source-<version> The kernel-headers-<version> package install to /usr/src/linux-headers-<version> It is also possible that we can make a postinst convenience symlink from /usr/src/linux-source to the currently installed linux-source, and likewise for the linux-headers package.
Hello Bruce, It seems that my bugzilla entry 13361 is duplicate of 13360. I apologize for that. I did not spot this one, although I searched the entries. My bad. @13360: It is simple impossible what you have wrote here. If you use minimalistic kernel with all source code in it, it is impossible that the same header tree would be 2x bigger. I did explicit investigation about my latest host (HP Elite Book 840-G1) where I have dual-boot system (WIN10 and Fedora 29) F29 kernel, and results are speaking for themselves (obviously opposing your statements). Here is what my F29 5.0.17 kernel headers are in size (installed): [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ pwd /usr/src/kernels/5.0.17-200.fc29.x86_64 [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ du --summarize 102124 . [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ And here is what the size of the whole kernel, compressed is: https://rpmfind.net/linux/RPM/fedora/updates/29/x86_64/Packages/k/kernel-5.0.17-200.fc29.x86_64.html You can download it, unpack, and see how big it is! Here is just the basic kernel source in .xz format (I did not include patches and support shell files, after unpacking): [vuser@fedora29-ssd kernel.fedora]$ pwd /home/vuser/projects/kernel.fedora [vuser@fedora29-ssd kernel.fedora]$ ls -al linux-5.0.tar.xz -rw-rw-r--. 1 vuser vboxusers 105235152 Mar 4 15:57 linux-5.0.tar.xz [vuser@fedora29-ssd kernel.fedora]$ I unpacked it, to see/experience for myself: [vuser@fedora29-ssd linux-5.0]$ pwd /home/vuser/projects/kernel.fedora/linux-5.0 [vuser@fedora29-ssd linux-5.0]$ du --summarize 955756 . [vuser@fedora29-ssd linux-5.0]$ I hope this clears the confusion. Zoran
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13360#c3 Zoran
(In reply to comment #3) > Hello Bruce, > > It seems that my bugzilla entry 13361 is duplicate of 13360. I apologize for > that. I did not spot this one, although I searched the entries. My bad. > > @13360: > > It is simple impossible what you have wrote here. If you use minimalistic > kernel with all source code in it, it is impossible that the same header > tree would be 2x bigger. This is *not* what I am saying. The existing kernel-devsrc package is not the entire kernel source tree. It is already a minimal set of headers + build infrastructure for building out of three modules. You cannot get any smaller than it via a headers package. As such, there's nothing really to do on that front. kernel-devsrc is already the smallest. This bug will provide two new packages for those that want them: linux-headers (which is slightly larger than kernel-devsrc) and linux-source, which is a compressed capture of the entire kernel source tree (which is large, but is only intended for reference). Bruce > > I did explicit investigation about my latest host (HP Elite Book 840-G1) > where I have dual-boot system (WIN10 and Fedora 29) F29 kernel, and results > are speaking for themselves (obviously opposing your statements). > > Here is what my F29 5.0.17 kernel headers are in size (installed): > [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ pwd > /usr/src/kernels/5.0.17-200.fc29.x86_64 > [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ du --summarize > 102124 . > [user@fedora29-ssd 5.0.17-200.fc29.x86_64]$ > > And here is what the size of the whole kernel, compressed is: > https://rpmfind.net/linux/RPM/fedora/updates/29/x86_64/Packages/k/kernel-5.0. > 17-200.fc29.x86_64.html > > You can download it, unpack, and see how big it is! > Here is just the basic kernel source in .xz format (I did not include > patches and support shell files, after unpacking): > [vuser@fedora29-ssd kernel.fedora]$ pwd > /home/vuser/projects/kernel.fedora > [vuser@fedora29-ssd kernel.fedora]$ ls -al linux-5.0.tar.xz > -rw-rw-r--. 1 vuser vboxusers 105235152 Mar 4 15:57 linux-5.0.tar.xz > [vuser@fedora29-ssd kernel.fedora]$ > > I unpacked it, to see/experience for myself: > [vuser@fedora29-ssd linux-5.0]$ pwd > /home/vuser/projects/kernel.fedora/linux-5.0 > [vuser@fedora29-ssd linux-5.0]$ du --summarize > 955756 . > [vuser@fedora29-ssd linux-5.0]$ > > I hope this clears the confusion. > > Zoran
Logging the latest numbers for reference: root@qemux86-64:/usr/src# ls -alF total 16 drwxr-xr-x 4 root root 4096 May 28 15:52 ./ drwxr-xr-x 10 root root 4096 May 28 13:38 ../ lrwxrwxrwx 1 root root 46 May 28 13:38 kernel -> ../../lib/modules/5.0.17-yocto-standard/source/ lrwxrwxrwx 1 root root 35 May 28 15:51 linux-headers -> linux-headers-5.0.17-yocto-standard/ drwxr-xr-x 23 root root 4096 May 28 15:49 linux-headers-5.0.17-yocto-standard/ lrwxrwxrwx 1 root root 34 May 28 15:52 linux-source -> linux-source-5.0.17-yocto-standard/ drwxr-xr-x 2 root root 4096 May 28 15:49 linux-source-5.0.17-yocto-standard/ root@qemux86-64:/usr/src# du -shL kernel 89M kernel root@qemux86-64:/usr/src# du -shL linux-headers 114M linux-headers root@qemux86-64:/usr/src# du -shL linux-source 123M linux-source root@qemux86-64:/usr/src# ls -alF linux-source/ total 125764 drwxr-xr-x 2 root root 4096 May 28 15:49 ./ drwxr-xr-x 4 root root 4096 May 28 15:52 ../ -rw-r--r-- 1 root root 128771907 May 28 13:41 linux-source-5.0.17-yocto-standard.tar.bz2 root@qemux86-64:/usr/src# rpm -qa | grep linux-headers linux-headers-5.0.17-yocto-standard-1.0-r0.qemux86_64 root@qemux86-64:/usr/src# rpm -qa | grep linux-source linux-source-5.0.17-yocto-standard-1.0-r0.qemux86_64
*** Bug 13361 has been marked as a duplicate of this bug. ***
> root@qemux86-64:/usr/src# du -shL kernel > 89M kernel > root@qemux86-64:/usr/src# du -shL linux-headers > 114M linux-headers > root@qemux86-64:/usr/src# du -shL linux-source > 123M linux-source > root@qemux86-64:/usr/src# ls -alF linux-source/ > total 125764 > drwxr-xr-x 2 root root 4096 May 28 15:49 ./ > drwxr-xr-x 4 root root 4096 May 28 15:52 ../ > -rw-r--r-- 1 root root 128771907 May 28 13:41 linux-source-5.0.17-yocto-standard.tar.bz2 Here I assume that the package linux-headers is made out of linux-source (linux-source-5.0.17-yocto-standard.tar.bz2 size 123M), NOT out of minimalistic kernel (how you called it). In this case, everything follows the right pattern. To be politically correct, you should make another package linux-headers out of your minimalistic kernel, and this (my best guess) will follow the rules, known for so long. Zoran
Implementation is done for this. The patches work, but there's an overhead to do the extra copying in devsrc. I can make the patches available, but really, this should probably be split to avoid the copying unless required. It may not make the release due to that last issue.
Do we have any plan to get this merged to Yocto? Thanks.
(In reply to comment #10) > Do we have any plan to get this merged to Yocto? Thanks. I may propose it in the next release cycle, but there are still some changes required. But it is still a bad idea. Your comment implies that you think you have a valid use case for it, so please, share that here.
Before https://git.openembedded.org/openembedded-core/commit/?id=007ef1f468110b2698a27ea9a6d43fed5a0a9fc2 We were able to build kernel from full source on target and our customers have been using that for further development since.
(In reply to comment #12) > Before > https://git.openembedded.org/openembedded-core/commit/ > ?id=007ef1f468110b2698a27ea9a6d43fed5a0a9fc2 > We were able to build kernel from full source on target and our customers > have been using that for further development since. That's a very old commit at this point, so what did, or didn't work before isn't really relevant when it wasn't an issue for the past two plus years. The intention of this package is not to support a full kernel rebuild, it is simply to make the sources available as a reference. That being said, I'll complete work on this and taint it properly such that it can't be used instead of the devsrc, but instead be a variant of devsrc and it should be acceptable.
> The intention of this package is not to support > a full kernel rebuild, it is simply to make the > sources available as a reference. Wrong reason, if you, Bruce, allow me. The main reason is to allow OOT module support/development on the target platforms, without full kernel source tree support (saving lot of mmc's target space). REFERENCES: Debian: linux-headers packages Fedora: kernel-devel packages. Zoran _______
(In reply to comment #14) > > The intention of this package is not to support > > a full kernel rebuild, it is simply to make the > > sources available as a reference. > > Wrong reason, if you, Bruce, allow me. The main reason is to allow OOT > module support/development on the target platforms, without full kernel > source tree support (saving lot of mmc's target space). > > REFERENCES: > Debian: linux-headers packages > Fedora: kernel-devel packages. > > Zoran > _______ No, I meant what I typed. kernel-devsrc already provides that capability. This variant was only added as a potential reference to the entire source tree.
> kernel-devsrc already provides that capability. So, all other distros (RHEL, Fedora, SuSE, Debian, Ubuntu...) are wrong, taking for the target OOT drivers' compilation ONLY kernel headers approach, NOT the entire kernel source. This is what you are trying to tell to/convince us, or ... ??? Z
(In reply to comment #16) > > kernel-devsrc already provides that capability. > > So, all other distros (RHEL, Fedora, SuSE, Debian, Ubuntu...) are wrong, > taking for the target OOT drivers' compilation ONLY kernel headers approach, > NOT the entire kernel source. > > This is what you are trying to tell to/convince us, or ... ??? > > Z kernel-devsrc IS the minimal subset of the kernel source for building, so Im not sure what you are trying to say above. That's not what I'm saying, and I'm aware of what is, or isn't done in other distros, and am not trying to validate or invalidate what they do, nor are we trying to copy what they do. We HAD full kernel source, it caused problems with I/O, we are using a curated kernel source, similar to what RHEL does (in fact, the basis of the package are file selections from RHEL). The points are: - The history and other practices are understood. - We already fully support out of tree module builds - I've said that I have a way to tweak the existing package to have a variant and not double the support, and I'll revisit it in the next dev cycle.
> I've said that I have a way to tweak the > existing package to have a variant and > not double the support, and I'll revisit > it in the next dev cycle. Argument accepted. But before revisiting, please, read the following important remark (Debian build for armv7 A8 on target and tested - prepared on target) Kernel >= 5.10.0: to prepare out-of-tree device driver compilation in /usr/src/$(uname -r)/ debian@arm:/usr/src/$(uname -r)# make modules_prepare root@arm:/usr/src/5.10.14-bone22# make modules_prepare CALL scripts/checksyscalls.sh CALL scripts/atomic/check-atomics.sh LDS scripts/module.lds <<======= NEW ADDITION ======= Please, do note the difference from previous kernels and 5.10.0 onward: LDS scripts/module.lds The 5.10.0 patch changing the behavior is the following: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/Makefile?id=746b25b1aa0f5736d585728ded70a8141da91edb Just mentioning, since this change might take you of the guard! ;-) Zoran
Bulk move of medium importance bugs from 5.1 to 5.2 on behalf of Bruce.
Parked in 6.99