Bug 10107 - Kernel artifacts not shown
Summary: Kernel artifacts not shown
Status: RESOLVED FIXED
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Medium normal
Target Milestone: 2.2 M4
Assignee: Ed Bartosh
QA Contact: Libertad
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-08-10 13:29 UTC by Belen Barros Pena
Modified: 2016-09-15 10:20 UTC (History)
4 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
missing-kernel-artifacts.png (125.75 KB, image/png)
2016-08-10 13:29 UTC, Belen Barros Pena
no flags Details
images-dir-content.png (134.74 KB, image/png)
2016-08-10 13:29 UTC, Belen Barros Pena
no flags Details
toaster_ui.log (564.65 KB, application/octet-stream)
2016-08-22 16:01 UTC, Belen Barros Pena
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Belen Barros Pena 2016-08-10 13:29:16 UTC
Created attachment 3344 [details]
missing-kernel-artifacts.png

Steps to reproduce:

* Create a Toaster project using the Yocto Project master release
* Add the meta-parallella layer to the project configuration. meta-xilinx should be added as well through dependencies
* Change the project machine to 'parallella'
* Build core-image-epiphany from the meta-parallella layer
* When the build completes, go to the build dashboard

Outcome: no kernel artifacts are shown (see attached missing-kernel-artifacts.png), but they do exist in the images/parallella directory (see attached images-dir-content.png).
Comment 1 Belen Barros Pena 2016-08-10 13:29:47 UTC
Created attachment 3345 [details]
images-dir-content.png
Comment 2 Joshua Lock 2016-08-10 15:21:36 UTC
I think it would be worth ensuring all of the kernel artefacts generated for machines in the yocto-bsp layer[1] are matched by the kernel artefacts pattern.

Those machines are: beaglebone, edgerouter, genericx86-64, genericx86 and mpc8315e-rdb

Rather than do builds for each of those machines you can just look at the files in the release downloads directory[2].

1. http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf/machine
2. http://downloads.yoctoproject.org/releases/yocto/yocto-2.1/machines/
Comment 3 brian avery 2016-08-10 18:19:31 UTC
Looking in http://downloads.yoctoproject.org/releases/yocto/yocto-2.1/machines/qemu


We have (to use to make regex's from) :

Machine             Kernel                             bootloader,etc                                                                 
qemuarm            zImage-qemuarm.bin                    
qemuarm64          Image-qemuarm64.bin (?)             
qemumips           vmlinux-qemumips.bin 
qemumips64         vmlinux-qemumips64.bin 
qemuppc            vmlinux-qemuppc.bin 
qemux86            bzImage-qemux86.bin
qemux86-64         bzImage-qemux86-64.bin
beaglebone:        zImage-beaglebone.bin                u-boot-beaglebone.img ,             
edgerouter         vmlinux-edgerouter.bin
imx53qsb           zImage-imx53qsb.bin                  u-boot-imx53qsb.imx 
mpc8315e-rdb       uImage-mpc8315e-rdb.bin              u-boot-mpc8315e-rdb.bin   



So, we have something like 
Modules - modules-<machine>.tgz
Images - core-image-XXX-<machine>.<type>
Device Tree: *.dtb
bootloader: u-boot*
kernel bzImage*/zImage*/vmlinux*/uImage*/Image*(?)

I’m not sure about the qemuarm64 Image* tbh.
Comment 4 Elliot Smith 2016-08-22 11:44:55 UTC
Note that we're not using regexes to find the modules* and bzImage* files produced by a build; we're using the image_license.manifest file, which has a FILES entry listing those files. The reason for using this is that other kernel image file formats/extensions might come along and break our regexes, so I was looking for a more consistent way to identify the kernel image files, ideally from some bitbake metadata.

Are the kernel image files listed above not listed in the image_license.manifest files for the corresponding builds? Should they be?

As for the original case Belen provided (parallella), are the files you expected to see listed in the image_license.manifest file?
Comment 5 Belen Barros Pena 2016-08-22 13:57:44 UTC
(In reply to comment #4)
> 
> As for the original case Belen provided (parallella), are the files you
> expected to see listed in the image_license.manifest file?

This is the content of the image_license.manifest file you get when following the steps to reproduce:

RECIPE NAME: linux-parallella
VERSION: 4.4-parallella+gitAUTOINC+3827cca516
LICENSE: GPLv2
FILES: uImage--4.4-parallella+git0+3827cca516-r0-parallella-20160822131820.bin uImage--4.4-zynq-parallella-headless+git0+3827cca516-r0-zynq-parallella-headless-20160822131820.dtb modules--4.4-parallella+git0+3827cca516-r0-parallella-20160822131820.tgz

RECIPE NAME: parallella-bitstream
VERSION: 1.0
LICENSE: MIT
FILES:


The file is in 

/home/belen/poky/build-toaster-6/tmp/deploy/licenses/core-image-epiphany-parallella-20160822131820
Comment 6 Belen Barros Pena 2016-08-22 16:01:11 UTC
Created attachment 3375 [details]
toaster_ui.log
Comment 7 Elliot Smith 2016-08-23 09:42:51 UTC
This problem is down to image_license.manifest files which contains multiple "FILES:" lines, as in Belen's example above. Because of how the code was written (assuming that the FILES line would always have a list of files, and there would only be one such line in the image_license.manifest file), the variable carrying the list of files was being reset to [] when the second "empty" FILES line was hit.

I've got a fix and I'm just testing it now.
Comment 8 Elliot Smith 2016-08-23 09:49:09 UTC
Posted patch to Toaster mailing list:
https://lists.yoctoproject.org/pipermail/toaster/2016-August/005119.html
Comment 9 Ed Bartosh 2016-09-15 10:20:12 UTC
Elliot's fix has been merged into poky master.

Unfortunately core-image-epifany fails to build on master due to qemu upgrade.

I built the image manually with qemu 2.6 and added debug output to the code and I was able to see kernel artifacts: ['modules--4.4-parallella+git0+eb7c3a109b-r0-parallella-20160915090103.tgz', 'uImage--4.4-zynq-parallella-headless+git0+eb7c3a109b-r0-zynq-parallella-headless-20160915090103.dtb', 'uImage--4.4-parallella+git0+eb7c3a109b-r0-parallella-20160915090103.bin', 'parallella_e16_headless_gpiose_7010.bit.bin', 'parallella_e16_headless_gpiose_7020.bit.bin']