Bug 10185

Summary:

Image builds not showing as complete in the UI after finishing

Product:

[Build System, Metadata & Runtime] Toaster

Reporter:

Belen Barros Pena <belen.barros.pena>

Component:

toaster

Assignee:

Ed Bartosh <eduard.bartosh>

Status:

RESOLVED FIXED

QA Contact:

Libertad <libertad.gonzalez.de.la.cruz>

Severity:

normal

Priority:

Medium+

CC:

belen.barros.pena, eduard.bartosh, jessica.zhang, joshuagloe, stanciux.mihail

Version:

2.2

Target Milestone:

2.2 M4

Hardware:

x86

OS:

Multiple

Whiteboard:

OS type for building Yocto:

---

Type of Regression:

---

Verified:

Documentation change:

No (bug/feature does not impact docs)

Attachments:

Description	Flags
Dump of all events for successful build and stalled ui.	none
Screenshot of the stalled ui	none

Description Belen Barros Pena 2016-08-24 16:54:31 UTC

Steps to reproduce:

1. Create a Toaster project using the master release
2. Add meta-raspberrypi and its dependencies to the project
3. Change the project machine to raspberrypi2
4. Build rpi-basic-image

Outcome: the toaster_ui log will show that the build has finished, but the UI will still show it in progress. The build never goes into the completed state. 

Apparently, meta-raspberrypi uses some custom tasks to generate its images. That could be causing the issue: some event Toaster expects as indicating build completion might not be firing. 

If that's indeed the problem, Brian has found a cooker.cookerexit event that fires reliably when a build finishes, and cannot be masked by the metadata. Using that event in Toaster to identify completed builds could fix the meta-raspberrypi issue.

Comment 1 brian avery 2016-08-31 18:04:22 UTC

I'm attaching a dump of all the events we got building rpi-basic-image from the CLI in Toaster master committish 087c580b2868.  The ui for the build gets to 100% tasks then sits there.  I'm also attaching a screen shot. the images are all present in the tmp/deploy/images/raspberrypi2 directory.

Comment 2 brian avery 2016-08-31 18:07:57 UTC

Created attachment 3399 [details]
Dump of all events for successful build and stalled ui.

Comment 3 brian avery 2016-08-31 18:08:50 UTC

Created attachment 3400 [details]
Screenshot of the stalled ui

Comment 4 Ed Bartosh 2016-09-24 12:36:51 UTC

the reason of this is that buildinfohelper breaks in scan_image_artifacts function trying to call 'stat' on the file cmdline.txt that mentioned in image_licence.manifest, but doesn't exist in deploy directory.

$ cat ../build-toaster-5/tmp/deploy/licenses/rpi-basic-image-raspberrypi2-20160924101049/image_license.manifest 
RECIPE NAME: linux-raspberrypi
VERSION: 4.4.16+gitAUTOINC+cff67c7e03
LICENSE: GPLv2
FILES: Image-1-4.4.16+git0+cff67c7e03-r0-bcm2710-rpi-3-b-20160924084250.dtb Image-1-4.4.16+git0+cff67c7e03-r0-pitft28-resistive-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-pi3-miniuart-bt-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-rpi-ft5406-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-pitft22-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-w1-gpio-pullup-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-iqaudio-dacplus-20160924084250.dtbo modules-1-4.4.16+git0+cff67c7e03-r0-raspberrypi2-20160924084250.tgz Image-1-4.4.16+git0+cff67c7e03-r0-lirc-rpi-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-hifiberry-amp-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-hifiberry-dac-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-pps-gpio-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-hifiberry-digi-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-hifiberry-dacplus-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-w1-gpio-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-i2c-rtc-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-bcm2708-rpi-b-20160924084250.dtb Image-1-4.4.16+git0+cff67c7e03-r0-iqaudio-dac-20160924084250.dtbo Image-1-4.4.16+git0+cff67c7e03-r0-bcm2709-rpi-2-b-20160924084250.dtb Image-1-4.4.16+git0+cff67c7e03-r0-bcm2708-rpi-b-plus-20160924084250.dtb Image-1-4.4.16+git0+cff67c7e03-r0-raspberrypi2-20160924084250.bin Image-1-4.4.16+git0+cff67c7e03-r0-vc4-kms-v3d-20160924084250.dtbo cmdline.txt


lrwxrwxrwx 2 ed users   56 Sep 24 13:13 rpi-basic-image-raspberrypi2.jffs2 -> rpi-basic-image-raspberrypi2-20160924101049.rootfs.jffs2
-rw-r--r-- 2 ed users  29M Sep 24 13:13 rpi-basic-image-raspberrypi2-20160924101049.rootfs.jffs2
lrwxrwxrwx 2 ed users   58 Sep 24 13:13 rpi-basic-image-raspberrypi2.tar.bz2 -> rpi-basic-image-raspberrypi2-20160924101049.rootfs.tar.bz2
lrwxrwxrwx 2 ed users   55 Sep 24 13:13 rpi-basic-image-raspberrypi2.ext3 -> rpi-basic-image-raspberrypi2-20160924101049.rootfs.ext3
-rw-r--r-- 2 ed users  88M Sep 24 13:13 rpi-basic-image-raspberrypi2-20160924101049.rootfs.ext3
-rw-r--r-- 2 ed users  21M Sep 24 13:13 rpi-basic-image-raspberrypi2-20160924101049.rootfs.tar.bz2
-rw-r--r-- 2 ed users  90K Sep 24 13:13 rpi-basic-image-raspberrypi2-20160924101049.rootfs.manifest
lrwxrwxrwx 2 ed users   59 Sep 24 13:13 rpi-basic-image-raspberrypi2.manifest -> rpi-basic-image-raspberrypi2-20160924101049.rootfs.manifest
-rw-r--r-- 2 ed users  294 Sep 24 13:12 README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt


This can be fixed by checking if file exist before calling stat on it. The obvious question is why OSError exception raised by os.stat was ignored. I hope ut shouldn't be hard to find out either.

Comment 5 Ed Bartosh 2016-09-24 15:05:30 UTC

These 2 fixes are related to this bug:
 https://lists.yoctoproject.org/pipermail/toaster/2016-September/005211.html
 https://lists.yoctoproject.org/pipermail/toaster/2016-September/005212.html

First is an actual fix for this bug and second fixes ignoring of all OSError and IOError exceptions.

Please, review.

Comment 6 Ed Bartosh 2016-10-04 07:34:39 UTC

Fix has been merged to master.