Bug 10270 - Raciness when invoking multiple instances of runqemu via testimage on the autobuilder
Summary: Raciness when invoking multiple instances of runqemu via testimage on the aut...
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: Scripts and Tools (show other bugs)
Version: 2.2
Hardware: x86 Multiple
: Undecided normal
Target Milestone: 2.2 M4
Assignee: Robert Yang
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-09-13 20:07 UTC by Joshua Lock
Modified: 2016-09-15 04:11 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Lock 2016-09-13 20:07:50 UTC 
It looks like invoking -c testimage with multiple image targets at once results in a race as runqemu is invoked multiple times and tries to mkdir the qemu-tap-locks

WARNING: core-image-minimal-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201609131633_qemu
WARNING: core-image-sato-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201609131633_qemu
ERROR: core-image-minimal-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: core-image-sato-1.0-r0 do_testimage: Function failed: do_testimage
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/temp/log.do_testimage.31726
Log data follows:
| DEBUG: Executing python function do_testimage
| NOTE: rootfs file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86/core-image-sato-qemux86.ext4
| NOTE: Qemu log file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/testimage/qemu_boot_log.20160913163326
| NOTE: SSH log file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/testimage/ssh_target_log.20160913163326
| NOTE: Created listening socket for qemu serial console on: 127.0.0.1:35544
| NOTE: Created listening socket for qemu serial console on: 127.0.0.1:58297
| NOTE: Using kvm for runqemu
| NOTE: runqemu started, pid is 2629
| NOTE: waiting at most 60 seconds for qemu pid
| NOTE: runqemu exited with code 1
| NOTE: Output from runqemu:
| runqemu - INFO - MACHINE: qemux86
| runqemu - INFO - DEPLOY_DIR_IMAGE: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86
| runqemu - INFO - CONFFILE: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86/core-image-sato-qemux86.qemuboot.conf
| runqemu - INFO - Continuing with the following parameters:
| 
| stty: 'standard input': Inappropriate ioctl for device
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 920, in <module>
|     ret = main()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 911, in main
|     config.setup_network()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 760, in setup_network
|     self.setup_tap()
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 697, in setup_tap
|     os.mkdir(lockdir)
| FileExistsError: [Errno 17] File exists: '/tmp/qemu-tap-locks'
| KERNEL: [/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86/bzImage]
| MACHINE: [qemux86]
| FSTYPE: [ext4]
| ROOTFS: [/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86/core-image-sato-qemux86.ext4]
| CONFFILE: [/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/build/tmp/deploy/images/qemux86/core-image-sato-qemux86.qemuboot.conf]
| 
| 
| NOTE: Sending SIGTERM to runqemu
| WARNING: Qemu ended unexpectedly, dump data from host is in /tmp/oe-saved-tests/201609131633_qemu
| DEBUG: Python function do_testimage finished
| ERROR: Function failed: do_testimage
NOTE: recipe core-image-sato-1.0-r0: task do_testimage: Failed

This appears to be related to using pre-generated tapdevs

Full output at: https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/builds/900/steps/Running%20Sanity%20Tests/logs/stdio