Bug 11085

Summary: Computer vision demo/testing setup
Product: [Yocto Project Subprojects] IoT Reference OS Kit Reporter: Ismo Puustinen <ismo.puustinen>
Component: intel-iot-refkit-profile-computer-visionAssignee: Ismo Puustinen <ismo.puustinen>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Low CC: brian.avery, christian.da.costa, matti-pekka.sivosuo, mikko.ylinen, randy.macleod
Version: unspecified   
Target Milestone: 2.3 M4   
Hardware: x86   
OS: x86_64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Ismo Puustinen 2017-02-20 11:11:42 UTC
Integrate various computer vision technologies into one demo / test case. Initial thinking is to use OpenCV for recognizing content in the picture. Then librealsense can be used for distance measurement to the recognized content.
Comment 1 Ismo Puustinen 2017-03-31 10:48:53 UTC
A demo setup is created here: https://github.com/intel/intel-iot-refkit/pull/57

It allows, among other things, the use of RealSense cameras using Python 3 (for scripting) and brings example models for image recognition. The PR can't be merged though, because the final location to demo layers isn't defined yet.

One example demo which can be built using the PR with Python3 OpenCV and RealSense bindings is measuring the distance to cats using a RealSense camera:

   #!/usr/bin/python3
   
   import sys
   
   import numpy as np
   import cv2
   import pyrealsense as pyrs
   
   n_devices = pyrs.start()
   
   if n_devices == 0:
       print("No RealSense devices found!")
       sys.exit(1)
   
   cam = pyrs.Device()
   
   cat_cascade = cv2.CascadeClassifier("/usr/share/OpenCV/haarcascades/haarcascade_frontalcatface.xml")
   
   for x in range(30):
       # stabilize exposure
       cam.wait_for_frame()
   
   while True:
       # get image from web cam
       cam.wait_for_frame()
       img = cam.colour
   
       cats = cat_cascade.detectMultiScale(img)
   
       for (x,y,w,h) in cats:
           # find center
           cx = x+(w/2)
           cy = y+(h/2)
       
           depth = cam.depth[cy][cx]
    
           print("Cat found, distance " + str(depth/10.0) + " cm")
Comment 2 Ismo Puustinen 2017-04-24 12:05:58 UTC
Before merging we need to finalize the layer discussion (where to put demos, testing coverage, ...).
Comment 3 Mikko Ylinen 2017-04-26 06:14:50 UTC
(In reply to comment #2)
> Before merging we need to finalize the layer discussion (where to put demos,
> testing coverage, ...).

The 'where' part is what we've already agreed. We can proceed with that as the first step. Please create a PR that adds meta-refkit-extra and all the recipes in it.

We can start with that layer disabled and I'd propose that satisfies 2.3M4.
Comment 4 Ismo Puustinen 2017-04-26 06:49:59 UTC
Ok, I'll add the layer. Should we in fact add several layers for different distro configurations? For example, we could have meta-refkit-extra-graphical for demos which require X11 DISTRO_FEATURE, and so on? If we have the layers disabled we can then just select which ones we need.
Comment 5 Ismo Puustinen 2017-04-27 04:58:15 UTC
The meta-refkit-extra layer PR is now there with computer vision demo content and documentation. Changing status of this issue to "review".
Comment 6 Randy MacLeod 2017-05-03 15:51:09 UTC
Brian is going to create a new BZ as a follow-up to this one. Could you like or mention the bug id here now that I've added you too the CC list? :)