Circle & identify

Abstract

In a smart home environment, users need a new interaction device to control the fully networked and intelligent appliances. One main requirement of spatial mouse, new interaction device, is the ability to identify objects that it is aiming at. Another is to provide an interface to control the applications or menus of the objects. Among many possible technologies that can be employed for object recognition, possibly the best solution would be object recognition by machine vision. The problem, however, is that object recognition algorithms are not yet reliable enough or light enough for hand-held devices. This thesis demonstrates that a simple object recognition algorithm can become a practical solution when augmented by interactivity. The user draws a circle around a target using a spatial mouse, and the mouse captures a series of camera frames. The frames can be easily stitched together by computing optical flow on consecutive images from camera to give a target image separated from the background, with which we need only additional steps of feature extraction and object classification. Using machine vision on the spatial mouse can provide a convenient interface like mouse of desktop computer to control the applications or menus of the objects that the users have identified. By the same optical flow method as mentioned above, the spatial mouse can control the mouse pointer on the computer display and command to the applications using click event and gestures. We performed experiments of the spatial mouse, which is Circle & Identify (C&I), to address its feasibility for object recognition and control interface by machine vision.