Object detection and tracking with audio and optical signals
First Claim
1. A method of operating an optical motion-capture system in low-light situations, the method including:
- monitoring at least two contact microphones coupled to the optical motion-capture system, wherein the contact microphones are in contact with a surface; and
in response to detection of audio signals generated by contact of an object with the surface determining a surface location by;
measuring a time difference of arrival (TDOA) of the audio signals at the contact microphones and mapping a curve, including a plurality of possible surface locations of the contact, using the measured time difference of arrival;
analyzing at least one image, captured by a camera of the optical motion-capture system, of the object in contact with the surface; and
using the image analysis, selecting one of the plurality of possible surface locations included in the mapped curve, as the surface location of the contact;
repeatedly measuring time difference of arrival of the audio signals at the contact microphones responsive to successive contacts of the object with the surface; and
analyzing a sequence of images of the successive contacts of the object with the surface;
thereby calculating a path of the object traveling across and in contact with the surface.
11 Assignments
0 Petitions
Accused Products
Abstract
The technology disclosed addresses these problems by supplementing optical gesture recognition with the ability to recognize touch gestures. This capability allows the user to execute intuitive gestures involving contact with a surface. For example, in low-light situations where free-form gestures cannot be recognized optically with a sufficient degree of reliability, a device implementing the invention may switch to a touch mode in which touch gestures are recognized. In implementations, two contact microphones or other vibratory or acoustical sensors, that are coupled to an optical motion-capture and are in contact with a surface that a user touches, are monitored. When the contact microphones detect audio signals (or other vibrational phenomena) generated by contact of an object with the surface that the user touches, a position of the object traveling across and in contact with the surface is tracked.
-
Citations
16 Claims
-
1. A method of operating an optical motion-capture system in low-light situations, the method including:
-
monitoring at least two contact microphones coupled to the optical motion-capture system, wherein the contact microphones are in contact with a surface; and in response to detection of audio signals generated by contact of an object with the surface determining a surface location by; measuring a time difference of arrival (TDOA) of the audio signals at the contact microphones and mapping a curve, including a plurality of possible surface locations of the contact, using the measured time difference of arrival; analyzing at least one image, captured by a camera of the optical motion-capture system, of the object in contact with the surface; and using the image analysis, selecting one of the plurality of possible surface locations included in the mapped curve, as the surface location of the contact; repeatedly measuring time difference of arrival of the audio signals at the contact microphones responsive to successive contacts of the object with the surface; and analyzing a sequence of images of the successive contacts of the object with the surface;
thereby calculating a path of the object traveling across and in contact with the surface. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method of tracking a path of an object of interest traveling across and in contact with a surface, the method comprising:
-
capturing, with a camera, a sequence of images of a field of view that includes at least a portion of the surface; capturing audio signals through the surface using at least two microphones; computationally constructing a 3D model of the object and tracking movement of the object over the surface based on the sequence of images and the audio signals, including; measuring a time difference of arrival (TDOA) of the audio signals at the microphones and mapping a curve, including a plurality of possible surface locations of the object, using the captured audio signals; analyzing at least one of the sequence of images; and using the image analysis, selecting one of the plurality of possible surface locations, included in the mapped curve, as a position of the object; and repeatedly measuring time difference of arrival of the audio signals at the microphones and performing the mapping, the analyzing and the selecting responsive to additional contacts of the object with the surface, thereby determining a path for the object traveling across the surface. - View Dependent Claims (7, 8, 9, 10, 11, 12)
-
-
13. An optical motion-capture system, including:
-
a camera for capturing images of an object within a field of view thereof; two contact microphones coupled to the camera and in contact with a surface; and a sensor for monitoring the contact microphones; a control module for tracking a position of the object, in response to detection of audio signals generated by contact of the object with the surface, by; measuring a time difference of arrival (TDOA) of the audio signals at the contact microphones and mapping a curve, including a plurality of possible surface locations of the contact, using the measured time difference of arrival; analyzing at least one image, captured by the camera of the optical motion-capture system, of the object in contact with the surface; and using the image analysis, selecting one of the plurality of possible surface locations included in the mapped curve, as the surface location of the contact; repeatedly measuring time difference of arrival of the audio signals at the contact microphones responsive to successive contacts of the object with the surface; and analyzing a sequence of images of the successive contacts of the object with the surface;
thereby calculating a path of the object traveling across and in contact with the surface. - View Dependent Claims (14, 15, 16)
-
Specification