Determining a position of a pointing device
First Claim
1. A system for determining a location of a pointing device in three-dimensional space and a location to where the pointing device is directed, the system comprising:
- one or more infrared light emitting diodes (LEDs);
a control system coupled to the one or more infrared LEDs, the control system capable of causing the one or more infrared LEDs to emit infrared light;
one or more cameras, capable of capturing one or more frames at substantially the same time as the one or more infrared LEDs are caused to emit the infrared light, at least one of the one or more frames recording an existence of an emission of the infrared light;
a processing system, capable of;
determining the location of the pointing device in three-dimensional space;
determining the location to where the pointing device is directed;
computing a similarity between an input sequence of sensor values output by the pointing device and at least one stored prototype sequence, each stored prototype sequence representing a sequence of sensor values that is generated if the pointing device is used to perform a unique gesture; and
determining if the computed similarity between the input sequence and any stored prototype sequence exceeds a prescribed similarity threshold; and
a communication interface, capable of transmitting a message across a transmission medium to a receiving unit, the message including data that can be used to determine the location of the pointing device in three-dimensional space and the location to where the pointing device is directed, and data relating to gesture determination, which the receiving unit can use to manipulate an object displayed on a display screen.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
-
Citations
24 Claims
-
1. A system for determining a location of a pointing device in three-dimensional space and a location to where the pointing device is directed, the system comprising:
-
one or more infrared light emitting diodes (LEDs); a control system coupled to the one or more infrared LEDs, the control system capable of causing the one or more infrared LEDs to emit infrared light; one or more cameras, capable of capturing one or more frames at substantially the same time as the one or more infrared LEDs are caused to emit the infrared light, at least one of the one or more frames recording an existence of an emission of the infrared light; a processing system, capable of; determining the location of the pointing device in three-dimensional space; determining the location to where the pointing device is directed; computing a similarity between an input sequence of sensor values output by the pointing device and at least one stored prototype sequence, each stored prototype sequence representing a sequence of sensor values that is generated if the pointing device is used to perform a unique gesture; and determining if the computed similarity between the input sequence and any stored prototype sequence exceeds a prescribed similarity threshold; and a communication interface, capable of transmitting a message across a transmission medium to a receiving unit, the message including data that can be used to determine the location of the pointing device in three-dimensional space and the location to where the pointing device is directed, and data relating to gesture determination, which the receiving unit can use to manipulate an object displayed on a display screen. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for determining a location of a pointing device in three-dimensional space, the method comprising:
-
emitting light from a light emitting diode (LED) in response to a control signal; recognizing the emission of the light from the LED with an imaging device having a lens; capturing a frame with the imaging device, wherein the frame includes a representation of the light emitted from the LED; determining a location of the representation in the frame; determining the location of the pointing device in three-dimensional space based upon the location of the representation in the frame; computing a similarity between an input sequence of sensor values output by the pointing device and at least one stored prototype sequence, each stored prototype sequence representing a sequence of sensor values that is generated if the pointing device is used to perform a unique gesture; and determining if the computed similarity between the input sequence and any stored prototype sequence exceeds a prescribed similarity threshold. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. A system for determining a location of a pointing device in three-dimensional space, comprising:
-
a light emitting diode (LED) in a first position, the LED capable of emitting infrared light; a controller coupled to the LED capable of causing the LED to emit the infrared light for a pre-defined time period; a camera capable of detecting the emitted infrared light; and a processor capable of; determining a first position of the emitted infrared light, wherein the first position corresponds with a first area of high intensity in an image, wherein the location of the pointing device in three-dimensional space is based at least in part on the first position; determining an orientation of the pointing device; computing a similarity between an input sequence of sensor values output by the pointing device and at least one stored prototype sequence, each stored prototype sequence representing a sequence of sensor values that is generated if the pointing device is used to perform a unique gesture; and determining if the computed similarity between the input sequence and any stored prototype sequence exceeds a prescribed similarity threshold. - View Dependent Claims (19, 20, 21, 22, 23, 24)
-
Specification