DETERMINING A POSITION OF A POINTING DEVICE

• US 20080192007A1
• Filed: 04/18/2008
• Published: 08/14/2008
• Est. Priority Date: 02/07/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: WILSON, ANDREW
• Granted Patent: US 8,456,419 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/31 Voice input
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/41 Remote control of gateways
• CPC Class Codes: G08C 2201/50 Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices

Abstract

The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.

Claims

1. A system for determining a position of a pointing device and a location to where the pointing device is directed, the system comprising:

one or more infrared light emitting diodes (LEDs);

a control system coupled to the one or more LEDs, the control system capable of causing the one or more LEDs to emit infrared light;

one or more cameras, capable of capturing one or more frames at substantially the same time as the one or more LEDs are caused to emit the infrared light, at least one of the one or more frames recording the existence of the emission of the infrared light;

a processing system, capable of analyzing the one or more frames to determine the position of the pointing device and the location to where the pointing device is directed; and

a communication interface, capable of transmitting a message across a transmission medium to a receiving unit, the message including data that can be used to determine the position of the pointing device and the location to where the pointing device is directed, which the receiving unit can use to manipulate the object displayed on the display screen.

2. The system of claim 1, wherein a portion of the system is coupled with the pointing device.

3. The system of claim 2, wherein the portion of the system is located at a front end of the pointing device.

4. The system of claim 1, wherein the position of the pointing device comprises a three-dimensional position of the pointing device.

5. The system of claim 1, wherein the processing system determines the position of the pointing device by identifying a high intensity area in one or more of the frames.

6. The system of claim 5, wherein the high intensity area comprises an image coordinate of a pixel having a highest intensity.

7. The system of claim 1, wherein a portion of the system is coupled with the pointing device and a portion of the system is coupled with the receiving unit.

8. The system of claim 1, wherein the processing system determines the position of the pointing device by applying a stereo imaging technique.

9. The system of claim 1, wherein the processing system determines the location to where the pointing device is directed using the position of the pointing device and an orientation of the pointing device.

10. The system of claim 1, wherein the transmission medium is a wireless transmission medium.

11. A method for determining a position of a pointing device, the method comprising:

emitting light from a light emitting diode (LED) in response to a control signal;

recognizing the emission of the light from the LED with an imaging device having a lens;

capturing a frame with the imaging device, wherein the frame includes a representation of the light emitted from the LED;

determining a location of the representation in the frame; and

determining a position of the pointer based upon the location of the representation in the frame.

12. The method of claim 11, further comprising utilizing infrared filters to filter the frame.

13. The method of claim 11, wherein the step of determining a location further comprises eliminating at least a portion of background infrared light in the frame.

14. The method of claim 11, wherein the step of determining a location further comprises:

determining a high intensity area in the frame; and

determining a coordinate position of the high intensity area.

15. The method of claim 11, further comprising determining an area to which the pointer is directed based at least on the position of the pointer.

16. The method of claim 15, further comprising implementing a training procedure associated with the area to which the pointer is directed.

17. The method of claim 15, further comprising providing data to a receiving unit, the data being associated with the area to which the pointer is directed to, the data being for manipulating a screen object at the area.

18. The method of claim 11, wherein the position of the pointer comprises a position in three-dimensional space.

19. The method of claim 18, further comprising performing a stereo imaging technique.

20. A system for determining a position of a pointing device, comprising:

a light emitting diode (LED) in a first position, the LED capable of emitting infrared light;

a controller coupled to the LED capable of causing the LED to emit the infrared light for a pre-defined time period;

a camera capable of detecting the emitted infrared light; and

a processor capable of determining a first position of the emitted infrared light, wherein the first position corresponds with a first area of high intensity in the image,

wherein the location of the pointer in three-dimensional space is based at least in part on the first position.

21. The system of claim 20, wherein the camera captures the emitted infrared light contemporaneously with additional data.

22. The system of claim 20, wherein the processor is further capable of determining a display screen region to which the pointer is directed using the location of the pointer in three-dimensional space in conjunction with additional orientation related data.

23. The system of claim 20, wherein the processor utilizes a pre-defined coordinate system in conjunction with the image to establish a first coordinate position for the first area of high intensity.

24. The system of claim 23, wherein the processor utilizes triangulation calculations associated with at least the first coordinate position to determine the location of the pointer in three-dimensional space.

25. The system of claim 23, wherein the processor utilizes stereo imaging techniques associated with at least the first coordinate position to determine the location of the pointer in three-dimensional space.

26. The system of claim 20, wherein a portion of the system is connected to the pointing device.