RECOGNIZING A MOVEMENT OF A POINTING DEVICE

• US 20080204411A1
• Filed: 05/07/2008
• Published: 08/28/2008
• Est. Priority Date: 02/07/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew
• Granted Patent: US 9,454,244 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/31 Voice input
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/41 Remote control of gateways
• CPC Class Codes: G08C 2201/50 Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices

Abstract

The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.

Claims

1. A movement recognition apparatus comprising:

receiving means for receiving a signal via a wireless medium, the signal originating from a pointing device;

demodulation means for demodulating the received signal;

control means for receiving the demodulated signal and providing a plurality of output data messages, the plurality of output data messages including at least one acceleration data value detected by an accelerometer during a motion of the pointing device;

storage means for storing the plurality of output data messages as well as a plurality of threshold definitions associated with accelerometer output of the pointing device, wherein each of the plurality of threshold definitions includes at least one threshold value utilized to indicate that the motion of the pointing device has occurred; and

processing means for processing the plurality of output data messages, wherein the processing includes accessing each of the plurality of threshold definitions, comparing each of the threshold definitions to the plurality of output data messages, identifying whether the at least one threshold value in each of the threshold definitions is exceeded by any of the plurality of output data messages, and selecting one of the threshold definitions, wherein the at least one threshold value in the selected one of the threshold definitions is exceeded by any of the plurality of output data messages.

2. One or more computer-readable storage media in a computing device having computer-useable instructions embodied thereon for performing a method for recognizing a movement associated with a pointing device, the method comprising:

referencing a threshold associated with the pointing device;

receiving sensor data at the computing device, the sensor data being output from the pointing device, the sensor data representing a motion of the pointing device;

identifying whether the threshold is exceeded by the sensor data; and

determining that the movement has occurred, if the threshold value is exceeded by the sensor data.

3. The one or more computer-readable storage media of claim 2 further comprising initiating a control action by the computing device if it is determined that the movement has occurred.

4. The one or more computer-readable storage media of claim 3, wherein initiating a control action comprises manipulating an object.

5. The one or more computer-readable storage media of claim 4 further comprising identifying the object for manipulation in accordance with the control action.

6. The one or more computer-readable storage media of claim 2 further comprising recording the sensor data received from the pointing device.

7. The one or more computer-readable storage media of claim 6 further comprising receiving an indication of an activation of a button on the pointing device before recording the sensor data.

8. The one or more computer-readable storage media of claim 6 further comprising:

receiving an indication of an activation of a button on the pointing device; and

receiving an indication of a deactivation of a button on the pointing device,

wherein recording the sensor data occurs between the activation of the button and the deactivation of the button.

9. The one or more computer-readable storage media of claim 2, wherein the movement includes an upward, a downward, a rightward, and/or a leftward motion with respect to the object.

10. The one or more computer-readable storage media of claim 2, wherein the sensor data includes acceleration data.

11. The one or more computer-readable storage media of claim 2 further comprising obtaining a plurality of thresholds, wherein each of the thresholds comprises at least one value.

12. The one or more computer-readable media of claim 11, wherein identifying whether the threshold is exceeded by the sensor data further comprises determining whether the at least one value in any of the plurality of thresholds is exceeded by the sensor data.

13. A method for recognizing a movement associated with a pointing device, the method comprising:

referencing at least one threshold value utilized to indicate a motion of the pointing device;

receiving accelerometer data at the computing device, the accelerometer data being output from the pointing device, the accelerometer data including at least one acceleration data value representing an actual motion of the pointing device;

comparing the at least one threshold value to the accelerometer data received from the pointing device;

identifying whether the at least one threshold value is exceeded by the at least one acceleration data value in the accelerometer data; and

manipulating an object on a display screen based on the identification of whether the at least one data value exceeds the at least one threshold value.

14. The method of claim 13, wherein the at least one threshold value corresponds to a specific motion of the pointing device.

15. The method of claim 14, wherein the specific motion comprises a downward motion, an upward motion, a leftward motion, and/or a rightward motion about or along an axis.

16. The method of claim 15, wherein the axis comprises an x-axis, a y-axis, a z-axis, or a combination thereof.

17. The method of claim 13, further comprising recording the accelerometer data received from the pointing device.

18. The method of claim 17, further comprising receiving an indication of an activation of a button on the pointing device before recording the accelerometer data at the computing device.

19. The method of claim 18, further comprising:

receiving an indication at the computing device of an activation of a button on the pointing device; and

receiving an indication at the computing device of a deactivation of a button on the pointing device,

wherein recording the accelerometer data occurs between the activation of the button and the deactivation of the button.

20. The method of claim 15, wherein the manipulation corresponds to an upward manipulation, a downward manipulation, a rightward manipulation, and/or a leftward manipulation.

21. A movement recognition system comprising:

an antenna configured to receive a signal via a wireless medium, the signal originating from a pointing device;

a demodulator configured to demodulate the received signal;

a controller configured to receive the demodulated signal and to provide a plurality of output data messages, each of the plurality of output data messages including at least one acceleration data value generated as a result of an accelerometer detecting a motion of the pointing device;

a first storage media configured to store the plurality of output data message;

a second storage media configured to store at least one threshold definition associated with accelerometer output of the pointing device, wherein the at least one threshold definition includes at least one threshold value utilized to indicate that the motion of the pointing device has occurred; and

a processor configured to process the plurality of output data messages, wherein the processing includes accessing the at least one threshold definition, comparing the at least one threshold definition to the plurality of output data messages, identifying whether the at least one threshold value in the at least one threshold definition is exceeded by any of the at least one acceleration data value.

22. The system of claim 21, wherein the first storage media or the second storage media includes instructions for control actions, and wherein the processor further executes the instructions.

23. The system of claim 22, wherein the processor is further configured to manipulate an object in accordance with the one of the instructions for control actions.

24. The system of claim 21, wherein the controller receives the demodulated signal from the demodulator and provides a plurality of output data messages from the demodulated signal, the plurality of output data messages including an orientation data value generated in response to an accelerometer or a gyroscope detecting the motion of the pointing device.

25. The system of claim 21, wherein the controller receives the demodulated signal from the demodulator and provides a plurality of output data messages from the received signal, the plurality of output data messages including an acceleration data value detected by an accelerometer during a motion of the pointing device as well as button activation data.

26. The system of claim 21, wherein the controller receives the demodulated signal from the demodulator and provides a plurality of output data messages from the received signal, the plurality of output data messages including an acceleration data value detected by an accelerometer during a motion of the pointing device as well as button activation data and button deactivation data, wherein the controller is configured to store only the plurality of output data messages that occur after the button activation data but before the button deactivation data.