Manipulating An Object Utilizing A Pointing Device

• US 20080259055A1
• Filed: 04/16/2008
• Published: 10/23/2008
• Est. Priority Date: 02/07/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew
• Granted Patent: US 10,488,950 B2
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/31 Voice input
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/41 Remote control of gateways
• CPC Class Codes: G08C 2201/50 Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices

Abstract

The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.

Claims

1. A wireless, hand-held pointing device utilized to manipulate an object displayed on a display screen, the pointing device comprising:

a case extending in a longitudinal direction;

at least one button positioned on a top surface of the case;

a power supply within the case to provide power to the pointing device;

an accelerometer within the case; and

a controller within the case configured to receive power from the power supply, to sense an activation or a deactivation of the button, and to process output from the accelerometer in order to detect a motion of the pointing device, wherein the object displayed on the display screen is manipulated in accordance with the detected motion of the pointing device.

2. The pointing device of claim 1 further comprising a component on a front surface of the case, which is coupled to the controller and utilized by the controller to obtain image data.

3. The pointing device of claim 2, wherein a location of the pointing device is determined using the image data.

4. The pointing device of claim 1, wherein the pointing device wirelessly transmits one or more data messages to a receiving unit remote from the pointing device.

5. The pointing device of claim 4, wherein the one or more data messages include acceleration data detected by the accelerometer.

6. The pointing device of claim 5, wherein the pointing device wirelessly transmits the one or more data messages periodically.

7. The pointing device of claim 6, wherein the receiving unit demodulates one or more received signals.

8. The pointing device of claim 1 further comprising a gyroscope within the case to detect motion of the pointing device.

9. The pointing device of claim 1 further comprising a magnetometer within the case to detect motion of the pointing device.

10. The pointing device of claim 1 further comprising one or more indicator lights.

11. The pointing device of claim 1 further comprising one or more light-emitting diodes (LEDs).

12. The pointing device of claim 11, wherein the LEDs provide an indication of status information.

13. The pointing device of claim 1, wherein the pointing device receives one or more data messages from a remote device and decodes the data messages with the controller.

14. The pointing device of claim 1, wherein the power supply comprises one or more batteries.

15. The pointing device of claim 1, wherein the at least one operation button, when selected, provides an indication of a function.

16. The pointing device of claim 1, wherein the pointing device outputs a state of the at least one operation button.

17. The pointing device of claim 1, wherein the pointing device is associated with an identifier, which uniquely identifies the pointing device.

18. The pointing device of claim 1, wherein information or commands are entered via the pointing device.

19. The pointing device of claim 1, wherein the output of the accelerometer describes an acceleration of the pointing device.

20. The pointing device of claim 1, wherein the output of the accelerometer describes acceleration of the pointing device along a x-axis, a y-axis, a z-axis, or a combination thereof.

21. The pointing device of claim 13, wherein the one or more data messages are configured to activate a vibration unit in the case after being decoded by the controller.

22. A method for manipulating an object on a display screen with a wireless pointing device, the wireless pointing device including a case, a button on a top of the case, a power supply in the case to supply power to the wireless pointing device, at least one sensor in the case, and a controller in the case coupled to the button, the power supply, and the at least one sensor, the method comprising:

detecting a motion of the wireless pointing device by the at least one sensor;

receiving with the controller an output from the at least one sensor associated with the motion of the wireless pointing device;

generating a signal associated with the output received by the controller;

transmitting the signal; and

manipulating an object on a display screen using the signal.

23. The method of claim 22 further comprising detecting with the controller a state of the button, wherein the signal further comprises information associated with the state of the button.

24. The method of claim 22, wherein the wireless pointing device further comprises a component configured to obtain image data positioned on a front of the case, the method further comprising:

receiving image data; and

determining a location of the wireless pointing device using the image data.

25. The method of claim 22, wherein the at least one sensor comprises an accelerometer, and wherein the step of detecting further comprises detecting an acceleration of the wireless pointing device by the accelerometer.

26. The method of claim 25, wherein the output received by the controller from the accelerometer includes acceleration data.

27. The method of claim 26, wherein the signal generated by the controller includes the acceleration data.

28. The method of claim 22, wherein the step of transmitting the signal further comprises transmitting the signal periodically.

29. The method of claim 22, further comprising receiving one or more data messages from a remote device.

30. The method of claim 22, wherein the step of detecting further comprises:

detecting a motion of the wireless pointing device along an x-axis;

detecting a motion of the wireless pointing device along a y-axis; and

detecting a motion of the wireless pointing device along a z-axis.

31. An object manipulation system comprising:

a switching system connected to a device, the switching system configured to be activated and deactivated;

an acceleration detection system within the device, the acceleration system configured to determine an acceleration of the device in 3-dimensional space;

a control system coupled to the switching system and the acceleration system, the control system configured to receive data associated with the activation and the deactivation of the switching system and the acceleration of the device in 3-dimensional space and to generate signals associated with the received data; and

a communication system configured to transmit the signal, wherein the transmitted signal is capable of being used to manipulate an object displayed on a display screen.