Rolling gesture detection using a multi-dimensional pointing device

• US 8,587,519 B2
• Filed: 01/04/2011
• Issued: 11/19/2013
• Est. Priority Date: 01/07/2009
• Status: Expired due to Fees

• Current Assignee: Sensor Platforms, Incorporated (Knowles Corporation)
• Original Assignee: Sensor Platforms, Incorporated (Knowles Corporation)
• Inventors: Shaw, Kevin A., Chen, Ian, Joseph, Benjamin E.
• Primary Examiner(s): Bolotin, Dmitriy
• Application Number:
  US12/984,547
• Publication Number:
  US 20110163947A1
• Time in Patent Office: 1,050 Days
• Field of Search: 345156-167
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4221 Dedicated function buttons, e.g. for the control of an EPG, subtitles, aspect ratio, picture-in-picture or teletext
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device

Abstract

A system and a method for performing a rolling gesture using a multi-dimensional pointing device. An initiation of a gesture by a user of the multi-dimensional pointing device is detected. A rolling gesture metric corresponding to performance of a rolling gesture comprising rotation of the multi-dimensional pointing device about a longitudinal axis of the multi-dimensional pointing device is determined. Information corresponding the rolling gesture metric is conveyed to a client computer system, wherein the client computer system is configured to manipulate an object in a user interface of the client computer system in accordance with the rolling gesture metric.

Claims

1. A method for detecting performance of a rolling gesture using a multi-dimensional pointing device, comprising:

at a multi-dimensional pointing device including a first button, one or more processors and memory storing one or more programs, the one or more processors executing the one or more programs to perform the operations of;

while the first button is not activated, detecting a change in attitude of the multi-dimensional pointing device;

conveying first information corresponding to the change in attitude of the multi-dimensional pointing device to a client computer system, wherein the first information enables the client computer system to perform an operation corresponding to the change in attitude of the multi-dimensional pointing device;

after detecting the change in attitude of the multi-dimensional pointing device, detecting a button press of a first button of the multi-dimensional pointing device;

determining, in conjunction with detecting the button press, a rolling gesture metric corresponding to performance of a rolling gesture comprising rotation of the multi-dimensional pointing device about a longitudinal axis of the multi-dimensional pointing device; and

after determining the rolling gesture metric, conveying second information corresponding to the rolling gesture metric to the client computer system, wherein the second information enables the client computer system to perform an operation corresponding to the button press of the first button, wherein the operation is performed in accordance with the rolling gesture metric.

2. The method of claim 1, wherein:

the multi-dimensional pointing device includes a plurality of buttons, each respective button corresponding to a respective predefined type of operation;

the first button corresponds to a first predefined type of operation; and

the operation performed in accordance with the rolling gesture metric is an operation of the first predefined type.

3. The method of claim 2, wherein the first button is selected from the group consisting of:

a volume button corresponding to a volume change operation;

a channel button corresponding to a channel change operation;

a video input button corresponding to a video input change operation;

an audio input button corresponding to an audio input change operation; and

a gesture button.

4. The method of claim 1, wherein the rolling gesture metric corresponds to a change in attitude of the pointing device after detecting the button press of the first button.

5. The method of claim 1, wherein the rolling gesture metric is selected from the group consisting of:

a roll angle;

a roll rate; and

a roll acceleration.

6. The method of claim 1, wherein determining the corresponding rolling gesture metric comprises:

calculating a change in attitude of the multi-dimensional pointing device, corresponding to rotation about a longitudinal axis of the multi-dimensional pointing device, based on one or more accelerometer measurements from one or more multi-dimensional accelerometers of the multi-dimensional pointing device and one or more magnetic field measurements from one or more multi-dimensional magnetometers of the multi-dimensional pointing device; and

calculating the rolling gesture metric based on the change in attitude of the multi-dimensional pointing device.

7. The method of claim 1, wherein determining the corresponding rolling gesture metric comprises:

calculating a change in attitude of the multi-dimensional pointing device based on one or more accelerometer measurements from one or more multi-dimensional accelerometers of the multi-dimensional pointing device and one or more magnetic field measurements from one or more multi-dimensional magnetometers of the multi-dimensional pointing device;

determining that the multi-dimensional pointing device is undergoing a rotation about a longitudinal axis of the multi-dimensional pointing device based on the change in attitude of the multi-dimensional pointing device; and

calculating the rolling gesture metric based on the change in attitude of the multi-dimensional pointing device.

8. The method of claim 1, wherein the operation corresponding to the button press of the first button is a scrolling operation that is performed on an object in a user interface of the client computer system.

9. The method of claim 8, wherein the object is selected from the group consisting of:

a web page;

a document; and

a list.

10. The method of claim 1, wherein the operation corresponding to the button press of the first button is a rotation operation that is performed on an object in a user interface of the client computer system.

11. The method of claim 10, wherein the object is selected from the group consisting of:

a dial;

a photograph; and

a page of a document.

12. The method of claim 1, wherein the rolling gesture metric is mapped to a number of clicks of a mouse wheel over a time interval.

13. A system for detecting performance of a rolling gesture using a multi-dimensional pointing device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions to;

while a first button of the multi-dimensional pointing device is not activated, detect a change in attitude of the multi-dimensional pointing device;

convey first information corresponding to the change in attitude of the multi-dimensional pointing device to a client computer system, wherein the first information enables the client computer system to perform an operation corresponding to the change in attitude of the multi-dimensional pointing device;

after detecting the change in attitude of the multi-dimensional pointing device, detect a button press of a first button of the multi-dimensional pointing device;

determine, in conjunction with detecting the button press, a rolling gesture metric corresponding to performance of a rolling gesture comprising rotation of the multi-dimensional pointing device about a longitudinal axis of the multi-dimensional pointing device; and

after determining the rolling gesture metric, convey second information corresponding to the rolling gesture metric to the client computer system, wherein the second information enables the client computer system to perform an operation corresponding to the button press of the first button, wherein the operation is performed in accordance with the rolling gesture metric.

14. The system of claim 13, wherein:

the multi-dimensional pointing device includes a plurality of buttons, each respective button corresponding to a respective predefined type of operation;

the first button corresponds to a first predefined type of operation; and

the operation performed in accordance with the rolling gesture metric is an operation of the first predefined type.

15. The system of claim 13, wherein the rolling gesture metric corresponds to a change in attitude of the pointing device after detecting the button press of the first button.

16. The system of claim 13, wherein the instructions to determine the corresponding rolling gesture metric comprise instructions to:

calculate a change in attitude of the multi-dimensional pointing device, corresponding to rotation about a longitudinal axis of the multi-dimensional pointing device, based on one or more accelerometer measurements from one or more multi-dimensional accelerometers of the multi-dimensional pointing device and one or more magnetic field measurements from one or more multi-dimensional magnetometers of the multi-dimensional pointing device; and

calculate the rolling gesture metric based on the change in attitude of the multi-dimensional pointing device.

17. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a device, cause the device to:

while the first button of a multi-dimensional pointing device is not activated, detect a change in attitude of the multi-dimensional pointing device;

convey first information corresponding to the change in attitude of the multi-dimensional pointing device to a client computer system, wherein the first information enables the client computer system to perform an operation corresponding to the change in attitude of the multi-dimensional pointing device;

after detecting the change in attitude of the multi-dimensional pointing device, detect a button press of a first button of a multi-dimensional pointing device;

determine, in conjunction with detecting the button press, a rolling gesture metric corresponding to performance of a rolling gesture comprising rotation of the multi-dimensional pointing device about a longitudinal axis of the multi-dimensional pointing device; and

after determining the rolling gesture metric, convey second information corresponding to the rolling gesture metric to the client computer system, wherein the second information enables the client computer system to perform an operation corresponding to the button press of the first button, wherein the operation is performed in accordance with the rolling gesture metric.

18. The computer readable storage medium of claim 17, wherein:

the multi-dimensional pointing device includes a plurality of buttons, each respective button corresponding to a respective predefined type of operation;

the first button corresponds to a first predefined type of operation; and

the operation performed in accordance with the rolling gesture metric is an operation of the first predefined type.

19. The computer readable storage medium of claim 17, wherein the rolling gesture metric corresponds to a change in attitude of the pointing device after detecting the button press of the first button.

20. The computer readable storage medium of claim 17, wherein the instructions to determine the corresponding rolling gesture metric comprise instructions to:

calculate a change in attitude of the multi-dimensional pointing device, corresponding to rotation about a longitudinal axis of the multi-dimensional pointing device, based on one or more accelerometer measurements from one or more multi-dimensional accelerometers of the multi-dimensional pointing device and one or more magnetic field measurements from one or more multi-dimensional magnetometers of the multi-dimensional pointing device; and

calculate the rolling gesture metric based on the change in attitude of the multi-dimensional pointing device.

21. A multi-dimensional pointing device, comprising:

one or more processors;

two or more multi-dimensional accelerometers;

one or more multi-dimensional magnetometers;

a transmitter circuit;

a first button;

memory; and

one or more programs, wherein the one more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions to;

while a first button of the multi-dimensional pointing device is not activated, detect a change in attitude of the multi-dimensional pointing device;

convey first information corresponding to the change in attitude of the multi-dimensional pointing device to a client computer system, wherein the first information enables the client computer system to perform an operation corresponding to the change in attitude of the multi-dimensional pointing device;

after detecting the change in attitude of the multi-dimensional pointing device, detect a button press of the first button of the multi-dimensional pointing device;

determine, in conjunction with detecting the button press, a rolling gesture metric corresponding to performance of a rolling gesture comprising rotation of the multi-dimensional pointing device about a longitudinal axis of the multi-dimensional pointing device; and

after determining the rolling gesture metric, convey, using the transmitter circuit, second information corresponding to the rolling gesture metric to the client computer system, wherein the second information enables the client computer system to perform an operation corresponding to the button press of the first button, wherein the operation is performed in accordance with the rolling gesture metric.