System and method for executing a process using accelerometer signals

• US 10,551,930 B2
• Filed: 07/20/2015
• Issued: 02/04/2020
• Est. Priority Date: 03/25/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Wilson, Andrew, Oliver, Nuria
• Primary Examiner(s): Deodhar, Omkar A
• Assistant Examiner(s): Hall, Shauna-Kay
• Application Number:
  US14/803,949
• Publication Number:
  US 20160116995A1
• Time in Patent Office: 1,660 Days
• Field of Search: 463 36- 38, 463 47, 345156-184
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals

Abstract

Apparatus and process for controlling a computer process with gestures and a handheld pointing device. The computer system employing the pointing device to determine what component a user wants to control and what control action is desired.

Claims

1. A method of controlling an object, the method comprising:

receiving orientation signals from one or more accelerometers based on use of a three-dimensional pointing device by a user, wherein the one or more accelerometers are contained in the three-dimensional pointing device;

normalizing the received orientation signals based in part on previously determined calibration factors, the previously determined calibration factors being associated with a profile of the user of the three-dimensional pointing device;

computing a three dimensional location of the three-dimensional pointing device based on the received orientation signals and the normalized signals;

determining an object at which the three-dimensional pointing device is being pointed; and

controlling an action of the object based on signals received from the three-dimensional pointing device and based on the profile associated with the user of the three-dimensional pointing device, the profile including at least one user command associated with a respective at least one three-dimensional movement of the three-dimensional pointing device.

2. The method according to claim 1, further comprising:

detecting movement of the three-dimensional pointing device;

recognizing a gesture associated with the detected movement of the three-dimensional pointing device three-dimensional pointing device; and

performing the action based on the recognized gesture.

3. The method according to claim 2, wherein the action comprises issuing one or more commands, wherein the issued one or more commands are based on the recognized gesture.

4. The method according to claim 2, wherein performing the action comprises issuing one or more commands controlling one or more application programs running on one or more computers.

5. The method according to claim 4, wherein controlling one or more application programs includes controlling connected devices of the one or more computers on which the one or more applications are running.

6. The method according to claim 4, wherein the recognized gesture comprises one of a set of gestures based on the one or more application programs running on the one or more computers.

7. The method according to claim 1, wherein the object is an icon displayed on a computer screen;

and

controlling the object consists of activating the icon to perform one or more functions.

8. The method according to claim 1, wherein the object is a cursor displayed on a computer screen;

and

controlling the object comprises moving the cursor on the computer screen.

9. The method according to claim 8, further comprising:

detecting activation of a button on the three-dimensional pointing device;

wherein the action controlled is further based on the location of the cursor on the computer screen and the button activated.

10. The method according to claim 1, further comprising:

detecting activation of a button on the three-dimensional pointing device,

wherein the action is initiated upon detecting activation of the button.

11. The method according to claim 1, wherein controlling the action comprises controlling one or more home appliances.

12. The method according to claim 1, wherein controlling one or more appliances is initiated by one or more of:

activating a button or switch on the three-dimensional pointing device;

pointing the three-dimensional pointing device at the one or more appliances;

performing a certain gesture with the three-dimensional pointing device; and

rolling the three-dimensional pointing device around the object.

13. The method according to claim 1, wherein controlling the action comprises controlling a video game.

14. The method according to claim 1, wherein the three dimensional location comprises a volume space;

and

wherein the method further comprises;

tracking the object as being either within the volume space or outside the volume space; and

controlling the action only if the object is within the volume space.

15. The method according to claim 14, wherein the three-dimensional pointing device is configured to control any device within the volume space.

16. The method according to claim 1, further comprising:

selecting one or more referent to be controlled by the action; and

initiating control actions on the one or more referent based on movement of the three-dimensional pointing device, wherein the particular referent being controlled is based on the movement of the three-dimensional pointing device.

17. A system comprising:

a processor and a memory communicatively coupled to the process, the memory storing thereon computer-executable instructions that, when executed by the processor, cause the system to perform operations comprising;

receiving orientation signals from one or more accelerometers based on use of a three-dimensional pointing device by a user, wherein the one or more accelerometers are contained in the three-dimensional pointing device;

normalizing the received orientation signals based in part on previously determined calibration factors, the previously determined calibration factors being associated with a profile of the user of the three-dimensional pointing device;

computing a three dimensional location of the three-dimensional pointing device based on the received orientation signals and the normalized signals;

determining an object at which the three-dimensional pointing device is being pointed; and

controlling an action of the object based on signals received from the three-dimensional pointing device and based on the profile associated with the user of the three-dimensional pointing device, the profile including at least one user command associated with a respective at least one three-dimensional movement of the three-dimensional pointing device.

18. A computer-readable storage device storing thereon computer-executable instructions that, when executed by a processor of a computing device, cause the computing device to perform operations comprising:

receiving orientation signals from one or more accelerometers based on use of a three-dimensional pointing device by a user, wherein the one or more accelerometers are contained in the three-dimensional pointing device;

normalizing the received orientation signals based in part on previously determined calibration factors, the previously determined calibration factors being associated with a profile of the user of the three-dimensional pointing device;

computing a three dimensional location of the three-dimensional pointing device based on the received orientation signals and the normalized signals;

determining an object at which the three-dimensional pointing device is being pointed; and

controlling an action of the object based on signals received from the three-dimensional pointing device and based on the profile associated with the user of the three-dimensional pointing device, the profile including at least one user command associated with a respective at least one three-dimensional movement of the three-dimensional pointing device.