Gesture shortcuts

• US 9,400,559 B2
• Filed: 05/29/2009
• Issued: 07/26/2016
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Latta, Stephen, Geisner, Kevin, Clavin, John, Tsunoda, Kudo, Perez, Kathryn Stone, Kipman, Alex, Markovic, Relja, Snook, Gregory N.
• Primary Examiner(s): Bhatia, Ajay
• Assistant Examiner(s): NGUYEN, PHUONG H
• Application Number:
  US12/474,781
• Publication Number:
  US 20100306714A1
• Time in Patent Office: 2,615 Days
• Field of Search: 715/863, 715/812, 715/864, 345/713, 345/719, 345/157, 345/158, 345/633, 345/173, 345/179, 348/333.02
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Systems, methods and computer readable media are disclosed for gesture shortcuts. A user'"'"'s movement or body position is captured by a capture device of a system, and is used as input to control the system. For a system-recognized gesture, there may be a full version of the gesture and a shortcut of the gesture. Where the system recognizes that either the full version of the gesture or the shortcut of the gesture has been performed, it sends an indication that the system-recognized gesture was observed to a corresponding application. Where the shortcut comprises a subset of the full version of the gesture, and both the shortcut and the full version of the gesture are recognized as the user performs the full version of the gesture, the system recognizes that only a single performance of the gesture has occurred, and indicates to the application as such.

Claims

1. A method for using a gesture shortcut in a system that takes user gestures as input to an application, comprising:

receiving data captured by a capture device, the data corresponding to a user motion or pose;

determining, based on the data, a first output identifying that the user motion or pose corresponds to the user performing a shortcut of a full gesture, a performance of the shortcut of the full gesture identifying an input to the application also identified by performance of the full gesture, the shortcut of the full gesture comprising a subset of a user motion or pose comprising the full gesture;

sending the first output to the application;

determining, based on the data, a second output identifying that the user motion or pose corresponds to the user performing the full gesture;

sending the second output to the application;

recognizing the first output and the second output as identifying the same intended gesture input to the application; and

based at least on recognizing the shortcut of the full gesture and the full gesture within a predetermined amount of time as identifying the same intended gesture input to the application, the application using only either the shortcut of the full gesture or the full gesture as the intended gesture input to the application.

2. The method of claim 1, further comprising:

receiving user input corresponding to defining a user motion or pose used to perform the shortcut of the full gesture.

3. The method of claim 1, wherein the shortcut of the full gesture corresponds to a second gesture that identifies a second input to the application, and wherein the first output identifies that the second input to the application has been provided.

4. The method of claim 1, wherein the shortcut of the full gesture corresponds to a gesture filter and a parameter, the full gesture corresponds to the gesture filter and the parameter, the parameter having a different value for the shortcut of the full gesture than for the full gesture, and wherein determining, based on the data, the first output identifying whether the user motion or pose corresponds to the user performing the shortcut of the full gesture comprises:

processing the data with the gesture filter and the parameter.

5. The method of claim 1, wherein determining, based on the data, the first output identifying whether the user motion or pose corresponds to the user performing the shortcut of a full gesture further comprises:

determining from the application to process the data to determine whether the shortcut of the full gesture has been performed before processing the data to determine the first output.

6. The method of claim 1, wherein determining, based on the data, the first output identifying whether the user motion or pose corresponds to the user performing the shortcut of the full gesture further comprises:

determining, based on the data, the first output identifying whether the user motion or pose corresponds to the user performing the shortcut of a gesture further based at least on receiving an indication from a user that gesture shortcuts are to be recognized.

7. The method of claim 1, wherein the first output is indicative of the input to the application, the application associates receiving the input to the application with a score or achievement, and wherein sending the first output to the application comprises:

sending the first output to the application, the application associating the first output with a smaller score or achievement than had the input to the application been provided through performance of the full gesture.

8. The method of claim 1, wherein the first output comprises a confidence level identifying a likelihood that the user motion or pose corresponds to the user performing the shortcut of the full gesture, and further comprising:

based at least on receiving the first output, the application recognizes that the input to the application has been provided, and based at least on receiving the second output, the application adds detail to the provided input to the application based on the second output.

9. The method of claim 8, wherein the second output indicates a distance that the user has moved a body part or a position of the body part.

10. The method of claim 8, further comprising:

based at least on determining that the user has performed the shortcut of the full gesture without then performing the full gesture, sending a third output indicative of the user performing the shortcut of the full gesture to the application, wherein the application recognizes that the input to the application has been provided, and based at least on determining that the user has not performed the full gesture, the application uses a default value for providing the input to the application.

11. A computer-readable storage device storing computer-readable instructions for using a gesture shortcut in a system that takes user gestures as input to an application, said computer-readable instructions, based at least on being executed on a processor, causing the processor to perform operations comprising:

receiving data captured by a capture device, the data corresponding to a user motion or pose;

determining, based on the data, a first output identifying that the user motion or pose corresponds to the user performing a shortcut of a full gesture, a performance of the shortcut of the full gesture identifying an input to the application also identified by performance of the full gesture, the shortcut of the full gesture comprising a subset of a user motion or pose comprising the full gesture;

sending the first output to the application;

determining, based on the data, a second output identifying that the user motion or pose corresponds to the user performing the full gesture;

sending the second output to the application;

recognizing the first output and the second output as identifying the same intended gesture input to the application; and

based at least on recognizing the shortcut of the full gesture and the full gesture within a predetermined amount of time as identifying the same intended gesture input to the application, the application using only either the shortcut of the full gesture or the full gesture as the intended gesture input to the application.

12. The computer-readable storage device of claim 11, further comprising computer-readable instructions that based at least on being executed on the processor cause the processor to perform operations comprising:

receiving user input corresponding to defining a user motion or pose used to perform the shortcut of the full gesture.

13. The computer-readable storage device of claim 11, wherein the shortcut of the full gesture corresponds to a second gesture that identifies a second input to the application, and wherein the first output identifies that the second input to the application has been provided.

14. The computer-readable storage device of claim 11, wherein the first output is indicative of providing the input to the application, the application associates providing the input to the application with a score or achievement, and wherein sending the first output to the application comprises:

sending the first output to the application, the application associating the first output with a smaller score or achievement than had the input to the application been provided through performance of the full gesture.

15. The computer-readable storage device of claim 11, wherein the first output comprises a confidence level identifying a likelihood that the user motion or pose corresponds to the user performing the shortcut of the full gesture, and further comprising computer-readable instructions that based at least on being executed on the processor cause the processor to perform the steps of:

based at least on receiving the first output, the application recognizes that the input to the application has been received, and

based at least on receiving the second output, the application adds detail to the provided input to the application based on the second output.

16. A system for using a gesture shortcut in a system that takes user gestures as input to an application, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, based at least on being executed on the processor, cause the system at least to;

receive data captured by a capture device, the data corresponding to a user motion or pose;

determine, based on the data, a first output identifying that the user motion or pose corresponds to the user performing a shortcut of a full gesture, a performance of the shortcut of the full gesture identifying an input to the application also identified by performance of the full gesture, the shortcut of the full gesture comprising a subset of a user motion or pose comprising the full gesture;

send the first output to the application;

determine, based on the data, a second output identifying that the user motion or pose corresponds to the user performing the full gesture;

send the second output to the application;

recognize the first output and the second output as identifying the same intended gesture input to the application; and

based at least on recognizing the shortcut of the full gesture and the full gesture within a predetermined amount of time as identifying the same intended gesture input to the application, use only either the shortcut of the full gesture or the full gesture as the intended gesture input to the application.

17. The system of claim 16, wherein the memory further bears processor-executable instructions that, based at least on being executed on the processor, cause the system at least to:

receive user input corresponding to defining a user motion or pose used to perform the shortcut of the full gesture.

18. The system of claim 16, wherein the shortcut of the full gesture corresponds to a second gesture that identifies a second input to the application, and wherein the first output identifies that the second input to the application has been provided.

19. The system of claim 16, wherein the first output is indicative of the input to the application, the application associates receiving the input to the application with a score or achievement, and wherein the instructions that, based at least on being executed on the processor, cause the system at least to send the first output to the application further cause the system at least to:

send the first output to the application, the application associating the first output with a smaller score or achievement than had the input to the application been provided through performance of the full gesture.

20. The system of claim 16, wherein the first output comprises a confidence level identifying a likelihood that the user motion or pose corresponds to the user performing the shortcut of the full gesture, and wherein based at least on receiving the first output, the application recognizes that the input to the application has been received, and based at least one receiving the second output, the application adds detail to the provided input to the application based on the second output.