Gesture recognizer system architecture

• US 8,782,567 B2
• Filed: 11/04/2011
• Issued: 07/15/2014
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Latta, Stephen G., Markovic, Relja, Tomlin, Arthur Charles, Snook, Gregory N.
• Primary Examiner(s): HAILU, TADESSE
• Application Number:
  US13/289,731
• Publication Number:
  US 20120050157A1
• Time in Patent Office: 984 Days
• Field of Search: 715/863, 715/784, 715/792, 463/36, 345/156, 345/157
• US Class Current: 715/863
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

Systems, methods and computer readable media are disclosed for a gesture recognizer system architecture. A recognizer engine is provided, which receives user motion data and provides that data to a plurality of filters. A filter corresponds to a gesture, that may then be tuned by an application receiving information from the gesture recognizer so that the specific parameters of the gesture—such as an arm acceleration for a throwing gesture—may be set on a per-application level, or multiple times within a single application. Each filter may output to an application using it a confidence level that the corresponding gesture occurred, as well as further details about the user motion data.

Claims

1. A method for providing recognition of gestures made by a user using a gesture filter representing a gesture, comprising:

receiving information indicative of a user motion or pose, the information being captured by a camera;

determining from the information that the information is further indicative of a change in the user'"'"'s fatigue, a change in the user'"'"'s competence, or a second user replacing the user, and in response, modifying a parameter of the gesture filter;

determining an output of the modified gesture filter based on the information; and

sending the output to an application.

2. The method of claim 1, further comprising:

receiving second information indicative of a user motion or pose, the second information being captured by the camera;

determining a second output of the gesture filter based on the second information; and

sending the application the second output.

3. The method of claim 1, further comprising:

receiving second information indicative of sound captured by a microphone; and

sending an indication of the second information to the application.

4. The method of claim 1, wherein the gesture filter has a plurality of contexts, and a parameter of each context of the base information about the gesture is unique.

5. The method of claim 1, wherein the gesture filter comprises a second parameter, and further comprising:

in response to receiving second information indicative of a user motion or pose, the information being captured by the camera, receiving from the application a value for the second parameter; and

wherein determining an output of the modified gesture filter based on the information further comprises;

determining an output of the modified gesture filter based on the value for the second parameter.

6. The method of claim 1, wherein determining the output of the modified gesture filter based on the information further comprises:

determine a confidence level that the information is indicative of the gesture being performed.

7. The method of claim 1, wherein the gesture comprises:

a crouch, a jump, a lean, an arm throw, a toss, a swing, a dodge, a kick, or a block.

8. The method of claim 1, wherein a gesture filter comprises a type of input and a type of output, and further comprising:

replacing the gesture filter with a second filter, the second filter having the same type of input and type of output as the gesture filter,

receiving second information captured by a camera;

determining a second output of the second filter based on the second information; and

sending the application the second output.

9. The method of claim 1, wherein sending the output to the application comprises:

sending the output to the application and a second application.

10. A system for providing recognition of gestures made by a user using a gesture filter representing a gesture comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed by the processor, cause the system at least to;

receive information indicative of a user motion or pose, the information being captured by a camera;

determine from the information that the information is further indicative of a change in the user'"'"'s fatigue, a change in the user'"'"'s competence, or a second user replacing the user, and in response, modifying a parameter of the gesture filter;

determine an output of the modified gesture filter based on the information; and

send the output to an application.

11. The system of claim 10, wherein the gesture filter has a plurality of contexts, and a parameter of each context of the base information about the gesture is unique.

12. The system of claim 10, wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

receive the gesture filter from the application.

13. The system of claim 10, wherein the gesture filter comprises a second parameter, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:

in response to receiving second information indicative of a user motion or pose, the information being captured by the camera, receive from the application a value for the second parameter; and

wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the modified gesture filter based on the information further cause the system to at least;

determine an output of the modified gesture filter based on the value for the parameter.

14. The system of claim 10, wherein processor-executable instructions that, when executed by the processor, cause the system at least to determine the output of the gesture filter based on the information further causes the system at least to:

determine a confidence level that the information is indicative of the gesture being performed.

15. The system of claim 10, wherein the gesture comprises:

a crouch, a jump, a lean, an arm throw, a toss, a swing, a dodge, a kick, or a block.

16. The system of claim 10, wherein a gesture filter comprises a type of input and a type of output, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:

replace the gesture filter with a second filter, the second filter having the same type of input and type of output as the gesture filter,

receive second information captured by a camera;

determine a second output of the second filter based on the second information; and

send the application the second output.

17. The system of claim 10, wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system at least to:

receive sound information; and

wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the second gesture filter based on the information further cause the system to at least;

determining the output of the second gesture filter based on the sound information.

18. The system of claim 10, wherein processor-executable instructions that, when executed by the processor, cause the system at least to send the output to the application further causes the system at least to:

send the output to the application and a second application.

19. A computer-readable storage medium for providing recognition of gestures made by a user using a gesture filter representing a gesture bearing computer-readable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving data indicative of a user motion or pose, the data being captured by a camera;

determine from the information that the information is further indicative of a change in the user'"'"'s fatigue, a change in the user'"'"'s competence, or a second user replacing the user, and in response, modifying a parameter of the gesture filter;

determine an output of the modified gesture filter based on the information; and

sending the output to an application.

20. The computer-readable storage medium of claim 19, wherein sending the output to the application comprises:

sending the output to the application and a second application.