GESTURE RECOGNIZER SYSTEM ARCHITECTURE

• US 20120050157A1
• Filed: 11/04/2011
• Published: 03/01/2012
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Latta, Stephen G., Markovic, Relja, Tomlin, Arthur Charles, Snook, Gregory N.
• Granted Patent: US 8,782,567 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

Systems, methods and computer readable media are disclosed for a gesture recognizer system architecture. A recognizer engine is provided, which receives user motion data and provides that data to a plurality of filters. A filter corresponds to a gesture, that may then be tuned by an application receiving information from the gesture recognizer so that the specific parameters of the gesture—such as an arm acceleration for a throwing gesture—may be set on a per-application level, or multiple times within a single application. Each filter may output to an application using it a confidence level that the corresponding gesture occurred, as well as further details about the user motion data.

Claims

1. A method for providing recognition of gestures made by a user using a gesture filter representing a gesture, comprising:

receiving information indicative of a user motion or pose, the information being captured by a camera;

determining an output of the gesture filter based on the information; and

sending the output to a first and a second application.

2. The method of claim 1, further comprising:

receiving second information indicative of a user motion or pose, the second information being captured by the camera;

determining a second output of the gesture filter based on the second information; and

sending the first and second applications the second output.

3. The method of claim 1, further comprising:

receiving second information indicative of sound captured by a microphone; and

sending an indication of the second information to the first and second applications.

4. The method of claim 1, wherein the gesture filter has a plurality of contexts, and a parameter of each context of the base information about the gesture is unique.

5. The method of claim 1, wherein the gesture filter comprises a parameter, and further comprising:

setting a value for the parameter in response to receiving data captured by the depth camera indicative of a change in the user'"'"'s fatigue, an change in the user'"'"'s competence, or a second user replacing the user; and

wherein determining an output of the gesture filter based on the information further comprises;

determining an output of the gesture filter based on the value for the parameter.

6. The method of claim 1, wherein the gesture filter comprises a parameter, and further comprising:

in response to receiving second information indicative of a user motion or pose, the information being captured by the camera, receiving from the first or second application a value for the parameter; and

wherein determining an output of the gesture filter based on the information further comprises;

determining an output of the gesture filter based on the value for the parameter.

7. The method of claim 1, wherein determining the output of the gesture filter based on the information further comprises:

determine a confidence level that the information is indicative of the gesture being performed

8. The method of claim 1, wherein the gesture comprises:

a crouch, a jump, a lean, an arm throw, a toss, a swing, a dodge, a kick, or a block.

9. The method of claim 1, wherein a gesture filter comprises a type of input and a type of output, and further comprising:

replacing the gesture filter with a second filter, the second filter having the same type of input and type of output as the gesture filter,

receiving second information captured by a camera;

determining a second output of the second filter based on the second information; and

sending the first and second applications the second output.

10. The method of claim 1, further comprising:

receiving sound information; and

wherein determining an output of the gesture filter based on the information further cause the system to at least;

determining the output of the gesture filter based on the sound information.

11. A system for providing recognition of gestures made by a user using a gesture filter representing a gesture comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed by the processor, cause the system to at least;

receive information indicative of a user motion or pose, the information being captured by a camera;

determine an output of the gesture filter based on the information; and

send the output to a first and second application.

12. The system of claim 11, wherein the gesture filter has a plurality of contexts, and a parameter of each context of the base information about the gesture is unique.

13. The system of claim 11, wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

receive the gesture filter from the first application or the second application.

14. The system of claim 11, wherein the gesture filter comprises a parameter, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

set a value for the parameter in response to receiving data captured by the depth camera indicative of a change in the user'"'"'s fatigue, an change in the user'"'"'s competence, or a second user replacing the user; and

wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the gesture filter based on the information further cause the system to at least;

determine an output of the gesture filter based on the value for the parameter.

15. The system of claim 11, wherein the gesture filter comprises a parameter, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

in response to receiving second information indicative of a user motion or pose, the information being captured by the camera, receive from the first or second application a value for the parameter; and

wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the gesture filter based on the information further cause the system to at least;

determine an output of the gesture filter based on the value for the parameter.

16. The system of claim 11, wherein processor-executable instructions that, when executed by the processor, cause the system to at least determine the output of the gesture filter based on the information further causes the system to at least:

determine a confidence level that the information is indicative of the gesture being performed

17. The system of claim 11, wherein the gesture comprises:

a crouch, a jump, a lean, an arm throw, a toss, a swing, a dodge, a kick, or a block.

18. The system of claim 11, wherein a gesture filter comprises a type of input and a type of output, and wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

replace the gesture filter with a second filter, the second filter having the same type of input and type of output as the gesture filter,

receive second information captured by a camera;

determine a second output of the second filter based on the second information; and

send the first and second applications the second output.

19. The system of claim 11, wherein the memory further bears processor-executable instructions that, when executed by the processor, cause the system to at least:

receive sound information; and

wherein the processor-executable instructions that, when executed by the system, cause the system to at least determine an output of the gesture filter based on the information further cause the system to at least;

determining the output of the gesture filter based on the sound information.

20. A computer-readable storage medium for providing recognition of gestures made by a user using a gesture filter representing a gesture bearing computer-readable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving data indicative of a user motion or pose, the data being captured by a camera;

determining an output of the gesture filter based on the data; and

sending a first application of a plurality of applications the output when the data corresponds to the first application and does not correspond to a system application of the plurality of applications.