MACHINE LEARNING GESTURE DETECTION

• US 20130077820A1
• Filed: 09/26/2011
• Published: 03/28/2013
• Est. Priority Date: 09/26/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Marais, Charles Claudius, Mathe, Zsolt, Seminatore, Mark, Nevraev, Ivan
• Application Number:
  US13/245,640
• Publication Number:
  US 20130077820A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A virtual skeleton includes a plurality of joints and provides a machine readable representation of a human subject observed with a sensor such as a depth camera. A gesture detection module is trained via machine learning to identify one or more features of a virtual skeleton and indicate if the feature(s) collectively indicate a particular gesture.

Claims

1. A computing system, comprising:

a sensor input to receive observation information from one or more sensors;

a skeletal modeling module to derive a virtual skeleton from the observation information received from the one or more sensors; and

a gesture detection module trained via machine learning to determine if at least one or more features of the virtual skeleton collectively indicate a human subject modeled by the virtual skeleton has performed a particular gesture.

2. The computing system of claim 1, wherein the one or more sensors includes a depth camera and the observation information includes a depth image.

3. The computing system of claim 2, wherein the one or more sensors includes a visible light camera and the observation information includes a color image.

4. The computing system of claim 1, wherein the gesture detection module is trained with a supervised learning algorithm.

5. The computing system of claim 4, wherein the supervised learning algorithm is a boosting algorithm.

6. The computing system of claim 5, wherein the boosting algorithm is an Adaboost algorithm.

7. The computing system of claim 1, wherein the gesture detection module is further trained via machine learning to determine if one or more features of the observation information used to derive the virtual skeleton indicate the human subject has performed the particular gesture.

8. The computing system of claim 1, wherein the gesture detection module is further trained via machine learning to determine if one or more features of an application context indicate the human subject has performed the particular gesture.

9. The computing system of claim 1, wherein the gesture detection module is configured to output a confidence that the particular gesture has been performed by the human subject.

10. The computing system of claim 1, wherein the one or more features include a vertical body axis angle.

11. The computing system of claim 1, wherein the one or more features include a horizontal body axis angle.

12. The computing system of claim 1, wherein the one or more features include a comparison of an attribute of a first joint of the virtual skeleton and an attribute of a second joint of the virtual skeleton.

13. The computing system of claim 1, wherein the one or more features include a joint speed.

14. The computing system of claim 1, wherein the one or more features include a joint velocity.

15. The computing system of claim 1, wherein the one or more features include a joint acceleration.

16. The computing system of claim 1, wherein the one or more features include a joint force.

17. The computing system of claim 1, wherein the one or more features include a joint angle over key frames.

18. A data-holding subsystem holding instructions executable by a logic subsystem to:

receive one or more runtime instances of features of a gesture model;

analyze the one or more runtime instances of features of the gesture model with a machine learning module previously trained with training instances of one or more features of the gesture model; and

output a confidence that a human subject modeled by the gesture model has performed a particular gesture.

19. A computing system, comprising:

a depth camera input to receive depth images from a depth camera;

a skeletal modeling module to derive a virtual skeleton from the depth images received from the depth camera; and

a gesture detection module trained via machine learning to determine if at least one or more features of the virtual skeleton collectively indicate a human subject modeled by the virtual skeleton has performed a particular gesture.

20. The computing system of claim 19, wherein the gesture detection module is further trained via machine learning to determine if one or more features of the depth images indicate the human subject has performed the particular gesture.