MOTION RECOGNITION

• US 20120214594A1
• Filed: 02/18/2011
• Published: 08/23/2012
• Est. Priority Date: 02/18/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kirovski, Darko, Raptis, Michail
• Granted Patent: US 8,761,437 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/46 Computing the game score
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Human body motion is represented by a skeletal model derived from image data of a user. Skeletal model data may be used to perform motion recognition and/or similarity analysis of body motion. An example method of motion recognition includes receiving skeletal motion data representative of a user data motion feature from a capture device relating to a position of a user within a scene. A cross-correlation of the received skeletal motion data relative to a plurality of prototype motion features from a prototype motion feature database is determined. Likelihoods that the skeletal motion data corresponds to each of the plurality of prototype motion features are ranked. The likelihoods are determined using the cross-correlation. A classifying operation is performed on a subset of the plurality of prototype motion features. The subset of the plurality of prototype motion features is chosen because its members have the relatively highest likelihoods of corresponding to the skeletal motion data.

Claims

1. A method of motion recognition and/or similarity analysis of body motion based on skeletal model data derived from image data of a user, comprising:

receiving skeletal motion data representative of a user data motion feature from a capture device, the skeletal motion data relating to a position of a user within a scene;

determining a cross-correlation of the received skeletal motion data relative to a plurality of prototype motion features from a prototype motion feature database;

ranking likelihoods that the skeletal motion data corresponds to each of the plurality of prototype motion features, the likelihoods being based on the cross-correlation; and

performing a classifying operation on a subset of the plurality of prototype motion features, the subset of the plurality of prototype motion features having the relatively highest likelihoods of corresponding to the skeletal motion data.

2. The method of claim 1, wherein the skeletal motion data comprises angular wireframe data represented by pairs of angles for a set of first-degree joints and a set of second-degree joints.

3. The method of claim 2, wherein the pairs of angles are represented as Tait-Bryan angles.

4. The method of claim 1, wherein the skeletal motion data comprises a rotation matrix of a torso frame with respect to a coordinate frame of the capture device.

5. The method of claim 1, comprising performing a training operation to modify data representing the plurality of prototype motion features based on a plurality of training samples for each of the plurality of prototype motion features.

6. The method of claim 1, comprising performing a testing operation to assess an ability of a model to obtain the likelihoods that the skeletal motion data corresponds to the plurality of prototype motion features.

7. The method of claim 1, comprising predicting that the user data motion feature matches one of the plurality of prototype motion features.

8. The method of claim 1, comprising determining an offset for the received skeletal motion data relative to each of the plurality of prototype motion features.

9. The method of claim 1, wherein the classifying operation comprises a pairwise classification using logistic regression, linear discriminant analysis or support vector machine (SVM) analysis.

10. The method of claim 1, wherein the user data motion feature corresponds to a periodic unit.

11. The method of claim 10, wherein the periodic unit is synchronized to a beat of music.

12. The method of claim 1, wherein the cross-correlation comprises a normalized, circular cross-correlation.

13. A system for performing motion recognition and/or similarity analysis of body motion based on skeletal model data derived from image data of a user, the system comprising:

a processing unit; and

a system memory, wherein the system memory comprises code configured to direct the processing unit to;

receive skeletal motion data representative of a user data motion feature from a capture device;

determine a cross-correlation of the received skeletal motion data relative to a plurality of prototype motion features from a prototype motion feature database;

rank likelihoods that the skeletal motion data corresponds to each of the plurality of prototype motion features, the likelihoods being based on the cross-correlation; and

perform a classifying operation on a subset of the plurality of prototype motion features, the subset of the plurality of prototype motion features having the relatively highest likelihoods of corresponding to the skeletal motion data.

14. The system of claim 13, comprising code configured to direct the processing unit to perform a training operation to modify data representing the plurality of prototype motion features based on a plurality of training samples for each of the plurality of prototype motion features.

15. The system of claim 13, comprising code configured to direct the processing unit to perform a testing operation to assess an ability of a model to obtain the likelihoods that the skeletal motion data corresponds to the plurality of prototype motion features.

16. The system of claim 13, comprising code configured to direct the processing unit to predict that the user data motion feature matches one of the plurality of prototype motion features.

17. The system of claim 13, comprising code configured to direct the processing unit to determine an offset for the received skeletal motion data relative to each of the plurality of prototype motion features.

18. The system of claim 13, wherein the classifying operation comprises a pairwise classification using logistic regression, linear discriminant analysis or support vector machine (SVM) analysis.

19. The system of claim 13, wherein the user data motion feature is synchronized to a beat of music.

20. One or more computer-readable storage media, comprising code configured to direct a processing unit to:

receive skeletal motion data representative of a user data motion feature from a capture device;

determine a cross-correlation of the received skeletal motion data relative to a plurality of prototype motion features from a prototype motion feature database;

rank likelihoods that the skeletal motion data corresponds to each of the plurality of prototype motion features, the likelihoods being based on the cross-correlation; and

perform a classifying operation on a subset of the plurality of prototype motion features, the subset of the plurality of prototype motion features having the relatively highest likelihoods of corresponding to the skeletal motion data.