RUNTIME ENGINE FOR ANALYZING USER MOTION IN 3D IMAGES

• US 20140267611A1
• Filed: 03/14/2013
• Published: 09/18/2014
• Est. Priority Date: 03/14/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kennett, Daniel G., Hoof, Jonathan R.
• Application Number:
  US13/804,444
• Publication Number:
  US 20140267611A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising

Abstract

Disclosed herein are systems and methods for a runtime engine for analyzing user motion in a 3D image. The runtime engine is able to use different techniques to analyze the user'"'"'s motion, depending on what the motion is. The runtime engine might choose a technique that depends on skeletal tracking data and/or one that instead uses image segmentation data to determine whether the user is performing the correct motion. The runtime engine might determine how to perform positional analysis or time/motion analysis of the user'"'"'s performance based on what motion is being performed.

Claims

1. A method, comprising:

accessing image data of a person;

inputting the image data into a runtime engine that executes on a computing device, the runtime engine has code for implementing different techniques to analyze gestures;

determining which of the techniques to use to analyze a particular gesture; and

executing code in the runtime engine to implement the determined techniques to analyze the particular gesture.

2. The method of claim 1, wherein the determining which of the techniques to use to analyze a particular gesture includes:

determining whether to use a first pose recognizer in the runtime engine that detects poses based on skeletal tracking data or a second pose recognizer in the runtime engine that detects poses based on image segmentation data.

3. The method of claim 2, wherein the determining whether to use the first pose recognizer or the second pose recognizer is based on location of the person relative to a floor.

4. The method of claim 1, wherein the determining which of the techniques to use to analyze a particular gesture includes:

determining which computations to use to perform a positional analysis of the gesture based on the particular gesture that is being analyzed by the runtime engine.

5. The method of claim 1, wherein the determining which of the techniques to use to analyze a particular gesture includes:

determining which computations to use to perform a time/motion analysis of the gesture based on the particular gesture that is being analyzed by the runtime engine.

6. The method of claim 1, wherein the determining which of the techniques to use to analyze a particular gesture includes:

determining whether to use computations that use skeletal tracking data or computations that use image segmentation data to perform an analysis of the gesture.

7. The method of claim 1, wherein the particular gesture is a physical exercise, further comprising providing feedback to the person regarding performance of the physical exercise.

8. The method of claim 1, wherein the determining which of the techniques to use to analyze a particular gesture includes:

accessing a description of the particular gesture from a database, the description having states that indicate which techniques to use to recognize the particular gesture and to analyze the particular gesture.

9. A system comprising:

a capture device that captures 3D image data that tracks a person (120);

a processor in communication with the capture device, the processor is configured to;

access the 3D image data of the person;

input the image data into a runtime engine, the runtime engine having code for analyzing gestures using plurality of different techniques;

determine which techniques of the plurality of different techniques to use to analyze a particular gesture; and

execute code in the runtime engine to implement the determined techniques to analyze the particular gesture.

10. The system of claim 9, wherein the processor being configured to determine which techniques of the plurality of different techniques to use to analyze a particular gesture includes the processor being configured to:

determine whether to use a first pose recognizer in the runtime engine that detect poses based on skeletal tracking data or a second pose recognizer in the runtime engine that detect poses based on image segmentation data.

11. The system of claim 10, wherein the processor being configured to determine whether to use the first pose recognizer or the second pose recognizer includes the processor being configured to:

select either the first pose recognizer or the second pose recognizer based on location of the person relative to a floor.

12. The system of claim 9, wherein the runtime engine performs positional analysis of gestures in accordance with a plurality of computations, the processor being configured to determine which techniques of the plurality of different techniques to use to analyze a particular gesture includes the processor being configured to:

determine which computations of the plurality of computations to use to perform a positional analysis based on the particular gesture that is being analyzed by the runtime engine.

13. The system of claim 9, wherein the runtime engine performs time/motion analysis of gestures in accordance with a plurality of computations, the processor being configured to determine which techniques of the plurality of different techniques to use to analyze a particular gesture includes the processor being configured to:

determine which computations of the plurality of computations to use to perform a time/motion analysis based on the particular gesture that is being analyzed by the runtime engine.

14. The system of claim 9, wherein the processor being configured to determine which of the techniques to use to analyze a particular gesture includes:

the processor being configured to whether to use computations that use skeletal tracking data or computations that use image segmentation data to perform an analysis of the gesture.

15. The system of claim 9, wherein the particular gesture is a particular physical exercise, and further comprising:

the processor being configured to access a description of the particular physical exercise from a database, the description having states associated with different poses that indicate which techniques to use to recognize the different poses and to analyze the particular physical exercise; and

the processor being configured to provide feedback to the person regarding performance of the physical exercise.

16. A computer readable storage medium comprising processor readable code for programming a processor to:

access 3D image data of a person performing a motion;

form skeletal tracking data from the 3D image data;

form image segmentation data from the 3D image data;

determine whether to use the skeletal tracking data or the image segmentation data to determine whether the person is performing a particular physical exercise;

determine which techniques of a runtime engine to use to analyze the person'"'"'s performance of the particular physical exercise based on the particular physical exercise; and

provide an assessment of the person'"'"'s performance of the particular physical exercise.

17. The computer readable storage medium of claim 16, wherein the processor readable code for programming the processor to determine whether to use the skeletal tracking data or the image segmentation data to determine whether the person is performing a particular physical exercise includes processor readable code for programming a processor to:

use the skeletal tracking data if the particular physical exercise is performed by person primarily standing; and

use the image segmentation data if the particular physical exercise is performed by person primarily on a floor.

18. The computer readable storage medium of claim 16, wherein the processor readable code for programming the processor to determine which techniques of a runtime engine to use to analyze the person'"'"'s performance of the particular physical exercise based on the particular physical exercise includes processor readable code for programming a processor to:

determine which computations to use to perform a positional analysis of the user'"'"'s performance of the particular physical exercise based on the particular exercise that is being analyzed by the runtime engine.

19. The computer readable storage medium of claim 16, wherein the processor readable code for programming the processor to determine which techniques of a runtime engine to use to analyze the person'"'"'s performance of the particular physical exercise based on the particular physical exercise includes processor readable code for programming a processor to:

determine which computations to use to perform a time/motion analysis of the user'"'"'s performance of the particular physical exercise based on the particular exercise that is being analyzed by the runtime engine.

20. The computer readable storage medium of claim 16, wherein the processor readable code for programming the processor to determine which techniques of a runtime engine to use to analyze the person'"'"'s performance of the particular physical exercise based on the particular physical exercise includes processor readable code for programming a processor to:

determine whether to use computations that use skeletal tracking data or computations that use image segmentation data to perform an analysis of the particular physical exercise.