System and method for motion capture

• US 7,457,439 B1
• Filed: 12/14/2004
• Issued: 11/25/2008
• Est. Priority Date: 12/11/2003
• Status: Active Grant

• Current Assignee: Motion Reality, Inc.
• Original Assignee: Motion Reality, Inc.
• Inventors: Wilmink, Ronnie J. H., Madsen, Nels Howard, Glover, Robert Michael, McLaughlin, Thomas M.
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/011,696
• Time in Patent Office: 1,442 Days
• Field of Search: 382/103, 382/107, 382/236, 348/14.1, 348/97, 348/154, 348/155, 348/208.4, 348/208.13, 348/208.16, 348/220.1, 348/352, 348/407.1, 348/451, 348/452, 348/620, 348/669, 348/700
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/251 involving models

Abstract

A system and method for measuring, animating, and analyzing motion in real-time is provided which operates with or without markers. System variables are used with a motion model to restrict the number of possible locations of a subject. The system variables are based on known or detected relationships between the motions of the various elements of the model. Visual comparisons and quantitative comparisons provide feedback and help a subject learn or improve a motor skill.

Claims

1. A method for capturing motion associated with a subject in real-time, comprising:

defining a plurality of elements, wherein each element corresponds to at least one interconnected segment of the model;

defining a plurality of marker positions, wherein each marker position is associated with an element;

defining a predetermined condition;

defining a value for a variable when the predetermined condition is satisfied, wherein a possible position of a first element is restricted by the value;

for each of a plurality of frames,

predicting locations for the markers, by;

determining whether the predetermined condition is satisfied; and

if the predetermined condition is satisfied, then associating the value with the variable so that the possible position of the first element is restricted; and

measuring locations of the markers; and

determining positions of the elements by comparing the predicted locations for the markers with the measured locations of the markers.

2. The method of claim 1, wherein defining a value for a variable when the predetermined condition is satisfied, comprises defining the value as a constant so that the possible position of the first element is a single position corresponding to the constant.

3. The method of claim 1, wherein defining a value for a variable when the predetermined condition is satisfied, comprises defining the value as a constant having a variance so that the possible position of the first element ranges around a position corresponding to the constant by an amount corresponding to the variance.

4. The method of claim 1, wherein defining a value for a variable when the predetermined condition is satisfied, comprises defining the value as a constant so that the possible position of the first element changes no more than an amount corresponding to the constant between frames.

5. The method of claim 1, wherein a pair of markers is identified and wherein a relationship between the pair of markers is predefined.

6. The method of claim 5, wherein the relationship is maintained throughout the motion.

7. The method of claim 5, wherein the relationship is maintained throughout only a portion of the motion.

8. The method of claim 1, wherein the predetermined condition is selected from the group consisting of:

a specific frame, an element configuration and a combination of element configurations.

9. The method of claim 1, wherein each element is associated with a parent element, a coordinate system and an origin.

10. The method of claim 1, wherein the predetermined condition is identified based on information known about the motion.

11. A method for capturing motion associated with a subject in real-time, wherein the subject includes a plurality of interconnected segments and selected ones of the segments are associated with an element, and wherein a plurality of markers are associated with selected ones of the elements, comprising:

for each of a plurality of frames;

measuring a location for each of the markers;

predicting a location for each of the markers by;

determining whether a predetermined condition is satisfied; and

if the predetermined condition is satisfied, then associating a value with a variable, wherein a relationship between the predetermined condition, the variable and the value is predefined, and wherein a possible position of a first element is restricted when the value is associated with the variable; and

determining a configuration for the model by comparing the measured locations with the predicted locations.

12. The method of claim 11, wherein the value comprises a constant so that the possible position of the first element is a single position corresponding to the constant.

13. The method of claim 11, wherein the value comprises a constant having a variance so that the possible position of the first element ranges around a position corresponding to the constant by an amount corresponding to the variance.

14. The method of claim 11, wherein the value comprises a constant so that the possible position of the first element changes no more than an amount corresponding to the constant from one frame to another frame.

15. The method of claim 11, wherein a pair of markers is identified and wherein a relationship between the pair of markers is predefined.

16. The method of claim 15, wherein the relationship is maintained throughout the motion.

17. The method of claim 15, wherein the relationship is maintained throughout only a portion of the motion.

18. The method of claim 11, wherein the predetermined condition is selected from the group consisting of:

a specific frame, an element configuration and a combination of element configurations.

19. The method of claim 11, wherein the predetermined condition is identified based on information associated with a current frame.

20. The method of claim 11, wherein the predetermined condition is identified based on information associated with a previous frame.

21. The method of claim 11, wherein the predetermined condition is identified based on information known about the motion.

22. The method of claim 11, wherein each element is associated with a parent element, a coordinate system and an origin.

23. A method for comparing motion of two subjects, comprising:

storing at least one frame representing a configuration of a first subject performing a task;

capturing motion of a second subject in real-time as the second subject performs the task by;

for each frame;

measuring a location for each of a plurality of markers associated with the second model;

predicting a location for each of the markers by determining possible positions for each marker, wherein the possible positions for a selected marker are based upon past positions of the marker; and

determining a configuration for the second subject by comparing the measured locations with the predicted locations;

comparing the configuration of the second subject to the configuration of the first subject; and

providing feedback to the second subject that quantifies any differences between the configuration of the second subject and the configuration of the first subject.

24. The method of claim 23, wherein the first subject includes a first plurality of interconnected segments and the second subject includes a second plurality of interconnected segments and wherein comparing the configuration of the first model to the configuration of the second model comprises:

calculating an overlapping volume of a first segment associated with the first model and a second segment associated with the second model.

25. The method of claim 23, wherein the first subject includes a first plurality of interconnected segments and the second subject includes a second plurality of interconnected segments and wherein comparing the configuration of the first model to the configuration of the second model comprises:

comparing a first line between a first endpoint and a second endpoint of a first segment associated with the first subject and a second line between a first endpoint and a second endpoint of a second segment associated with the second subject.

26. The method of claim 23, wherein the first subject includes a first plurality of interconnected segments and the second subject includes a second plurality of interconnected segments and wherein comparing the configuration of the first model to the configuration of the second model comprises:

comparing a subset of the first interconnected segments to a corresponding subset of the second interconnected segments.

27. The method of claim 23, wherein comparing the configuration of the first subject to the configuration of the second subject comprises:

displaying a model of the second subject overlaid on a model of the first subject.

28. The method of claim 23, wherein providing feedback to the second subject that quantifies any differences between the configuration of the second subject and the configuration of the first subject comprises providing visual feedback selected from the group consisting of a graph, a chart, a score and a difference indicator.

29. A method for markerless motion capture, comprising:

receiving a captured image having a plurality of pixels arranged in rows and columns;

analyzing the captured image on a pixel-by-pixel basis to determine a candidate, wherein the candidate includes a set of points in three-dimensional space that potentially correspond to a subject;

defining an area of interest proximate to the candidate;

performing a Hough transform on the area of interest to create a Hough image;

analyzing the Hough image on a pixel-by-pixel basis to identify a glob, wherein the glob is a set of connected hot pixels;

determining three-dimensional coordinates for candidate points that correspond to the glob;

using the three-dimensional coordinates for the candidate points to determine a candidate of interest; and

comparing the candidates of interest to predetermined characteristics of the subject to track the subject'"'"'s motion.

30. The method of claim 29, further comprising:

comparing characteristics of the glob with expected characteristics of the subject; and

if the characteristics of the glob satisfy the expected characteristics of the subject, then identifying the candidate points corresponding to the glob.

31. The method of claim 30, further comprising:

if characteristics of the candidate of interest satisfy the predetermined characteristics of the subject, then storing data corresponding to the candidate of interest, so that the data can be used to recreate the motion of the subject.

32. The method of claim 30, wherein the subject is a person'"'"'s head.