Apparatus and method for high-speed marker-free motion capture

• US 7,239,718 B2
• Filed: 05/28/2003
• Issued: 07/03/2007
• Est. Priority Date: 12/20/2002
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Lee, Ji Hyung, Lee, In Ho, Park, Chang Joon, Kim, Do-Hyung, Nam, Seung Woo, Baek, Seong Min
• Primary Examiner(s): Chauhan; Ulka J.
• Assistant Examiner(s): Pappas; Peter-Anthony
• Application Number:
  US10/445,825
• Publication Number:
  US 20040119716A1
• Time in Patent Office: 1,497 Days
• Field of Search: 345/475
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/579 from motion

Abstract

A high-speed marker-free motion capture, which is capable of powerfully detecting a body'"'"'s feature points corresponding to a body'"'"'s end portions such as a head, hands, feet, trunk, arms and legs at a high speed in an illumination change or background or noises of cameras. The extracted feature points of the body can be directly tracked stably in a 3-dimensional space. The position errors of the feature points due to the change of the illumination conditions or a shadow can be automatically corrected and the feature points can be stably tracked with respect to overlapping and disappearance of the feature points. Further, when coordinates of the middle joints are estimated using 3-dimensional coordinates of the extracted feature points of the body, the present invention restores a human model by estimating the positions of the middle joints of the actor with high accuracy without using a motion database, thereby securing the stability and reality of the 3-dimensional motion data required in the motion capture.

Claims

1. An apparatus for a high-speed marker-free motion capture comprising:

a 2-dimensional feature point extraction module for obtaining an image with respect to a background with no actor by using a plurality of cameras having different viewing angles from each other to store the obtained image as RGB color space and HSI color space, obtaining an image with respect to an initial posture of the actor to make blob models with respect to a head, hands, feet, trunk, arms and legs, extracting an adjacent connection relationship between the blob models and storing the extracted relationship, receiving the photographed image of the movement of the actor to extract pixels having the color different from the background, detecting pixels having color and shape similar to the stored blob models with respect to the respective parts of the body to make the detected pixels into the blobs, distinguishing the blobs corresponding to the parts of the body from the correlation of the adjacent blobs with respect to the detected blob, and determining the distinguished blobs as 2-dimensional body feature points;

a 3-dimensional feature point tracking module for performing 3-dimensional matching with respect to the 2-dimensional feature points corresponding to the body'"'"'s end portions extracted by the 2-dimensional feature point extraction module, restoring 3-dimensional coordinates, and performing a tracking operation by predicting shape changes of the feature points and the body'"'"'s center of gravity; and

a 3-dimensional motion restoration module for predicting a position of a middle joint using the 3-dimensional coordinates of the 2-dimensional feature points tracked by the 3-dimensional feature point tracking module, generating 3-dimensional motion data, and restoring the 3-dimensional motion data to a human model;

wherein if the 2-dimensional feature point extraction module receives the photographed image of the movement of the actor and extracts the pixels having the color different from the background, the 2-dimensional feature point extraction module extracts candidate pixels having the color different from the background through a comparison of the color components in the RGB color space; and

wherein the 2-dimensional feature point extraction module first carries out a comparison sequence with respect to the RGB color space to select candidate pixels, and transforms the candidate pixels into the HSI color space to find pixels having color similar to the models of the respective parts of the body.

2. The apparatus of claim 1, wherein the 2-dimensional feature point extraction module constructs the blob models with respect to the head, the hands, the feet, the trunk using color component information in the HSI color space, and a magnitude and a shape information with respect to the respective parts of the body.

3. The apparatus of claim 1, wherein the 2-dimensional feature point extraction module detects the pixels having color and shape similar to the stored blob models with respect to the respective parts of the body using the HSI color space, and makes the detected pixels into the blobs.

4. The apparatus of claim 1, wherein the 2-dimensional feature point extraction module selects the pixels having the color difference to separate the actor from the background, and retrieves the pixels similar to the models of the respective parts of the body.

5. The apparatus of claim 1, wherein the 2-dimensional feature point extraction module distinguishes the blobs corresponding to the parts of the body and determines the distinguished blobs as 2-dimensional body feature points.

6. The apparatus of claim 1, wherein the 3-dimensional feature point tracking module predicts and calculates the movement of the center of the gravity in the movement up to the previous frame, and predicts the shape change of the feature points and the body'"'"'s center of the gravity by correctly tracking the feet using a prediction of the center of the gravity, which limits the movement of the foot to which the center of the gravity is applied.

7. The apparatus of claim 1, wherein the 3-dimensional motion restoration module sets boundary values with respect to the actor'"'"'s body, and estimates positions of the middle joint of the actor by requiring that there is no collision with the actor'"'"'s own body based on the boundary values.