Method for tracking the motion of a 3-D figure

• US 6,269,172 B1
• Filed: 04/13/1998
• Issued: 07/31/2001
• Est. Priority Date: 04/13/1998
• Status: Expired due to Term

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Rehg, James Matthew, Morris, Daniel D.
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): CHOOBIN, BARRY
• Application Number:
  US09/059,651
• Time in Patent Office: 1,205 Days
• Field of Search: 345/473-474, 382/107, 382/103, 356/27, 734/88, 342/1
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

In a computerized method, a moving articulated figure is tracked in a sequence of 2-D images measured by a monocular camera. The images are individually registered with each other using a 2-D scaled prismatic model of the figure. The 2-D model includes a plurality of links connected by revolute joints to form is a branched, linear-chain of connected links. The registering produces a state trajectory for the figure in the sequence of images. During a reconstructing step, a 3-D model is fitted to the state trajectory to estimate kinematic parameters, and the estimated kinematic parameters are refined using an expectation maximization technique.

Claims

1. A computerized method for tracking a moving articulated figure in a sequence of 2-D images, comprising the steps of:

fitting a 3-D kinematic model of the figure to a state trajectory of a 2-D model of the figure to estimate 3-D kinematic model parameters, the state trajectory comprising a sequence of state vectors, each state vector comprising parameters describing the figure in a corresponding image; and

refining the estimated 3-D kinematic model parameters by fitting a 3-D kinematic model of the figure to the original sequence of images, the fitting comprising adjusting the 3-D kinematic model parameters so as to minimize residual error between a 2-D projection of the 3-D kinematic model and pixel measurements in each frame of the image sequence.

2. The method of claim 1 wherein the 3-D kinematic model parameters are estimated by minimizing a first cost function of the form:

3. The method of claim 2 wherein the refining includes minimizing a second cost function of the form:

4. The method of claim 3 including partitioning the kinematic parameters of the 3-D kinematic model into state kinematic parameters and intrinsic kinematic parameters, the state kinematic parameters having varying values over the sequence of images while the intrinsic kinematic parameters remaining fixed throughout the sequence of images.

5. The method of claim 4 including estimating the state kinematic parameters and the intrinsic kinematic parameters using an expectation-maximization algorithm, the expectation-maximization algorithm iteratively estimating the state kinematic parameters while holding the intrinsic kinematic parameters fixed, and iteratively estimating the intrinsic kinematic parameters while holding the state kinematic parameters fixed until the state kinematic parameters and the intrinsic kinematic parameters each converge.

6. The method of claim 5 including initializing the intrinsic kinematic parameters to a set of nominal values during a first iteration of estimating the state kinematic parameters.

7. The method of claim 2 wherein intrinsic kinematic parameters comprise 3-D link lengths.

8. The method of claim 2 wherein intrinsic kinematic parameters comprise 3-D rotation axis directions.

9. The method of claim 2 wherein state kinematic parameters comprise joint angles.

10. The method of claim 2 wherein the function F₁ determines a residual error between a projection of the 3-D kinematic model on an image plane and corresponding measurements determined by the state trajectory.

11. The method of claim 10 wherein the residual error is related to the image plane distance between joint centers of a scaled prismatic model and a projection of the 3-D kinematic model.

12. The method of claim 3 wherein the function F₂ determines a residual error that measures similarity of corresponding pixels as determined by the projection of the 3-D kinematic model across the image sequence.

13. A computer program product for tracking a moving articulated figure in a sequence of 2-D images, the computer program product comprising usable medium having computer readable code thereon, including program code which:

fits a 3-D kinematic model of the figure to a state trajectory of a 2-D model of the figure to estimate 3-D kinematic model parameters, the state trajectory comprising a sequence of state vectors each state vector comprising parameters describing the figure in a corresponding image; and

refines the estimated 3-D kinematic model parameters by fitting a 3-D kinematic model of the figure to the original sequence of images, the fitting comprising adjusting the 3-D kinematic model parameters so as to minimize residual error between a 2-D projection of the 3-D kinematic model and pixel measurements in each frame of the image sequence.

14. A system for tracking a moving articulated figure in a sequence of 2-D images, comprising the steps of:

means for fitting a 3-D kinematic model of the figure to a state trajectory of a 2-D model of the figure to estimate 3-D kinematic model parameters, the state trajectory comprising a sequence of state vectors, each state vector comprising parameters describing the figure in a corresponding image; and

means for refining the estimated 3-D kinematic model parameters by fitting a 3-D kinematic model of the figure to the original sequence of images, the fitting comprising adjusting the 3-D kinematic model parameters so as to minimize residual error between a 2-D projection of the 3-D kinematic model and pixel measurements in each frame of the image sequence.