Mode- based multi-hypothesis tracking using parametric contours

• US 20060078163A1
• Filed: 11/17/2005
• Published: 04/13/2006
• Est. Priority Date: 06/07/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Chen, Yunqiang
• Granted Patent: US 7,231,064 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/755 Deformable models or variational models, e.g. snakes or active contours
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A system and method for object tracking using probabilistic mode-based multi-hypothesis tracking (MHT) provides for robust and computationally efficient tracking of moving objects such as heads and faces in complex environments. A mode-based multi-hypothesis tracker uses modes that are local maximums which are refined from initial samples in a parametric state space. Because the modes are highly representative, the mode-based multi-hypothesis tracker effectively models non-linear probabilistic distributions using a small number of hypotheses. Real-time tracking performance is achieved by using a parametric causal contour model to refine initial contours to nearby modes. In addition, one common drawback of conventional MHT schemes, i.e., producing only maximum likelihood estimates instead of a desired posterior probability distribution, is addressed by introducing an importance sampling framework into MHT, and estimating the posterior probability distribution from the importance function.

Claims

1. A method for tracking objects in sequential image frames comprising performing steps for:

acquiring at least two sequential images of a scene that includes at least one object of interest;

performing an active contour analysis of each sequential image frame for identifying at least one contour in each sequential image frame which corresponds to a local maximum in a distribution of possible states for the object of interest;

generating an importance function from a sampling of the identified contours for each sequential image frame;

for each contour, using the importance function to estimate a posterior probability of a tracking state for the object of interest with respect to each image frame; and

identifying a contour having a highest estimated posterior probability as representing a current tracking state of the object of interest for each sequential image frame.

2. The method of claim 1 further comprising steps for storing the acquired sequential images on a computer readable medium.

3. The method of claim 1 wherein the active contour analysis comprises steps for performing an iterative search of a 2D image plane using a 2D image model for identifying the at least one contour in each sequential image frame.

4. The method of claim 3 further comprising steps for limiting potential deformations of the identified contours by fitting the identified contours to a parametric contour which acts as a limiting shape prior.

5. The method of claim 1 wherein the active contour analysis comprises steps for performing a single iteration analysis of a 1D causal contour model for identifying the at least one contour in each sequential image frame.

6. The method of claim 5 wherein the active contour analysis of the 1D causal contour model comprises steps for determining deformations along at least one normal line of a contour representing a prior state of the object of interest.

7. The method of claim 5 further comprising steps for limiting potential deformations of the identified contours by fitting the identified contours to a parametric contour which acts as a limiting shape prior.

8. A computer-readable medium having computer executable instructions for probabilistically tracking at least one object throughout a sequence of images, said computer executable instructions comprising:

for each image, extracting at least one contour that represents a local maximum in a distribution that is refined from initial samples in a parametric state space representing an object of interest in each image;

refining each contour using a parametric causal contour model;

constructing an importance sampling function from a sampling of the contours for each image;

estimating a posterior probability of a tracking state for each contour; and

for each image, choosing a contour having a highest posterior probability as representing a current object state.

9. The computer-readable medium of claim 8 wherein extracting at least one contour from each image comprises performing an active contour search of each image given a prior object state.

10. The computer-readable medium of claim 9 wherein the active contour search comprises performing an iterative search of a 2D image plane representing the image using a 2D contour model of the object.

11. The computer-readable medium of claim 10 further comprising limiting potential deformations of the identified contours by fitting the identified contours to a parametric contour which acts as a limiting shape prior.

12. The computer-readable medium of claim 9 wherein the active contour search is performed using a 1D contour model of the object by determining deformations along at least one normal line of a contour representing a prior state of the object.

13. The computer-readable medium of claim 12 further comprising limiting potential deformations of the identified contours by fitting the identified contours to a parametric contour which acts as a limiting shape prior.

14. The computer-readable medium of claim 8 wherein refining each contour using a parametric causal contour model comprises using a causal smoothness constraint to limit potential roughness of the identified contours.

15. The computer-readable medium of claim 8 further comprising fitting each contour to a parametric contour which is used to model the at least one object prior to constructing the importance sampling function from a sampling of the contours for each image.

16. A system for automatically tracking a human head through at least two sequential image frames, comprising using a computing device to perform the following steps:

capturing at least two sequential image frames of a scene including a human head to be tracked;

providing an initial head state;

using the initial head state for performing an active contour analysis of a first image frame for identifying at least one contour that potentially represents the human head being tracked in the first image frame;

constructing an importance sampling function from the identified contours for the first image frame;

estimating a posterior probability of a tracking state for each identified contour; and

choosing an identified contour having a highest posterior probability as best representing a current object state for the first image frame.

17. The system of claim 16 wherein the identified contour which is chosen as best representing the current object state for the first image frame is used in place of the initial head state in performing an active contour analysis of a subsequent image frame for identifying at least one contour that potentially represents the human head being tracked in the subsequent image frame.

18. The system of claim 17 wherein the active contour analysis comprises an iterative search of a 2D image plane using a 2D image model for identifying the at least one contour in each sequential image frame.

19. The system of claim 18 further comprising limiting potential deformations of the identified contours by fitting each of the identified contours to a parametric ellipse.

20. The system of claim 19 wherein the active contour analysis comprises performing a single iteration analysis of a 1D causal contour model by determining deformations along at least one normal line of a contour representing a prior state of the human head, and limiting potential deformations of the identified contours by fitting each of the identified contours to a parametric ellipse.