System and process for bootstrap initialization of nonparametric color models

• US 20050190964A1
• Filed: 04/26/2005
• Published: 09/01/2005
• Est. Priority Date: 06/13/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Toyama, Kentaro
• Granted Patent: US 7,539,327 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A system and process for automatically learning a reliable color-based tracking system is presented. The tracking system is learned by using information produced by an initial object model in combination with an initial tracking function to probabilistically determine the configuration of one or more target objects in a temporal sequence of images, and a data acquisition function for gathering observations relating to color in each image. The observations gathered by the data acquisition function include information that is relevant to parameters desired for a final color-based object model. A learning function then uses probabilistic methods to determine conditional probabilistic relationships between the observations and probabilistic target configuration information to learn a color-based object model automatically tailored to specific target objects. The learned object model is then used in combination with the final tracking function to probabilistically locate and track specific target objects in one or more sequential images.

Claims

1. A method in a computer system for tracking at least one object in at least one sequential image, comprising using a computer to perform the following steps:

(a) a step for generating a state estimate defining probabilistic configurations of each object for each sequential image by automatically generating a first probability distribution function modeled using a first histogram to represent a range of observed pixel colors;

(b) a step for generating observations of pixel color for each sequential image;

(c) a step for automatically learning a color-based object model using the state estimate and the observations;

(d) a step for computing a second probability distribution function modeled using a second histogram to represent a background for each image;

(e) a step for automatically weighting the first and second histograms in relation to the expected relative areas of object and non-object areas, respectively, within each image; and

(f) a step for automatically tracking each object using the learned color-based model with a color-based tracking function.

2. The method of claim 1 wherein the step for generating the state estimate further comprises a step for determining the probabilistic configurations of each object by performing an initial processing of each sequential image.

3. The method of claim 2 wherein the step for performing an initial processing of each sequential image includes a step for employing a tracking system comprising a tracking function in combination with an object model for probabilistically detecting object configuration information.

4. The method of claim 2 wherein the step for performing an initial processing of each sequential image includes a step for employing a contour-based tracking function in combination with a contour-based object model for probabilistically detecting object configuration information.

5. The method of claim 1 wherein the step for generating the observations of pixel color comprises a step for collecting pixel color information over the entirety of each image.

6. The method of claim 1 wherein the step for generating the observations of pixel color comprises a step for collecting pixel color information over specific portions of each image.

7. The method of claim 6 wherein the step for generating the observations of pixel color includes a step for employing the state estimate to identify specific relevant regions of each image over which pixel color information will be collected.

9. The method of claim 1 further comprising a step for representing the first and second histograms using a Dirichlet function.

11. The method of claim 1 further comprising a step for weighting the first and second histograms using a preliminary color-based model represented by a third probability distribution function modeled using a third histogram.

13. The method of claim 1 wherein the step for automatically learning the color-based object model comprises a step for performing a bin-by-bin comparison between the first histogram and the second histogram.

14. The method of claim 13 wherein bins in the first histogram having values exceeding corresponding bins in the second histogram correspond to those color ranges representing the learned color-based object model.

8. (canceled)

10. (canceled)

12. (canceled)

15. A method for generating a color-based object model, comprising computer program modules for performing the following steps:

a state estimate module for performing steps for generating a state estimate defining probabilistic states of an object for each of at least one sequential images;

an observation module for performing steps for generating observations of pixel color for each sequential image;

wherein the observations of pixel color are represented by a first probability distribution function modeled using a first histogram;

a background image module for performing steps for providing a background image for probabilistically representing a known fixed state relative to each image, wherein the background image is represented by a second probability distribution function modeled using a second histogram;

a first learning module for performing steps for automatically learning a preliminary color-based model for roughly representing each target object using a third probability distribution function modeled using a third histogram; and

a second learning module for performing steps for automatically learning the color-based object model using the state estimates and the observations.

16. The method of claim 15, further comprising a target configuration module for performing steps for using the learned color-based object model in a tracking system for identifying a configuration of at least one target object in each sequential image.

28. The method of claim 15 wherein the step for generating the state estimate comprises a step for processing each image with an initial object model and an initial tracking function.

29. The method of claim 28 further comprising an object model processing module for performing steps for iteratively replacing the initial object model with the learned color-based object model and for replacing the initial tracking function with a color-based tracking function to improve the accuracy of the learned color-based object model.

30. The method of claim 15 further comprising an color model processing module for performing steps for iteratively replacing the preliminary color-based model with the learned color-based object model to improve the accuracy of the learned color-based object model.

17-27. -27. (canceled)

31-33. -33. (canceled)

34. A method for identifying the configuration of objects of interest in a scene, comprising performing the following steps:

an estimate generation step for generating an initial configuration estimate for objects of interest within the scene;

a model generation step for generating an initial object model and an initial tracking function, and wherein the initial object model is comprised of parameters used by the initial tracking function for determining the configuration of objects within the scene;

a color identification step for identifying pixel color information within the scene that is relevant to a learned color-based object model, wherein the pixel color information is represented using a probability distribution function modeled by a first Dirichlet function;

a background image generation step for generating a background image representing the scene using a probability distribution function modeled by a second Dirichlet function;

a learning step for automatically learning the color-based object model by determining probabilistic relationships between the initial configuration estimates and the pixel color information using a preliminary color-based object model represented by a third Dirichlet function for establishing a probabilistic baseline to assist in learning the learned color-based object model, and,

a configuration generation step for generating a final configuration estimate for objects of interest in the scene by using the color-based object model in combination with a color-based tracking function.

39. The method of claim 34 wherein the learning step for automatically learning the color-based object model includes steps for automatically scaling the first and second Dirichlet functions based on expected areas of objects of interest in the scene relative to areas of the scene not expected to contain objects of interest.

40. The method of claim 34 wherein the learning step for automatically learning the color-based object model includes steps for using the third Dirichlet function to weight the first and second Dirichlet functions and steps for determining the difference between the first and second Dirichlet functions to generate the learned color-based object model.

41. The method of claim 39 wherein the learning step for automatically learning the color-based object model includes steps for determining the difference between the first and second Dirichlet functions to generate the learned color-based object model.

35-38. -38. (canceled)

42-47. -47. (canceled)