System and process for bootstrap initialization of nonparametric color models

• US 20050008193A1
• Filed: 08/04/2004
• Published: 01/13/2005
• Est. Priority Date: 06/13/2000
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Toyama, Kentaro
• Application Number:
  US10/911,777
• Publication Number:
  US 20050008193A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A system and process for automatically learning a reliable color-based tracking system is presented. The tracking system is learned by using information produced by an initial object model in combination with an initial tracking function to probabilistically determine the configuration of one or more target objects in a temporal sequence of images, and a data acquisition function for gathering observations relating to color in each image. The observations gathered by the data acquisition function include information that is relevant to parameters desired for a final color-based object model. A learning function then uses probabilistic methods to determine conditional probabilistic relationships between the observations and probabilistic target configuration information to learn a color-based object model automatically tailored to specific target objects. The learned object model is then used in combination with the final tracking function to probabilistically locate and track specific target objects in one or more sequential images.

Claims

1. A system for tracking at least one object in at least one sequential image, comprising:

a general purpose computing device; and

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to;

(a) generate a state estimate defining probabilistic configurations of each object for each sequential image;

(b) generate observations of pixel color for each sequential image;

(c) automatically learn a color-based object model using the state estimate and the observations, and without using any of known and predefined object contours; and

(d) automatically track each object using the learned color-based model with a color-based tracking function.

2. The system of claim 1 wherein generating the state estimate comprises determining the probabilistic configurations of each object using an initial image processing program module.

3. The system of claim 2 wherein the initial image processing program module employs a tracking system comprising a tracking function in combination with an object model for probabilistically detecting object configuration information.

4. The system of claim 2 wherein the initial image processing program module employs a contour-based tracking function in combination with a contour-based object model for probabilistically detecting object configuration information.

5. The system of claim 1 wherein generating the observations of pixel color comprises collecting pixel color information over the entirety of each image.

6. The system of claim 1 wherein generating the observations of pixel color comprises collecting pixel color information over specific portions of each image.

7. The system of claim 6 wherein the program module for generating the observations of pixel color employs the state estimate to identify specific relevant regions of each image over which pixel color information will be collected.

8. The system of claim 1 wherein generating the observations of pixel color comprises automatically generating a first probability distribution function modeled using a first histogram to represent a range of observed pixel colors.

9. The system of claim 8 wherein the histogram is represented by a Dirichlet function.

10. The system of claim 8 wherein the program module for automatically learning the color-based object model automatically computes a second probability distribution function modeled using a second histogram to represent a background for each image.

11. The system of claim 10 where a preliminary color-based model represented by a third probability distribution function modeled using a third histogram is used to weight the first and second histograms.

12. The system of claim 10 wherein the first and second histograms are automatically weighted in relation to the expected relative areas of object and non-object areas, respectively, within each image.

13. The system of claim 10 wherein automatically learning the color-based object model comprises performing a bin-by-bin comparison between the first histogram and the second histogram.

14. The system of claim 13 wherein bins in the first histogram having values exceeding corresponding bins in the second histogram correspond to those color ranges representing the learned color-based object model.

15. A computer-implemented process for generating a color-based object model, comprising:

generating a state estimate defining probabilistic states of an object for each of at least one sequential images;

generating observations of pixel color for each sequential image; and

automatically learning the color-based object model using the state estimates and the observations and without using any of known and predefined object contours.

16. The computer-implemented process of claim 15, further comprising using the learned color-based object model in a tracking system for identifying a configuration at least one target object in each sequential image.

17. The computer-implemented process of claim 15 wherein a confidence measure is associated with the observations of pixel color.

18. The computer-implemented process of claim 17 wherein the observations of pixel color are weighted in proportion to the confidence measure.

19. The computer-implemented process of claim 15 wherein the observations of pixel color are collected for each entire image.

20. The computer-implemented process of claim 15 wherein observations of pixel color are collected over specific portions of each image wherein the state estimate has a probability greater than a minimum threshold level.

21. The computer-implemented process of claim 15 wherein the observations of pixel color are represented by a first probability distribution function modeled using a first histogram.

22. The computer-implemented process of claim 21 further comprising a background image for probabilistically representing a known fixed state relative to each image, and wherein the background image is represented by a second probability distribution function modeled using a second histogram.

23. The computer-implemented process of claim 22 further comprising a preliminary color-based model for roughly representing each target object, and wherein the preliminary color-based model is represented by a third probability distribution function modeled using a third histogram.

24. The computer-implemented process of claim 23 wherein the first and second histograms are scaled in relation to expected relative areas of object and non-object areas, respectively, within each image.

25. The computer-implemented process of claim 24 wherein the first and second histogram are weighted in relation to the third histogram.

26. The computer-implemented process of claim 24 wherein the second histogram is subtracted from the first histogram via a bin-by-bin comparison between the first and second histogram.

27. The computer-implemented process of claim 26 wherein the subtraction yields a fourth histogram for representing the learned color-based object model.

28. The computer-implemented process of claim 15 wherein generating the state estimate comprises processing each image with an initial object model and an initial tracking function.

29. The computer-implemented process of claim 28 wherein the initial object model is iteratively replaced with the learned color-based object model and the initial tracking function is replaced with a color-based tracking function to improve the accuracy of the learned color-based object model.

30. The computer-implemented process of claim 23 wherein the preliminary color-based model is iteratively replaced with the learned color-based object model to improve the accuracy of the learned color-based object model.

31. The computer-implemented process of claim 30 wherein generating the state estimate comprises processing each image with an initial object model and an initial tracking function.

32. The computer-implemented process of claim 31 wherein the initial object model is iteratively replaced with the learned color-based object model and the initial tracking function is replaced with a color-based tracking function to improve the accuracy of the learned color-based object model.

33. The computer-implemented process of claim 15 further comprising a process for gathering the sequential images.

34. A computer-readable memory for identifying the configuration of objects of interest in a scene, comprising:

a computer-readable storage medium; and

a computer program comprising program modules stored in the storage medium, wherein the storage medium is so configured by the computer program that it causes the computer to,

generate an initial configuration estimate for objects of interest within the scene,

identify pixel color information within the scene that is relevant to a learned color-based object model,

automatically learn the color-based object model by determining probabilistic relationships between the initial configuration estimates and the pixel color information without using any of known and predefined object contours, and,

generate a final configuration estimate for objects of interest in the scene by using the color-based object model in combination with a color-based tracking function.

35. The computer-readable memory of claim 34 wherein the program module for generating the initial configuration estimate further includes an initial object model and an initial tracking function, and wherein the initial object model is comprised of parameters used by the initial tracking function for determining the configuration of objects within the scene.

36. The computer-readable memory of claim 35 wherein the pixel color information is represented using a probability distribution function modeled by a first Dirichlet function.

37. The computer-readable memory of claim 36 further comprising a background image representing the scene, and wherein the background image is represented using a probability distribution function modeled by a second Dirichlet function.

38. The computer-readable memory of claim 37 wherein the program module for automatically learning the color-based object model further includes a preliminary color-based object model represented by a third Dirichlet for establishing a probabilistic baseline to assist in learning the learned color-based object model.

39. The computer readable memory of claim 37 wherein the program module for automatically learning the color-based object model automatically scales the first and second Dirichlet functions based on expected areas of objects of interest in the scene relative to areas of the scene not expected to contain objects of interest.

40. The computer readable memory of claim 38 wherein the program module for automatically learning the color-based object model automatically uses the third Dirichlet function to weight the first and second Dirichlet functions.

41. The computer readable memory of claim 39 wherein the program module for automatically learning the color-based object model automatically determines the difference between the first and second Dirichlet functions to generate the learned color-based object model.

42. The computer readable memory of claim 40 wherein the program module for automatically learning the color-based object model automatically determines the difference between the first and second Dirichlet functions to generate the learned color-based object model.

43. The computer-readable memory of claim 41 wherein the learned color-based object model is represented using a probability distribution function modeled by a fourth Dirichlet function.

44-47. -47. (Cancelled)