Object tracking system using statistical modeling and geometric relationship

• US 5,845,009 A
• Filed: 03/21/1997
• Issued: 12/01/1998
• Est. Priority Date: 03/21/1997
• Status: Expired due to Term

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Burns, John Brian, Marks, Richard Lee
• Primary Examiner(s): Teska, Kevin J.
• Assistant Examiner(s): FREJD, RUSSELL WARREN
• Application Number:
  US08/828,211
• Time in Patent Office: 620 Days
• Field of Search: 382/115, 382/118, 382/128, 382/159, 382/160, 382/162, 382/165, 382/168, 382/170, 382/181, 382/190, 382/192, 382/194, 382/199, 382/203, 382/217, 382/218, 382/228, 348/169, 364/516, 364/526, 364/578
• US Class Current: 382/228
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

A system and method which identifies an object in an image. The system accesses a statistical model corresponding to a statistically identifying feature of the object and accesses a characteristic corresponding to a second identifying feature of the object. The system generates a value set from the image, the statistical model, the characteristic, and a geometric relationship. The value set indicates a likely location of the object in the image. The object is located using the value set.

Claims

1. A method of identifying an object in a first image using a computer and a second image, the second image having a first set of points that aid in identifying the object and a second set of points that aid in identifying the object, the first set of points having a geometric relationship with the second set of points, the method comprising the steps of:

accessing a statistical model corresponding to the first set of points;

accessing a characteristic corresponding to the second set of points;

generating a value set from the first image, the statistical model, the characteristic, and the geometric relationship, the value set indicating a likely location of the object in the first image; and

locating the object in the first image using the value set.

2. The method of claim 1 wherein the step of generating the value set comprises generating at least a value indicating a correspondence between a portion of the first image and the statistical model, the characteristic, and the geometric relationship.

3. The method of claim 2 wherein the portion of the first image includes a pixel, and wherein the correspondence indicates a degree of matching between the pixel, the statistical model, and the characteristic.

4. The method of claim 1 wherein the statistical model includes a histogram of the first set of points.

5. The method of claim 4 wherein the characteristic includes a second histogram of the second set of points.

6. The method of claim 5 wherein the first set of points corresponds to a first region of the second image, the second set of points corresponds to a second region of the second image, and the geometric relationship corresponds to a predetermined distance between the first region and the second region.

7. The method of claim 6 wherein the second image comprises a plurality of pixels and wherein the predetermined distance corresponds to an offset in pixels between the first region and the second region.

8. The method of claim 5 wherein the histogram and the second histogram are color histograms.

9. The method of claim 5 further comprising the steps of:

generating a third histogram of the first image; and

prior to generating the value set, normalizing the histogram and the second histogram using the third histogram.

10. A method of identifying an object in a first image, the method comprising the steps of:

accessing a second image, the second image including a third image identifying the object;

generating a first model from the third image;

the first model corresponding to a first statistical analysis of a first part of the third image;

generating a second model from the third image, the second model corresponding to a characteristic of a second part of the third image, the second part having a geometric relationship with the first part;

accessing the first image;

generating a value set from the first image, the first model, the second model, and the geometric relationship, the value set indicating a likely location of the object in the first image; and

using the value set to identify the object in the first image.

11. The method of claim 10 wherein the step of generating the value set comprises generating a value indicating a correlation between the first image and the first model, the second model, and the geometric relationship.

12. The method of claim 11 wherein generating the first model includes generating a first histogram of the first part, wherein generating the second model includes generating a second histogram of the second part, and the method further comprises:

generating a third histogram of the first image; and

prior to generating the value set, normalizing the first histogram and the second histogram using the third histogram.

13. The method of claim 12 wherein the normalizing the first histogram and the second histogram includes dividing the first histogram by the third histogram, and dividing the second histogram by the third histogram.

14. The method of claim 12 wherein the first image includes a plurality of color values, the first model includes a first histogram of the first part, the second model includes a second histogram of the second part, and generating the value indicating the correlation between the first part, the second part, the first model and the second model includes:

accessing a first color value at a first location in the first image, the first color value having a first correspondence to the first histogram;

accessing a second color value at a second location in the first image, the second color value having a second correspondence to the second histogram, the second location having a second geometric relationship with the first location, the second geometric relationship corresponding to the geometric relationship;

determining a third correspondence between the first color value and the second color value using the first histogram and the second histogram; and

determining the value from the first correspondence, the second correspondence, and the third correspondence.

15. The method of claim 10 wherein the object is a human, wherein the first part corresponds to the head of the human, and wherein the second part corresponds to the torso of the human.

16. The method of claim 10 further comprising the step of using a mouse pointing device to define the first part with a window.

17. The method of claim 10 wherein the accessing the second image includes receiving a the image from a video camera.

18. The method of claim 17 wherein the accessing the second image includes receiving the second image from a network.

19. The method of claim 17 wherein the accessing the second image includes extracting the second image from a digital movie.

20. A method of determining a likely location of an object in an first image, the method comprising the steps of:

receiving a modeling image, the modeling image including an image of the object;

generating a first color histogram from the modeling image, the first color histogram corresponding to a first region of the object, the first region also corresponding to the image of the object;

generating a second color histogram from the modeling image, the second color histogram corresponding to a second region of the object;

receiving the first image; and

generating a value set from the first image, the first color histogram, the second color histogram, and a geometric relationship between the first region and the second region, the value set having values indicating a likely location of the object in the first image.

21. The method of claim 20, wherein the step of generating the value set comprises generating at least one value indicating a first relationship between a first portion of the first image and the first color histogram, the one value also indicating a second relationship between a second portion of the first image and the second color histogram.

22. The method of claim 21 wherein the geometric relationship comprises an offset between the first region of the object and the second region of the object.

23. The method of claim 22, wherein the first portion of the first image has a first color, second portion of the first image has a second color and the step of generating as least one value further comprises:

selecting a first population value from the first color histogram, the first population value corresponding to the first color;

selecting a second population value from the second color histogram, the second population value corresponding to the second color; and

determining a relationship between the first population value and the second population value;

using the relationship to generate the one value.

24. The method of claim 21, wherein the first portion of the first image comprises a pixel.

25. The method of claim 21, wherein the first portion of the first image has a first color and the step of generating at least one value comprises:

selecting a first population value from the first color histogram, the first population value corresponding to the first color; and

using the first population value to generate the at least one value.

26. The method of claim 20, wherein the step of generating the first color histogram comprises the step of measuring hue and saturation values of the modeling image.

27. The method of claim 26, wherein the step of generating the first color histogram further comprises the step of measuring luminance values of the modeling image.

28. The method of claim 20, wherein the step of generating the first color histogram further comprises the step of measuring red, blue, and green values of the modeling image.

29. The method of claim 20, wherein the step of generating the first color histogram further comprises the step of determining a count of each occurrence of a color value of a set of colors.

30. An object tracking system for tracking an object in a tracking image, the object tracking system comprising:

a camera; and

a computer system coupled to the camera, the computer system having a memory, a processor, and an object identification procedure, the object identification procedure including,

a modeling procedure that generates a first statistical model of a first region of the object and a second statistical model of a second region of the object based on a modeling image, the first region having a first geometric relationship with the second region, and

a tracking procedure that locates the object at a location in the tracking image and generates a set of camera controls for the camera responsive to determining the location, the location having a first portion of the tracking image corresponding to the first statistical model and having a second portion of the tracking image corresponding to the second statistical model, wherein the first portion has a second geometric relationship with the second portion, the second geometric relationship corresponding to the first geometric relationship.

31. The object tracking system of claim 30, further comprising a camera control procedure that uses the set of camera controls to control the camera in response to the object identification procedure.

32. The object tracking system of claim 30, wherein the first portion comprises a plurality of pixels.

33. The object tracking system of claim 30, wherein the first statistical model comprises a color histogram.

34. The object tracking system of claim 30, wherein the first statistical model comprises a texture histogram.

35. The object tracking system of claim 30, wherein the first statistical model comprises a intensity histogram.

36. The object tracking system of claim 30, wherein the modeling procedure generates second models of a second object and wherein the tracking procedure locates the second object at a second location in the tracking image using the second models.