Method of dealing with occlusion when tracking multiple objects and people in video sequences

• US 6,542,621 B1
• Filed: 08/31/1999
• Issued: 04/01/2003
• Est. Priority Date: 08/31/1998
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Olson, Thomas J., Brill, Frank Z.
• Primary Examiner(s): Johnson, Timothy M.
• Application Number:
  US09/386,871
• Time in Patent Office: 1,309 Days
• Field of Search: 382/107, 382/209, 382/257, 382/103, 382/291, 734/88, 348/154, 348/155, 356/27
• US Class Current: 382/103
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

This invention employs probabilistic templates, or p-templates, which probabilistically encode the rough position and extent of the tracked object'"'"'s image. The p-templates track objects in the scene, one p-template per object. They can be used to incorporate three-dimensional knowledge about the scene, and to reason about occlusion between the objects tracked by the p-templates. This invention requires video capture and digitization hardware, image processing hardware such as a digital signal processor, and a method for estimating the image size of a person standing at a given location in the image.

Claims

1. A computer implemented method of tracking moving objects in series of video images comprising the steps of:

forming a probablistic model of tracked objects;

determining a probability that said pixel corresponds to a background or to each tracked object for each pixel of a current video image dependent upon a pixel value of a current video image and a probablistic model of each tracked object from an immediately prior video image;

selecting a source for each pixel of said current video image from among said background and said tracked objects dependent upon said determined probabilities;

determining a location for each tracked object dependent upon said selected pixel sources; and

projecting a probablistic model of each tracked object into a next video image based upon said determined location for each corresponding tracked object.

2. The computer implemented method of claim 1, wherein:

said step of determining a location for each tracked object includes tracking said probablistic model corresponding to each tracked object where said probablistic model produces a higher probability than said probablistic model of any other tracked object.

3. The computer implemented method of claim 1, wherein:

said step of projecting a probablistic model of each tracked object includes tracking three dimensional location of each tracked object and projecting an actual size of said probablistic model actual size into apparent size within said video image.

4. The computer implemented method of claim 3, wherein:

said step of forming a probablistic model of a tracked object includes forming a probabilistic model of a standing person as a Gaussian oval with vertical dimension corresponding to an actual height of about six feet and a horizontal dimension corresponding to an actual width of about two feet.

5. The computer implemented method of claim 1, wherein:

said step of determining a probability that said pixel corresponds to a background or to each tracked object includes

computing a pixel value mean and pixel value variance for each pixel of background over a plurality of video images, and

computing a raw probability that said pixel corresponds to background based upon a current pixel value relative to a Gaussian distribution of pixel values having said computed pixel value- mean- and pixel value variance of said pixel.

6. The computer implemented method of claim 5, wherein:

said step of determining a probability that said pixel corresponds to a background or to each tracked object includes computing a raw probability that said pixel corresponds to each tracked object based upon said probabilistic model of said object at said pixel and assuming all pixel values are equally likely for each tracked object.

7. The computer implemented method of claim 6, wherein:

said step of determining a probability that said pixel corresponds to a background or to each tracked object further includes computing an normalized probability for background and each tracked object.

8. The computer implemented method of claim 5, wherein:

said step of determining a probability that said pixel corresponds to a background or to each tracked object further includes

setting a raw probability that said pixel corresponds to an unknown object as a small constant probability at every pixel,

computing an normalized probability for background, each tracked object and said unknown object;

said step of selecting a source for each pixel of said current video image further includes selecting said unknown object as said source if said unknown object has a highest probability; and

instantiating a new tracked object when pixels selected as from said unknown object are sufficient in number and distribution to match a probabilistic model of an object.

9. A apparatus for tracking moving objects in a defined space comprising:

a video imaging device forming a series of two dimensional video images of said defined space; and

a data processing apparatus receiving said series of video images from said video imaging device, said data processing apparatus programmed to

form a probablistic model of tracked objects;

determine a probability that said pixel corresponds to a background or to each tracked object for each pixel of a current video image dependent upon a pixel value of a current video image and a probablistic model of each tracked object from an immediately prior video image;

select a source for each pixel of said current video image from among said background and said tracked objects dependent upon said determined probabilities;

determine a location for each tracked object dependent upon said selected pixel sources; and

project a probablistic model of each tracked object into a next video image based upon said determined location for each corresponding tracked object.

10. The apparatus for tracking moving objects of claim 9, wherein:

said data processing apparatus is further programmed to determine a location of for each tracked object by tracking said probablistic model corresponding to each tracked object where said probablistic model produces a higher probability than said probablistic model of any other tracked object.

11. The apparatus for tracking moving objects of claim 9, wherein:

said data processing apparatus is further programmed to project a probablistic model of each tracked object by tracking three dimensional location of each tracked object and projecting an actual size of said probablistic model actual size into apparent size within said video image.

12. The apparatus for tracking moving objects of claim 11, wherein:

said data processing apparatus is further programmed to form a probablistic model of a tracked object by forming a probabilistic model of a standing person as a Gaussian oval with vertical dimension corresponding to an actual height of about six feet and a horizontal dimension corresponding to an actual width of about two feet.

13. The apparatus for tracking moving objects of claim 9, wherein:

said data processing apparatus is further programmed to determine the probability that said pixel corresponds to a background or to each tracked object by

computing a pixel value mean and pixel value variance for each pixel of background over a plurality of video images, and

computing a raw probability that said pixel corresponds to background based upon a current pixel value relative to a Gaussian distribution of pixel values having said computed pixel value mean and pixel value variance of said pixel.

14. The apparatus for tracking moving objects of claim 13, wherein:

said data processing apparatus is further programmed to determine the probability that said pixel corresponds to a background or to each tracked object by computing a raw probability that said pixel corresponds to each tracked object based upon said probabilistic model of said object at said pixel and assuming all pixel values are equally likely for each tracked object.

15. The apparatus for tracking moving objects of claim 14, wherein:

said data processing apparatus is further programmed to determine the probability that said pixel corresponds to a background or to each tracked object by computing an normalized probability for background and each tracked object.

16. The apparatus for tracking moving objects of claim 13, wherein:

said data processing apparatus is further programmed to determine the probability that said pixel corresponds to a background or to each tracked object by

setting a raw probability that said pixel corresponds to an unknown object as a small constant probability at every pixel,

computing an normalized probability for background, each tracked object and said unknown object;

said data processing apparatus is further programmed to select the source for each pixel of said current video image by selecting said unknown object as said source if said unknown object has a highest probability; and

said data processing apparatus is further programmed to instantiate a new tracked object when pixels selected as from said unknown object are sufficient in number and distribution to match a probabilistic model of an object.