Method and apparatus for personnel detection and tracking

• US 6,188,777 B1
• Filed: 06/22/1998
• Issued: 02/13/2001
• Est. Priority Date: 08/01/1997
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Interval Research Corporation
• Inventors: Woodfill, John, Baker, Harlyn, Darrell, Trevor, Gordon, Gaile, Harville, Michael
• Primary Examiner(s): MEHTA, BHAVESH M
• Application Number:
  US09/102,101
• Time in Patent Office: 967 Days
• Field of Search: 382/103, 382/106, 382/118, 382/154, 382/162, 382/164, 382/165, 382/224, 348/169-173
• US Class Current: 382/103
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Techniques from computer vision and computer graphics are combined to robustly track a target (e.g., a user) and perform a function based upon the image and/or the identity attributed to the target'"'"'s face. Three primary modules are used to track a user'"'"'s head: depth estimation, color segmentation, and pattern classification. The combination of these three techniques allows for robust performance despite unknown background, crowded conditions, and rapidly changing pose or expression of the user. Each of the modules can also provide an identity classification module with valuable information so that the identity of a user can be estimated. With an estimate of the position of a target in 3-D and the target'"'"'s identity, applications such as individualized computer programs or graphics techniques to distort and/or morph the shape or apparent material properties of the user'"'"'s face can be performed. The system can track and respond to a user'"'"'s face in real-time using completely passive and non-invasive techniques.

Claims

1. An image detecting and tracking system, comprising:

at least two image sensing devices, each of said image sensing devices producing image data;

a first tracking module which receives the image data, generates a disparity image based upon the image data from each of said image sensing devices, and determines a target region in said disparity image;

a second tracking module which classifies and tracks said target region through color segmentation; and

a third tracking module which distinguishes individual features and tracks identified features which are located within a classified target region.

2. The image detecting and tracking system of claim 1, wherein said third tracking module distinguishes individual features based upon intensity patterns in a sensed image.

3. The image detecting and tracking system of claim 1, wherein said first tracking module determines the relative distances of respective target regions in the sensed image.

4. The image detecting and tracking system of claim 1, wherein said first tracking module locates a target area using a connected components grouping analysis.

5. The image detecting and tracking system of claim 4, wherein said image data is received in the form of video frames and said connected components grouping analysis is performed for each pair of video frames received from said image detecting devices.

6. The image detecting and tracking system of claim 1, wherein said classification in said second module is performed with a Gaussian prior probability model.

7. The image detecting and tracking system of claim 1, wherein said first tracking module generates a boolean mask based upon a determined target region, and said second and third tracking modules only process image data contained within the mask.

8. The image detecting and tracking system of claim 7, wherein said boolean mask corresponds to the silhouette of a person detected in the sensed image.

9. The image detecting and tracking system of claim 1, wherein said individual features are human features.

19. The image detecting and tracking system of claim 1, wherein said disparity image represents depth information.

20. The image detecting and tracking system of claim 3, wherein said relative distances are distances between the sensing devices and said respective target regions.

10. A method for image detecting and tracking comprising:

detecting an image via two separate optical paths;

receiving image data from said paths, generating a disparity image based upon the image data from each of said paths, and determining a target region in said disparity image;

classifying and tracking said target region through color segmentation;

detecting facial patterns within said target region based on said image data; and

displaying a image of the facial patterns detected within said target region.

11. The image detecting and tracking method of claim 10 wherein said disparity image is generated using the census algorithm.

12. The image tracking and detecting method of claim 10, wherein said step of locating a target area uses a connected components grouping analysis.

13. The image tracking and detecting method of claim 12, wherein said image data is received in the form of video frames and said connected components grouping analysis is performed for each set of video frames received from said cameras.

14. The image tracking and detecting method of claim 10, wherein said classification employs a Gaussian prior probability model.

15. The image tracking and detecting method of claim 10, wherein said displayed facial patterns are distorted relative to the originally detected image.

21. The image detecting and tracking method of claim 10, wherein said disparity image represents depth information.

16. An identification system, comprising:

at least two image sensing devices, each of said image sensing devices producing image data;

a first tracking module which receives the image data from each of said image sensing devices, locates and tracks a target area in the sensed image and provides a range identity description;

a second tracking module which classifies said target area through color segmentation and outputs a color identity description;

a third tracking module which distinguishes individual features located within said classified target area and outputs a face identity description; and

a classification module which receives said range identity description, said color identity description and said face identity description, and estimates an identity of a person whose image is contained within said target area.

17. The identification classification system of claim 16, wherein said range identity description is a height of the target area and is estimated from a generated disparity image.

18. The identification classification system of claim 16, wherein said color identity description is a skin color and hair color designation.

22. The identification classification system of claim 17, wherein said disparity image is generated from said image data from each of said image sensing devices and represents depth information.