Robust stereo-driven video-based surveillance

• US 7,321,386 B2
• Filed: 07/31/2003
• Issued: 01/22/2008
• Est. Priority Date: 08/01/2002
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Ramesh, Visvanathan, Paragios, Nikolaos, Mittal, Anurag
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US10/632,002
• Publication Number:
  US 20040125207A1
• Time in Patent Office: 1,636 Days
• Field of Search: None
• US Class Current: 348/169
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method of tracking an object comprises providing a plurality of cameras, determining an image from each camera, and determining a common plane in the images. The method further comprises determining a parallax for scene points across the images, incorporating the parallax as a feature in a background model, and estimating a change in the scene using the background model. A camera can be a pan-tilt-zoom camera. A camera can be uncalibrated.

Claims

1. A method of tracking an object comprising:

providing a plurality of cameras;

determining an image from each camera;

determining a common plane in the images;

determining a parallax for scene points across the images;

incorporating the parallax as a feature in a background model;

determining a change in the background model incorporating the parallax; and

tracking the object according to the change, wherein the change in the background model incorporating the parallax corresponds to the object, wherein the tracking of the object according to the change further comprises,

obtaining a measure from the parallax that is invariant to motion of the object through the image, and

tracking the object relative to the invariant measure.

2. The method of claim 1, wherein at least one camera is a pan-tilt-zoom camera.

3. The method of claim 1, wherein at least one camera is uncalibrated.

4. The method of claim 2, further comprising:

providing a pan-tilt-zoom camera;

determining a mosaic for the pan-tilt-zoom camera from images captured from the pan-tilt-zoom camera; and

registering the mosaic and the images from the pan-tilt-zoom camera and the plurality of cameras according to the common plane in the image.

5. The method of claim 2, further comprising inter-frame registration of images captured from the pan-tilt-zoom camera.

6. The method of claim 1, wherein the background model comprises a feature.

7. The method of claim 6, wherein the feature is one of an intensity and an edge.

8. The method of claim 1, further comprising determining the background model by one of a mixture-of-Gaussians and a non-parametric kernel.

9. The method of claim 8, wherein an error associated with object detection and velocity is propagated to determine a maximum possible zoom at which an image of the desired region of the object may be acquired.

10. The method of claim 1, further comprising providing a control strategy for controlling at least one camera of the plurality of cameras to view a portion of a scene estimated to include a visible portion of the object subsequent to an occlusion of the object.

11. The method of claim 1, further comprising obtaining a relationship between observations from different cameras via a homography relationship for the common registered plane.

12. The method of claim 1, further comprising providing a control strategy for acquiring user defined relevant information for the object.

13. The method of claim 12, wherein providing the control strategy further comprises:

providing a probability density function for the object; and

providing a model for object motion.

14. The method of claim 12, wherein providing the control strategy further comprises providing a user specification.

15. The method of claim 14, further comprising controlling the cameras according to the user specification and the change in the background model.

16. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for tracking an object, the method steps comprising:

providing a plurality of cameras;

determining an image from each camera;

determining a common plane in the images;

determining a parallax for scene points across the images;

incorporating the parallax as a feature in a background model; and

determining a change in the background model incorporating the parallax; and

tracking the object according to the change, wherein the change in the background model incorporating the parallax corresponds to the object, wherein the tracking of the object according to the change further comprises,

obtaining a measure from the parallax that is invariant to motion of the object through the image, and

tracking the object relative to the invariant measure.

17. A system for tracking an object comprising:

a plurality of cameras capturing images including the object;

a memory device storing a plurality of instructions embodying an object tracker;

a processor in communication with the memory device and the plurality of cameras for receiving the images including the object and executing the object tracker to perform a method comprising;

determining a common plane in the images;

determining a parallax for scene points across the images;

incorporating the parallax as a feature in a background model;

determining a change in the background model incorporating the parallax; and

tracking the object according to the change, wherein the change in the background model incorporating the parallax corresponds to the object, wherein the tracking of the object according to the change further comprises,

obtaining a measure from the parallax that is invariant to motion of the object through the image, and

tracking the object relative to the invariant measure.

18. The system of claim 17, wherein at least one camera is a pan-tilt-zoom camera.

19. The system of claim 17, wherein at least one camera is uncalibrated.

20. The system of claim 18, wherein the processor further executes the object tracker to perform the method further comprising:

determining a mosaic for the pan-tilt-zoom camera from images captured from the pan-tilt-zoom camera; and

registering the mosaic and the images from the pan-tilt-zoom camera and the plurality of cameras according to the common plane in the image.