Method and system for performing surveillance

• US 7,385,626 B2
• Filed: 08/12/2003
• Issued: 06/10/2008
• Est. Priority Date: 10/21/2002
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Hanna, Keith J., Samarasakera, Supun, Burt, Peter, Kumar, Rakesh, Sawhney, Harpreet, Aggarwal, Manoj, Eledath, Jayan
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US10/638,984
• Publication Number:
  US 20040119819A1
• Time in Patent Office: 1,764 Days
• Field of Search: 348/143, 348/169, 348/170, 348/171, 348/172, 348/174, 348/139, 348/162, 348/135, 348/137, 348/580, 348/584, 348/598, 348/159, 348/157, 348/46, 348/39, 348/57, 348/47, 348/50, 348/164, 348/167, 348/168, 382/190, 382/103, 382/218, 382/294, 382/154, 382/282, 382/284, 382/107, 382/106, 382/255, 382/295, 382/296, 382/219, 340/541, 340/549, 340/565, 352/140
• US Class Current: 348/143
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method and system for detecting moving objects and controlling a surveillance system includes a processing module adapted to receive image information from at least one imaging sensor. The system performs motion detection analysis upon captured images and controls the camera in a specific manner upon detection of a moving object. The image processing uses the camera'"'"'s physical orientation to a surveillance area to facilitate mapping images captured by the camera to a reference map of the surveillance area. Using the camera orientation, a moving object'"'"'s position (e.g., latitude, longitude and altitude) within a scene can be derived.

Claims

1. A method for performing surveillance comprising:

monitoring an indicia of manual imaging sensor control;

switching to automatic imaging sensor control when said indicia indicates that manual imaging sensor control is not being used and switching to manual imaging sensor control when said indicia indicates that manual imaging sensor control is being used;

determining imaging sensor coordinates for at least one imaging sensor;

mapping the imaging sensor coordinates to image pixel coordinates; and

processing the image pixel coordinates to determine information regarding an actual detection of at least one object in motion within a scene that is captured by the at least one imaging sensor;

wherein said processing comprising generating parameters indicative of a first motion detection threshold, generating parameters indicative of a first consistent motion of the object over a predetermined number of captured images;

generating parameters indicative of isolation of the pixel coordinates in a particular image and generating parameters indicative of a secondary consistent motion of the object over the predetermined number of captured images.

2. The method of claim 1 wherein the manual imaging sensor control comprises manipulating a joystick to control an orientation of an imaging sensor and said indicia comprises movement of the joystick.

3. The method of claim 1 wherein the orientation of the imaging sensor comprises pan, tilt, and zoom.

4. The method of claim 1 further comprises detecting object motion within the scene.

5. The method of claim 4 wherein, upon detecting object motion during automatic imaging sensor control, switching to manual imaging sensor control.

6. A method of performing surveillance comprising:

pointing an imaging sensor at a location in a scene;

detecting at least one moving object within the scene;

determining imaging sensor coordinates for at least one imaging sensor;

mapping the imaging sensor coordinates to image pixel coordinates;

processing the image pixel coordinates to determine information regarding an actual detection of the moving object within a scene that is captured by the at least one imaging sensor;

wherein said processing comprising generating parameters indicative of a first motion detection threshold, generating parameters indicative of consistent motion of the object over a predetermined number of captured images;

generating parameters indicative of isolation of the pixel coordinates in a particular image and generating parameters indicative of a secondary consistent motion of the object over the predetermined number of captured images; and

controlling pan, tilt and zoom functions of the imaging sensor to automatically track the moving object.

7. The method of claim 6 wherein the pointing step further comprises:

manually directing the imaging sensor to image the location.

8. The method of claim 6 wherein the pointing step further comprises automatically scanning the imaging sensor across the scene to image the location;

and continue scanning the scene until the moving object is detected.

9. The method of claim 6 further comprising deriving a latitude, longitude and altitude of the moving object based upon the pan, tilt and zoom parameters of the imaging sensor.

10. The method of claim 6 further comprising:

predicting a sight line between the imaging sensor and the moving object; and

using the sight line to optimally image the moving object.

11. The method of claim 10 further comprising a plurality of imaging sensors and using the sight line to select an imaging sensor from the plurality of imaging sensors that provides an optimal view of the moving abject.

12. The method of claim 6 wherein the location is a region of the scene that is imaged by the imaging sensor, and further comprising establishing a sensitivity level for processing information within the region.

13. The method of claim 12 wherein the sensitivity level establishes a motion detection sensitivity level.

14. The method of claim 6 wherein the information produced by the imaging sensor is displayed in combination with a zone map that depicts the scene.

15. The method of claim 14 wherein coordinates of locations in the zone map are mapped to pan/tilt/zoom parameters using a look-up table.

16. The method of claim 14 further comprising combining information from a plurality of imaging sensors with the zone map.

17. The method of claim 9 wherein a plurality of imaging sensors image the scene and the latitude, longitude and altitude of the moving object are used to handoff imaging of one imaging sensor to another imaging sensor.

18. A surveillance system comprising:

an imaging sensor control module for pointing an imaging sensor at a location in a scene;

an image processor for detecting a moving object within the scene, determining imaging sensor coordinates for the imaging sensor and mapping the imaging sensor coordinates to image pixel coordinates.

said image processor comprise a plurality of object detection filters to determine an actual detection of the moving object, wherein a first filter of the plurality of filters generates parameters indicative of a first motion detection threshold, a second filter of the plurality of filters generates parameters indicative of consistent motion of the object over a predetermined number of captured images, a third filter of the plurality of filters generate parameters indicative of the isolation of the pixel coordinates in a particular image, and a fourth filter of the plurality filters generates parameters indicative of a secondary consistent motion of the object over a predetermined number of captured images; and

a controller for controlling pan, tilt and zoom functions of the imaging sensor to automatically track the moving object.

19. The surveillance system of claim 18 wherein the image processor derives a latitude, longitude and altitude of the moving object based upon pan, tilt and zoom parameters of the imaging sensor.

20. The surveillance system of claim 18 wherein the image processor predicts a sight line between the imaging sensor and the moving object and uses the sight line to enable the imaging sensor control module to point the imagines sensor to optimally image the moving object.

21. The surveillance system of claim 20 further comprising a plurality of imaging sensors and using the sight line to select an imaging sensor from the plurality of imaging sensors that provides an optimal view of the moving object.

22. The surveillance system of claim 18 wherein the location is a region of the scene that is imaged by the imaging sensor, and the image processor establishes a sensitivity level for processing information within the region.

23. The surveillance system of claim 22 wherein the sensitivity level establishes a motion detection sensitivity level.

24. The surveillance system of claim 18 wherein the information produced by the imaging sensor is displayed in combination with a zone map that depicts the scene.

25. The surveillance system of claim 24 wherein coordinates of locations in the zone map are mapped to pan/tilt/zoom parameters using a look-up table.

26. The surveillance system of claim 24 further comprising combining information from a plurality of imaging sensors with the zone map.

27. The surveillance system of claim 18 wherein a plurality of imaging sensors image the scene and the latitude, longitude and altitude of the moving object are used to handoff imaging of one imaging sensor to another imaging sensor.