Video surveillance system with omni-directional camera

• US 7,884,849 B2
• Filed: 09/26/2005
• Issued: 02/08/2011
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Yu, Li, Brewer, Paul C., Chosak, Andrew J., Haering, Niels, Zhang, Zhong, Venetianer, Peter L., Lipton, Alan J., Yin, Weihong
• Primary Examiner(s): Grant, II; Jerome
• Application Number:
  US11/234,377
• Publication Number:
  US 20070070190A1
• Time in Patent Office: 1,961 Days
• Field of Search: 340/517, 340/525, 250/206.02, 250/208.1, 250/214.DC, 348/211.12, 348/14.08, 348/211.99, 348/211.1, 348/211.4, 348/14.07, 348/14.11, 348/14.12, 348/36, 348/143, 348/211.2, 348/211.3, 381/92
• US Class Current: 348/143
• CPC Class Codes: G08B 13/19626 optical details, e.g. lenses, mirrors or multiple lenses
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system

Abstract

A method of operating a video surveillance system is provided. The video surveillance system including at least two sensing units. A first sensing unit having a substantially 360 degree field of view is used to detect an event of interest. Location information regarding a target is sent from the first sensing unit to at least one second sensing unit when an event of interest is detected by the first sensing unit.

Claims

1. A video surveillance system, comprising:

a first sensing unit having a substantially 360 degree field of view and adapted to detect an event in the field of view;

a communication medium connecting the first sensing unit and at least one second sensing unit, the at least one second sensing unit receiving commands from the first sensing unit to follow a target when an event of interest is detected by the first sensing unit wherein the first sensing unit comprises;

an omni-directional camera;

a video processing unit to receive video frames from the omni-directional camera; and

an event detection unit to receive target primitives from the video processing unit, to receive user rules to detect the event of interest based on the target primitives and the rules and to generate the commands for the second sensing unit.

2. The system of claim 1, wherein the first sensing unit comprises an omni-directional camera.

3. The system of claim 1, wherein the video processing unit further comprises a first module for automatically calibrating the omni-directional camera.

4. The system of claim 1, further comprising a camera placement module to determine a monitoring range of the first sensing unit based on user input regarding a configuration of the first sensing unit.

5. The system of claim 1, further comprising a rule module to receive user input defining the event of interest.

6. The system of claim 1, wherein the at least one second sensing unit comprises a PTZ camera.

7. The system of claim 6, wherein the at least one second sensing unit operates as an independent sensor when an event is not detected by the first sensing unit.

8. The system of claim 1, further comprising a target classification module for classifying the target by target type.

9. The system of claim 8, wherein the target classification module is adapted to determine a warped aspect ratio for the target and to classify the target based at least in part on the warped aspect ratio.

10. The system of claim 8, wherein the target classification module is adapted to classify the target based at least in part on a target size map.

11. The system of claim 10, wherein the target classification module is adapted to compare a size of the target the target size map.

12. The system of claim 8, wherein the target classification module is adapted to classify the target based at least in part on a comparison of a location of a target in an image to a region map, the region map specifying types of targets present in that region.

13. A method of operating a video surveillance system, the video surveillance system including at least two sensing units, the method comprising:

using a first sensing unit having a substantially 360 degree field of view detect an event in a field of view of the first sensing unit wherein the first sensing unit comprises an omni-directional camera;

sending location information regarding a target from the first sensing unit to at least one second sensing unit when an event is detected by the first sensing unit, automatically calibrating the omni-directional camera wherein the automatic calibration process comprises;

determining if a video frame from the omni-directional camera is valid;

performing edge detection to generate a binary edge image if the frame is valid;

performing circle detection based on the edge detection; and

creating a camera model for the omni-directional camera based on results of the edge detection and circle detection.

14. The method of claim 13, wherein the at least one second sensing unit comprises a PTZ camera.

15. The method of claim 13, further comprising determining if an auto calibration flag is set and performing the method of claim only when the flag is set.

16. The method of claim 13, further comprising determining a monitoring range of the first sensor based on user input regarding a configuration of the first sensing unit.

17. The method of claim 13, further comprising determining the location information based on a common reference frame.

18. A computer readable medium containing software implementing the method of claim 13.

19. The method of claim 13, wherein the location information is based on a common reference frame.

20. The method of claim 19, further comprising calibrating the omni-directional camera to the common reference frame.

21. The method of claim 20, wherein calibrating the omni-directional camera further comprises:

receiving user input indicating the camera location in the common reference frame, a location of a point in an image and a corresponding point in the common reference frame; and

calibrating the camera based at least in part on the input.

22. The method of claim 20, wherein calibrating the omni-directional camera further comprises:

receiving user input indicating four pairs of points including four image points in an image and four points in the common reference frame corresponding to the four image points, respectively; and

calibrating the camera based at least in part on the user input.

23. The method of claim 20, wherein calibrating the omni-directional camera further comprises:

dividing the image into a plurality of regions;

calculating calibration parameters for each region;

projecting the target to the common reference frame using the calibration parameters for that region which includes the target.

24. The method of claim 13, further comprising classifying the target by target type.

25. The method of claim 24, further comprising determining a warped aspect ratio for the target.

26. The method of claim 25, wherein classifying comprises classifying the target based at least in part on the warped aspect ratio.

27. The method of claim 26, wherein determining the warped aspect ration further comprises:

determining a contour of the target in an omni image;

determining a first distance from a point on the contour closest to a center of the omni image to the center of the omni image;

determining a second distance from a point of the contour farthest from the center of the omni image to the center of the omni image;

determining a largest angle between any two points on the contour; and

calculating a warped height and a warped width based at least in part on a camera model, the largest angle, and the first and second distances.

28. The method of claim 24, further comprising classifying the target based at least in part on a comparison of a location of the target to a region map, the region map specifying types of targets present in that region.

29. The method of claim 28, wherein classifying further comprises:

receiving user input defining regions in the region map and the target types present in the regions; and

selecting one of the specified target types as the target type.

30. The method of claim 24, further comprising classifying the target based at least in part on a target size map.

31. The method of claim 30, wherein the target size map is a human size map.

32. The method of claim 31, further comprising generating the human size map by:

selecting a pixel in an image;

transforming the pixel to a ground plane based on the camera model;

determining projection points for a head, left and right sides on the ground plane based on the transformed pixel;

transforming the projections points to the image using the camera model;

determining a size of a human based on distances between the projection points; and

storing the size information at the pixel location in the map.

33. The method of claim 31, further comprising;

determining a footprint of the target;

determining a reference value for a corresponding point in the target size map; and

classifying the target based on a comparison of the two values.

34. The method of claim 32, wherein determining the footprint comprises:

determining a centroid of the target; and

determining a point on a contour of the target closest to a center of the image;

projecting the point to a line between the center of the image and the centroid; and

using the projected point as the footprint.

35. The method of claim 33, further comprising determining the footprint based on a distance of the target from the omni-directional camera.

36. A video surveillance system, comprising:

a first sensing unit having a substantially 360 degree field of view and adapted to detect an event in the field of view;

a communication medium connecting the first sensing unit and at least one second sensing unit, the at least one second sensing unit receiving commands from the first sensing unit to follow a target when an event of interest is detected by the first sensing unit; and

a target classification module for classifying the target by target type.