Combined wide angle and narrow angle imaging system and method for surveillance and monitoring

• US 6,215,519 B1
• Filed: 03/04/1998
• Issued: 04/10/2001
• Est. Priority Date: 03/04/1998
• Status: Expired due to Term

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Gluckman, Joshua M., Swaminathan, Rahul, Nayar, Shree K.
• Primary Examiner(s): Britton, Howard
• Assistant Examiner(s): Diep, Nhon T
• Application Number:
  US09/034,745
• Time in Patent Office: 1,133 Days
• Field of Search: 348/143, 348/153, 348/154, 348/159, 348/169, 348/705, 348/148
• US Class Current: 348/159
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19626 optical details, e.g. lenses, mirrors or multiple lenses
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance and monitoring system and method for monitoring an area which includes a first imaging system having a wide-angle field of view approximately equal to or greater than the area. The system also includes one or more second imaging systems having adjustable view settings, each of one or more second imaging systems being positioned to view portions of the area and being capable of producing images of the portions with a resolution greater than the first imaging system. The system also includes one or more controls for controlling the adjustable view settings of the one or more second imaging systems so as to obtain high-resolution, magnified images of regions of interest within the area being monitored. The adjustable view settings may include pan, tilt, and zoom settings for adjusting the panning, tilting, and zooming of one or more second imaging systems with respect to the monitored area. In the method, a global image of the area being monitored is sensed with a first imaging system having a field of view approximately equal to or greater than the area, a region of interest is detected within the global image, and one or more detailed images of the region of interest with at least one of one or more second imaging systems are sensed. At least one of the detailed images may have a higher resolution than that of the global image.

Claims

1. The surveillance and monitoring system for monitoring an area, comprising:

(a) a first imaging system having a wide-angle field of view approximately equal to or greater than said area, said first imaging system providing a direction information for at least one portion of said area;

(b) one or more second imaging systems having adjustable view settings and receiving said direction information from said first imaging system, said one or more second imaging systems positioned to view said at least one portion of said area and being capable of producing images of said at least one portion using said direction information and predetermined depth information relating a depth of said at least one portion of said area at a fixed height from a predetermined base level; and

(c) one or more controls for controlling said adjustable view settings of said one or more imaging systems.

2. The surveillance and monitoring system of claim 1, wherein said area is substantially hemispherical and said first imaging system has a substantially hemispherical field of view.

3. The surveillance and monitoring system of claim 1, wherein said adjustable view settings include a pan setting for adjusting the pan angle of said one or more second imaging systems with respect to said area.

4. The surveillance and monitoring system of claim 1, wherein said adjustable view settings include a tilt setting for adjusting the tilt angle of said one or more second imaging systems with respect to said area.

5. The surveillance and monitoring system of claim 1, wherein said adjustable view settings include zoom settings for adjusting the zoom of said one or more second imaging systems with respect to said portions of said area.

6. The surveillance and monitoring system of claim 1, wherein said first imaging system is a catadioptric system comprising a reflecting surface positioned to reflect an image of said area and one or more image sensors optically coupled to said reflecting surface.

7. The surveillance and monitoring system of claim 6, wherein said reflecting surface comprises a truncated, substantially paraboloid-shaped mirror.

8. The surveillance and monitoring system of claim 6, wherein said reflecting surface comprises a truncated, substantially hyperboloidal-shaped mirror.

9. The surveillance and monitoring system of claim 6, wherein said reflecting surface comprises a substantially ellipsoidal-shaped mirror.

10. The surveillance and monitoring system of claim 6, wherein said reflecting surface comprises one or more curved mirrors.

11. The surveillance and monitoring system of claim 6, wherein said reflecting surface comprises one or more planar mirrors.

12. The surveillance and monitoring system of claim 1, wherein said first imaging system comprises imaging optics and one or more image sensors optically coupled to said imaging optics.

13. The surveillance and monitoring system of claim 12, wherein said imaging optics comprises a wide-angle lens.

14. The surveillance and monitoring system of claim 12, wherein said imaging optics comprises a fish-eye lens.

15. The surveillance and monitoring system of claim 1, wherein said first imaging system comprises a plurality of camera systems.

16. The surveillance and monitoring system of claim 1, further comprising a monitoring station including:

one or more displays, at least one display coupled to said first imaging system for displaying an image of said area and at least one display coupled to said one or more second imaging systems; and

an input arrangement selecting a region of interest displayed on said at least one display coupled to said first imaging system, said input arrangement operatively coupled to said one or more controls for controlling said adjustable view settings such that at least one of said one or more second imaging systems is positioned to view said region of interest.

17. The surveillance and monitoring system of claim 1, wherein said first imaging system provides an image signal representative of said area, and further comprising an image signal processing apparatus coupled to said first imaging system to convert said image signal into image signal data.

18. The surveillance and monitoring system of claim 17, wherein said image signal processing apparatus includes an arrangement which maps said image signal data into a Cartesian-coordinate system to produce a perspective image.

19. The surveillance and monitoring system of claim 17, wherein said image signal processing apparatus includes an arrangement which maps said image signal data into a cylindrical-coordinate system to produce a panoramic image.

20. The surveillance and monitoring system of claim 1, further comprising a motion detection arrangement coupled to said first imaging system which detects objects in motion in said area.

21. The surveillance and monitoring system of claim 20, further comprising a tracking arrangement which tracks one or more of said objects in motion in said area, said tracking arrangement having an input coupled to said motion detection arrangement and an output coupled to said one or more controls for controlling said adjustable view settings.

22. A surveillance and monitoring system for monitoring an area, comprising:

(a) a first imaging system having a wide-angle field of view approximately equal to or greater than said area;

(b) a motion detection arrangement coupled to said first imaging system which detects objects in motion in said at least one portion;

(c) a tracking arrangement coupled to said motion detection arrangement which tracks at least one of said detected objects in motion and generates direction information relating to said at least one of said detected objects; and

(d) at least one second imaging system receiving said direction information and being capable of producing images of said at least one of said detected objects using said direction information and predetermined depth information relating a depth of said at least one of said detected objects at a fixed height from a predetermined base level.

23. A method for monitoring an area, comprising the steps of:

(a) sensing a global image of said area with a first imaging system having a field of view approximately equal to or greater than said area;

(b) detecting a region of interest within said global image;

(c) sensing one or more detailed images of said region of interest with at least one second imaging system;

(d) providing, from said first imaging system, direction information for said region of interest; and

(e) producing, with at least one second imaging system, said one or more images of said region of interest using said direction information and predetermined depth information relating a depth of said at region of interest at a fixed height from a predetermined base level.

24. A method for monitoring an area, comprising the steps of:

(a) positioning a first imaging system to view said area, said first imaging system having a wide-angle field of view approximately equal to or greater than said area;

(b) positioning one or more second imaging systems having adjustable view settings to view at least one portion of said area, each of said one or more second imaging systems being capable of producing images of said at least one portion with a resolution greater than said first imaging system;

(c) sensing an image of said area with said first imaging system;

(d) detecting a region of interest within said sensed image;

(e) generating, from said first imaging system, direction information for said detected region;

(f) producing, with said one or more second imaging systems, said image using said direction information and predetermined depth information relating a depth of said at least one portion of said area at a fixed height from a predetermined base level; and

(g) controlling said adjustable view settings so as to view said region of interest with at least one of said one or more second imaging systems.

25. A method for monitoring one or more objects in motion in an area, comprising the steps of:

(a) positioning a first imaging system to view said area, said first imaging system having a wide-angle field of view approximately equal to or greater than said area;

(b) positioning one or more second imaging systems having adjustable view settings to view at least one portion of said area, each of said one or more second imaging systems being capable of producing images of said at least one portion with a resolution greater than said first imaging system;

(c) sensing images of said area with said first imaging system;

(d) detecting said one or more objects in motion from said sensed images;

(e) tracking one or more of said detected objects;

(f) generating, from said first imaging system, direction information for said one or more detected objects;

(g) producing, with said one or more second imaging systems, said image using said direction information and predetermined depth information relating a depth of said at least one portion of said area at a fixed height from a predetermined base level; and

(h) controlling said adjustable view settings so as to view said tracked objects with at least one of said one or more second imaging systems.

26. The method for monitoring one or more objects in motion in an area of claim 25, wherein said step of detecting comprises:

generating a series of image frames at predetermined time intervals from said sensed images;

filtering said series of image frames for noise;

calculating a moving average frame from a subset of said series of image frames comprising the set from the first image frame to the next-to-last image frame;

subtracting the filtered, last image frame of said series of image frames from said moving average frame to produce a difference image frame;

comparing each pixel of said difference image frame to a predetermined threshold value to produce a threshold image frame indicative of regions of motion in said area;

defining a first group of objects associated with said subset of said series of image frames;

associating a second group of objects with said regions of motion in said threshold image frame; and

generating a third group of objects comprising all unique objects in said first and second groups.

27. The method for monitoring one or more objects in motion in an area of claim 26, wherein the step of tracking comprises:

selecting a fourth group of objects to be tracked from said third group of objects on the basis of a predetermined criteria;

determining a focus point for each object in said fourth group of objects; and

applying a smoothing function to the focus point of each object.

28. The method for monitoring one or more objects in motion in an area of claim 25, wherein said step of controlling comprises mapping the coordinates of said tracked objects from the coordinate system of said first imaging system into the coordinate system of at least one of said one or more second imaging systems.