IMAGING SYSTEMS AND METHODS FOR IMMERSIVE SURVEILLANCE

• US 20150271453A1
• Filed: 03/05/2015
• Published: 09/24/2015
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Chuang, Daniel B., Candell, Lawrence M., Ross, William D., Beattie, Mark E., Fang, Cindy Y., Ren, Bobby, Blanchard, Jonathan P., Long, Gary M. Jr., White, Lauren L., Panasyuk, Svetlana V., Bury, Mark
• Granted Patent: US 10,306,186 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19619 Details of casing
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Security guards at large facilities, such as airports, monitor multiple screens that display images from individual surveillance cameras dispersed throughout the facility. If a guard zooms with a particular camera, he may lose image resolution, along with perspective on the surrounding area. Embodiments of the inventive Imaging System for Immersive Surveillance (ISIS) solve these problems by combining multiple cameras in one device. When properly mounted, example ISIS systems offer up to 360-degree, 240-megapixel views on a single screen. (Other fields of view and resolutions are also possible.) Image-stitching software merges multiple video feeds into one scene. The system also allows operators to tag and follow targets, and can monitor restricted areas and sound an alert when intruders breach them.

Claims

1. A method of determining a model representing views of a scene from cameras in an array of cameras, each camera in the array of cameras having a field of view that overlaps with the field of view of another camera in the array of cameras, the method comprising:

(A) for each pair of overlapping fields of view;

(A1) selecting image features in a region common to the overlapping fields of view;

(A2) matching points corresponding to a subset of the image features in one field of view in the pair of overlapping fields of view to points corresponding the subset of the image features in the other field of view in the pair of overlapping fields of view to form a set of matched points;

(B) merging at least a subset of each set of the matched points to form a set of merged points; and

(C) estimating parameters associated with each field of view based on the set of merged points to form the model representing the views of the scene.

2. The method of claim 1 wherein (A) comprises identifying candidate image features using a random sample and consensus procedure.

3. The method of claim 1 wherein (A2) comprises calculating a homography that maps points from one field of view in the pair of overlapping fields of view to the other field of view in the pair of overlapping fields of view.

4. The method of claim 1 wherein (A2) comprises:

projecting points corresponding to the image features from one field of view in the pair of overlapping fields of view onto the other field of view in the pair of overlapping fields of view; and

for each projected point, determining whether the projected point lies within a predefined distance of a corresponding point in the other field of view.

5. The method of claim 4 further comprising:

selecting the subset of each set of matched points from among those projected points that lie within the predefined distance of the corresponding points.

6. The method of claim 1 wherein (A) further comprises:

(A3) repeating steps (A1) and (A2) for different image features in the region common to the overlapping fields of view.

7. The method of claim 1 further comprising:

(D) calculating a re-projection error associated with model representing the views of the scene; and

(E) weighting the parameters to reduce the re-projection error.

8. A method of compensating for imbalances in color and white levels in color images of respective portions of a wide-area scene, each color image comprising red, green, and blue color channels acquired by a respective camera in a camera array disposed to image the wide-area scene, the method comprising:

(A) normalizing values representing the red, green, and blue color channels to a reference value representing a response of the cameras in the camera array to white light;

(B) equalizing the values representing the red, green, and blue color channels to red, green, and blue equalization values, respectively;

(C) identifying high- and low-percentile values among each of the red, green, and blue color channels; and

(D) scaling each of the red, green, and blue color channels based on the high- and low-percentile values to provide compensated values representing the red, green, and blue color channels.

9. The method of claim 8 wherein (B) further comprises:

(B1) calculating red, green, and blue median values for the red, green, and blue color channels, respectively;

(B2) calculating red, green, and blue equalization coefficients by dividing the red, green, and blue median values, respectively, by the mean of the red, green, and blue median values; and

(B3) scaling the values representing the red, green, and blue color channels by the red, green, and blue equalization coefficients, respectively.

10. The method of claim 8, wherein (A) further comprises:

(A1) disposing the camera array inside a calibration hemisphere, the calibration hemisphere comprising;

a diffusive hemisphere,

a reflective hemisphere disposed outside the diffusive hemisphere, and

a plurality of light emitting diodes (LEDs), disposed between the diffusive hemisphere and the reflective hemisphere, to emit light that is reflected by the reflective hemisphere and transmitted by the diffusive hemisphere;

(A2) capturing a first image of the plurality of LEDs using the camera array when the plurality of LEDs is off;

(A3) capturing a second image of the plurality of LEDs using the camera array when the plurality of LEDs is on; and

(A4) generating the reference value based at least in part on the first image captured in (A2) and the second image captured in (A3).

11. An interface for a surveillance system that monitors a scene, the interface comprising:

a full-scene view configured to render a real-time panoramic image of the entire scene monitored by the surveillance system; and

a zoom view configured to render a close-up of a region of the panoramic view.

12. The interface of claim 11 wherein the full-scene view is further configured to enable a user to select a region of the scene for display in the zoom view.

13. The interface of claim 11 wherein the interface is further configured to enable at least one user to set at least one zone in the panoramic image to be monitored for activity.

14. The interface of claim 13 wherein the interface is further configured to alert the at least one user upon detection of activity in the at least one zone.

15. The interface of claim 13 wherein the interface is further configured to populate an activity database with an indication of detected activity in the at least one zone.

16. The interface of claim 15 further comprising:

an activity view configured to display the indication of detected activity to the at least one user in a manner that indicates a time and a location of the detected activity.

17. The interface of claim 15 wherein the interface is further configured to display images of detected activity in at least one of the full-scene view and the zoom view.

18. The interface of claim 13 wherein the interface is further configured to track a target throughout the scene and to display an indication of the target'"'"'s location in at least one of the full-scene view and the zoom view.

19. The interface of claim 18 wherein the interface is further configured to enable a user to select the target.

20. The interface of claim 19 wherein the interface is further configured to display a pre-warped image in at least one of the full-scene view and the zoom view.