Imaging systems and methods for immersive surveillance

• US 9,007,432 B2
• Filed: 12/15/2011
• Issued: 04/14/2015
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Chuang, Daniel B., Candell, Lawrence M., Ross, William D., Beattie, Mark E., Fang, Cindy Y., Ren, Bobby, Blanchard, Jonathan P., Long, Gary M. Jr., White, Lauren L., Panasyuk, Svetlana V., Bury, Mark
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): LUO, KATE H
• Application Number:
  US13/327,416
• Publication Number:
  US 20120169842A1
• Time in Patent Office: 1,216 Days
• US Class Current: 348/39
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19619 Details of casing
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Security guards at large facilities, such as airports, monitor multiple screens that display images from individual surveillance cameras dispersed throughout the facility. If a guard zooms with a particular camera, he may lose image resolution, along with perspective on the surrounding area. Embodiments of the inventive Imaging System for Immersive Surveillance (ISIS) solve these problems by combining multiple cameras in one device. When properly mounted, example ISIS systems offer up to 360-degree, 240-megapixel views on a single screen. (Other fields of view and resolutions are also possible.) Image-stitching software merges multiple video feeds into one scene. The system also allows operators to tag and follow targets, and can monitor restricted areas and sound an alert when intruders breach them.

Claims

1. A system for monitoring a wide-area scene, the system comprising:

an array of first cameras, each first camera having a first field of view and configured to provide respective first imagery of a corresponding portion of the wide-area scene;

an array of second cameras, each second camera having a second field of view different than the first field of view and configured to provide respective second imagery of a corresponding portion of the wide-area scene;

a processor operably coupled to the array of first cameras and the array of second cameras and configured to decompose the first imagery and the second imagery into image tiles and to compress the image tiles at each of a plurality of resolutions;

a memory operably coupled to the processor and configured to store the compressed image tiles; and

a server operably coupled to the memory and configured to receive a request for an image of a particular portion of the wide-area scene, to determine a minimum resolution of the plurality of resolutions necessary to render the image, and to serve one or more image tiles at the minimum resolution in response to the request.

2. The system of claim 1 wherein each first camera comprises a first lens having a first focal length and wherein each second camera comprises a second lens having a second focal length different than the first focal length.

3. The system of claim 1 wherein each first field of view is a first angular field of view and wherein each second field of view is a second angular field of view.

4. The system of claim 3 wherein each first angular field of view is about 10 degrees to about 30 degrees and each second angular field of view is about 30 degrees to about 50 degrees.

5. The system of claim 1 wherein each first field of view overlaps with one or more other first fields of view and wherein each second field of view overlaps with one or more other second fields of view.

6. The system of claim 1 wherein at least one first field of view overlaps with one or more second fields of view.

7. The system of claim 1 wherein at least one of the first imagery and the second imagery comprises an annular portion of the wide-area scene.

8. The system of claim 1 further comprising:

an array of third cameras, each third camera having a respective third field of view different than the second field of view.

9. The system of claim 1, wherein the processor is further configured to form a spatially continuous image of substantially all of the wide-area scene from the first imagery and the second imagery.

10. The system of claim 9 wherein the processor is further configured to register the first imagery to the second imagery to form the spatially continuous image.

11. The system of claim 9 wherein the image processor is further configured to correct and balance color of the first imagery and the second imagery to form the spatially continuous image.

12. The system of claim 1,

wherein the processor is further configured to register the first imagery and the second imagery to a reference image of the wide-area scene, and further comprising:

a panoramic camera operably coupled to the processor and configured to provide the reference image.

13. The system of claim 1, wherein:

the first imagery and the second imagery, taken together, provide an image of substantially all of the wide-area scene, and

the image of substantially all of the wide-area scene has a spatial resolution of about 0.5 cm to about 50 cm.

14. A system for monitoring a wide-area scene, the system comprising:

an array of first cameras, each first camera having a first field of view and configured to provide respective first imagery of a corresponding portion of the wide-area scene;

an array of second cameras, each second camera having a second field of view different than the first field of view and configured to provide respective second imagery of a corresponding portion of the wide-area scene;

a housing to hold the array of first cameras and the array of second cameras;

a heat exchanger in thermal communication with at least a portion of the housing to remove heat from the housing;

two surfaces defining a channel to convey gas through the housing and across at least a portion of the heat exchanger so as to remove heat from the heat exchanger; and

a chamfer connecting the two surfaces defining the channel.

15. A method of acquiring an image of a wide-area scene, the method comprising:

(A) acquiring first imagery of a plurality of first portions of the wide-area scene at a first field of view;

(B) acquiring second imagery of a plurality of second portions of the wide-area scene at a second field of view different than the first field of view;

(C) forming the image of the wide-area scene from the first imagery and the second imagery;

(D) decomposing the first imagery and the second imagery into image tiles;

(E) compressing the image tiles at each of a plurality of resolutions;

(F) receiving a request for an image of a particular portion of the wide-area scene;

(G) determining a minimum resolution of the plurality of resolutions necessary to render the image; and

(H) serving one or more image tiles at the minimum resolution in response to the request for the image.

16. The method of claim 15 wherein each first field of view is a first angular field of view and wherein each second field of view is a second angular field of view.

17. The method of claim 16 wherein each first field of view is about 10 degrees to about 30 degrees and each second field of view is about 30 degrees to about 50 degrees.

18. The method of claim 15 wherein each first field of view overlaps with one or more other first fields of view and wherein each second field of view overlaps with one or more other second fields of view.

19. The method of claim 15 wherein at least one first field of view overlaps with one or more second fields of view.

20. The method of claim 15 wherein at least one of the first imagery and the second imagery comprises an annular portion of the wide-area scene.

21. The method of claim 15 wherein the first imagery and the second imagery, taken together, provide an image of substantially all of the wide-area scene.

22. The method of claim 15 further comprising:

acquiring third imagery of a plurality of third portions of the wide-area scene at a third field of view different than the second field of view; and

wherein (C) further comprises forming the image of the wide-area scene from the first, second, and third imagery.

23. The method of claim 15 wherein (C) further comprises:

forming a spatially continuous image of substantially all of the wide-area scene from the first imagery and the second imagery.

24. The method of claim 23 wherein (C) further comprises:

registering the first imagery to the second imagery to form the spatially continuous image.

25. The method of claim 17, wherein (C) further comprises registering the first imagery and the second imagery to a reference image of the wide-area scene.

26. The method of claim 15 wherein (C) further comprises:

correcting and balancing color of the first imagery and the second imagery.

27. The method of claim 15, wherein the image of the wide-area scene has a spatial resolution of about 0.5 cm to about 50 cm.

28. The method of claim 15 further comprising:

storing the compressed image tiles in an image format.

29. A surveillance system comprising:

an array of first cameras, each first camera having a first angular field of view and configured to provide respective first real-time imagery of a corresponding portion of the wide-area scene; and

an array of second cameras, each second camera having a second angular field of view different than the first angular field of view and configured to provide respective second real-time imagery of a corresponding portion of the wide-area scene;

a processor operably coupled to the array of first cameras and the array of second cameras and configured to decompose the first real-time imagery and the second real-time imagery into image tiles and to compress the image tiles at each of a plurality of resolutions;

a server operably coupled to the processor and configured to receive a request for an image of a particular portion of the wide-area scene and to serve one or more image tiles at a minimum resolution in the plurality of resolutions necessary to render the image in response to the request for the image of the particular portion of the wide-area scene; and

an interface communicatively coupled to the server and configured to render a real-time image of the wide-area scene represented by the one or more image tiles.