System for Panoramic Image Processing

• US 20090058988A1
• Filed: 03/14/2008
• Published: 03/05/2009
• Est. Priority Date: 03/16/2007
• Status: Active Grant

• Current Assignee: L3 Technologies, Inc. (L3Harris Technologies, Inc.)
• Original Assignee: Kollmorgen Corporation (Regal Rexnord Corp.)
• Inventors: Rose, Michael, LaValley, Daniel Lawrence, Strzempko, Thaddeus J., Chamberlain, Jesse David
• Granted Patent: US 8,106,936 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

A system for low-latency, high-resolution, continuous motion panoramic video imaging includes a plurality of high-resolution video cameras, each video camera generating about at least 500 kilopixel near-real time video. The cameras can be supported for positioning the plurality of cameras at predetermined angular locations to generate a full 360 degree field of view. The system can also include an image processor for processing video image signals in parallel and providing panoramic images. In one embodiment, the system can include a display to provide seamless panoramic images.

Claims

1. A low-latency, high-resolution, continuous motion panoramic video imaging system, comprising:

a plurality of high-resolution video cameras, each video camera generating at least 500 kilopixel near-real time video camera image signals representative of images in the field of view of the respective camera;

a support for positioning the plurality of cameras at predetermined angular locations around the support to enable the plurality of cameras to together generate video camera image signals encompassing a full 360 degree field of view around the plurality of cameras;

a communication link between the plurality of cameras and an image processor for directing the video camera image signals from the plurality of cameras to the image processor;

the image processor receiving the video camera image signals from the plurality of video cameras and processing the video camera image signals together in parallel to generate video signals representative of panoramic images around the plurality of cameras; and

a communication link between the image processor and a display for communicating the video signals to the display, with the display showing panoramic images in the field of view around the plurality of cameras in near-real time.

2. A video imaging system as set forth in claim 1, wherein the image processor includes field programmable gate arrays for parallel processing of the video camera image signals.

3. A video imaging system as set forth in claim 1, further comprising at least one sensor operable in the non-visual spectrum to gathering non-visual data in the field of vision of the plurality of cameras, and a communication link for communicating the non-visual data from the at least one sensor to the image processor, with the image processor associating the non-visual data with the panoramic images generated from the video camera image signals gathered at the same time as the non-visual data.

4. A video imaging system as set forth in claim 3, wherein the at least one sensor is an infra-red sensor.

5. A video imaging system as set forth in claim 3, further comprising a global positioning sensor providing global positioning data to the image processor, and the image processor associating the global positioning data with the panoramic images generated from the video camera image signals gathered at the same time as the non-visual data.

6. A video imaging system as set forth in claim 1, wherein the image processor provides motion compensation algorithm for stabilization of the panoramic images.

7. A video imaging system as set forth in claim 6, wherein the support is carried on a mobile platform subject to movement and the motion compensation algorithm modifies the video signals to adjust for such movement.

8. A video imaging system as set forth in claim 7, wherein the motion compensation circuitry further comprises stabilization circuitry.

9. A video imaging system as set forth in claim 7, further comprising a motion sensor on the support, wherein the sensor generates a signal representative of relative movement of the support and a communication link between the sensor and the motion compensation circuitry.

10. A video imaging system as set forth in claim 7, wherein the motion compensation utilizes a predetermined generally fixed object in the field of vision of one of the plurality of video cameras to adjust the video camera image signals from the others of the plurality of video cameras relative to the video camera image signals from said one of the plurality of cameras.

11. A video imaging system as set forth in claim 1, further comprising detection circuitry for detecting movement of an object in the field of vision of at least one of the plurality of cameras.

12. A video imaging systems as set forth in claim 1, further comprising an operator controlled device for identifying an area of interest in the panoramic images and controlling the image processor to provide additional image signal information to display a magnified view of the area of interest.

13. A video imaging system as set forth in claim 1, wherein the image processor is positioned proximate to the plurality of video cameras.

14. A video imaging system as set forth in claim 13, further comprising a system center for communication with image processors of a plurality of video imaging systems and for controlling operation of the plurality of video imaging systems from a location remote from the video imaging systems.

15. The imaging system as set forth in claim 1, wherein the video signals representative of panoramic images generated by the image processor relate to seamless panoramic images.

16. The imaging system as set forth in claim 1, wherein the plurality of high-resolution video cameras are mounted in a housing configured to arrange the high-resolution video cameras to provide the 360 degree field of vision and to protect the camera from the environment.

17. The imaging system as set forth in claim 1, wherein the plurality of high-resolution video cameras are mounted to a quadrant of the mobile platform.

18. The imaging system as set forth in claim 1, wherein the plurality of high-resolution video cameras are separately mounted at spaced locations around the support in separate housings, one for each camera for protecting the cameras from the environment.