Dual-swath imaging system

• US 8,462,209 B2
• Filed: 06/26/2009
• Issued: 06/11/2013
• Est. Priority Date: 06/26/2009
• Status: Active Grant

• Current Assignee: KEYW Corporation
• Original Assignee: KEYW Corporation
• Inventors: Sun, Xiuhong
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US12/492,458
• Publication Number:
  US 20100328499A1
• Time in Patent Office: 1,446 Days
• Field of Search: 348/36, 348/42, 348/144, 348/218.1
• US Class Current: 348/144
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

A portable, aerial, dual-swath photogrammetric imaging system comprising twin nadir pointing CCD cameras for simultaneously acquiring twin adjacent digital images for merging into a large panorama. A pair of lens shifters symmetrically shift twin images to the left and right sides of the focal points of two parallel lenses to extend imaging swathwidth. The twin adjacent images of the imaging system have a strip of narrow overlap at the center of the whole scene that are reserved for photogrammetric processing and stitching twin images into a seamless panorama. Each camera is connected to an embedded computer which controls imaging data acquisition, attaches GPS/IMU measurements, generates KML metadata files for its snapshots, and stores acquired images and metadata into removable SSDs. Direct geo-referenced panoramic digital stills are immediately registered on to Google™ Earth precisely. Its images can be further processed for advanced mapping, change detection and GIS applications.

Claims

1. A dual-swath imaging system comprising:

a pair of twin photogrammetric CCD cameras mounted on a remote sensing platform adjacent to each other;

a first of said twin photogrammetric CCD cameras comprises a first CCD imager and a nadir pointing first large format optical lens that is capable of imaging an extended panoramic rectangular area with a width twice as wide as that of said CCD imager of said twin cameras;

a first lens shift mount attached between a body of said first of said twin photogrammetric CCD cameras and said first large format optical lens for physically shifting said first large format optical lens to a predetermined point x_left with respect to said first CCD imager, to make said predetermined point x_left be on a focal point said first large format optical lens;

a second of said twin photogrammetric CCD cameras is symmetrically mounted on said platform adjacent to said first of said twin photogrammetric CCD cameras, said second of said twin photogrammetric CCD cameras comprises a second CCD imager and a nadir pointing second large format optical lens that are made the same as said corresponding first CCD imager and said first large format optical lens;

a second lens shift mount being mounted the same with the shift mirrored relative to the first lens shift mount and attached between a body of said second of said twin photogrammetric CCD cameras and said second large format optical lens for shifting the focal point of said second large format optical lens a mirrored reference point x_right with respect to said second CCD imager to form a symmetrical focal point x_right;

said first of said twin photogrammetric CCD cameras comprises said first CCD imager shifted to a predetermined right side location and orientation of the focal point of said first large format optical lens for capturing a left scene image on a left side of said twin CCD cameras;

said second of said twin photogrammetric CCD cameras comprises said second CCD imager shifted to a predetermined left side location and orientation of the focal point of said second large format optical lens for capturing a right scene image on a right side of said twin CCD cameras; and

a first principal axis of said first large format optical lens and a second principal axis of said second large format optical lens being aligned in parallel for obtaining a panorama from said left scene image and said right scene image.

2. The dual-swath imaging system as recited in claim 1 wherein said system comprises means for forming a nadir dual-swath panoramic view image with a rectangular imaging zone and uniform scale over a flat terrain.

3. The dual-swath imaging system as recited in claim 1 wherein:

said first lens shift mount shifts said first large format optical lens a predetermined distance for effective use of pixel resources of said first CCD imager for effectively extend a geospatial area coverage of the twin cameras; and

said second lens shift mount shifts said second optical lens a predetermined mirrored distance for effective use of pixel resources of said second CCD imager for achieving double swath or maximizing geospatial area coverage of the twin camera imaging system.

4. The dual-swath imaging system as recited in claim 1 wherein said system comprises means for generating a trigger pulse for simultaneously activating said first of said twin photogrammetric CCD cameras and said second of said twin photogrammetric CCD cameras to acquire said left scene image and said right scene image simultaneously.

5. A photogrammetric imaging system comprising:

a first CCD camera of symmetric twin photogrammetric CCD cameras mounted on a platform, said first CCD camera having a nadir pointing first large format optical lens;

a second CCD camera of said symmetric twin photogrammetric CCD cameras mounted on said platform symmetrically adjacent to said first CCD camera, said second CCD camera having a nadir pointing second large format optical lens;

said first CCD camera comprises a first CCD imager positioned on a right side of a first focal point of said first large format optical lens for capturing a left scene image on a left side of said twin CCD cameras;

said second CCD camera comprises a second CCD imager positioned on a left side of a second focal point of a second large format optical lens for capturing a right scene image on a right side of said twin CCD cameras;

a first principal axis of said first large format optical lens and a second principal axis of said second large format optical lens being aligned in parallel or obtaining panoramic images from said left scene images and said right scene images for photomapping applications;

means for direct geo-referencing said left scene image using a sensor model based on a collinearity condition associated with said first CCD camera and an integrated GPS/IMU measurement of a dual swath imaging system;

means for storing said geo-referenced left scene image in a geographic information system (GIS) compatible image format with geospatial metadata attached thereto;

means for direct geo-referencing said right scene image using a sensor model based on said collinearity condition associated with said second CCD camera and the integrated GPS/IMU measurement of said dual swath imaging system;

means for storing said geo-referenced right scene image in GIS compatible image format with geospatial metadata attached thereto;

means for performing a fast photogrammetric air-triangulation to estimate an average above ground level (AGL) range between a ground elevation associated with the middle of the panorama and a perspective center of said twin cameras;

means for seamlessly stitching said geo-referenced left scene image and said geo-referenced right scene image to form a dual-swath panorama;

means for providing every frame of said left scene and said right scene images with digital elevation models (DEMs) data from predetermined DEM databases; and

means for said symmetric twin photogrammetric cameras to collect a pair of focal point shifted images with integrated GPS/IMU measurements for post photogrammetric processing.

6. The symmetric twin camera imaging system as recited in claim 5 wherein said seamlessly stitching means comprises means for forming a registered panorama by purposefully narrowing the perspective center distance of two cameras and the width of the overlapping strip of said left scene image and said right scene image at adjacent edges.

7. The imaging system as recited in claim 5 wherein said system comprises means for generating a trigger pulse for activating said first CCD camera and said second CCD camera to acquire said left scene image and said right scene image simultaneously.

8. The imaging system as recited in claim 5 wherein a left scene processing means comprises a first data acquisition system coupled to a common central processing unit.

9. The imaging system as recited in claim 5 wherein a right scene processing means comprises a second data acquisition system coupled to a common central processing unit.

10. The imaging system as recited in claim 5 wherein said left scene image geo-referencing means comprises means for attaching real time GPS/IMU metadata measurements to each image captured and recorded with known interior orientation estimates including effective focal length, and focal point offset of said first of said twin CCD cameras.

11. The imaging system as recited in claim 5 wherein said right scene image geo-referencing means comprises means for attaching real time GPS/IMU metadata measurements to each image captured and recorded with known interior orientation estimates including effective focal length, and focal point offset of said second of said twin CCD cameras.

12. The imaging system as recited in claim 6 wherein said means for forming a panoramic image by processing the overlap of said left scene image and said right scene image comprises means for measuring the width of said overlapping field and using a width to height lookup table to estimate a real time AGL height.

13. The imaging system as recited in claim 5 wherein said system comprises a size and weight for portability and use in manned and unmanned aerial vehicles.

14. A method for providing a dual-swath imaging system comprising the steps of:

mounting a pair of twin photogrammetric CCD cameras on a remote sensing platform symmetrically adjacent to each other;

providing a first of said twin photogrammetric cameras with a first CCD imager and a nadir pointing first large format optical lens that is capable of imaging an extended panoramic rectangular area with a width twice as wide as that of said CCD imager of said twin cameras;

attaching a first lens shift mount between a body of said first of said twin photogrammetric CCD cameras and said first large format optical lens for physically shifting said first large format optical lens to have the focal point of said first large format optical lens shifted on to the predetermined reference point x_left;

symmetrically mounting a second of said twin photogrammetric CCD cameras on said platform adjacent to said first of said twin photogrammetric CCD cameras, said second of said twin photogrammetric CCD cameras comprises a second CCD imager and a nadir pointing second large format optical lens;

attaching a second lens shift mount between a body of said second of said twin photogrammetric CCD cameras and said second large format optical lens for shifting said second large format optical lens with a mirrored displacement that makes the focal point of said second large format optical lens on said reference point x_right;

capturing a left scene image on a left side of a panoramic imaging area of said twin CCD cameras by said first of said twin photogrammetric CCD cameras having a predetermined focal point offset;

capturing a right scene image on a right side of a panoramic imaging area of twin CCD cameras by said second of said twin photogrammetric CCD cameras having a predetermined focal point offset; and

aligning in parallel said first principal axis of said first large format optical lens and said second principal axis of said second large format optical lens to obtain seamlessly stitched panoramic images.

15. The method as recited in claim 14 wherein said method comprises the step of forming a dual-swath panoramic image by overlapping of said registered left scene image and right scene image within a predetermined narrow strip.

16. A method for providing a dual-swath imaging system comprising the steps of:

mounting a pair of twin photogrammetric COD cameras on a remote sensing platform symmetrically adjacent to each other;

providing a first of said twin photogrammetric cameras with a first CCD imager and a nadir pointing first large format optical lens that is capable of imaging an extended panoramic rectangular area with a width twice as wide as that of said CCD imager of said twin cameras;

attaching a first lens shift mount between a body of said first of said twin photogrammetric CCD cameras and said first large format optical lens for physically shifting said first large format optical lens to a predetermined point x_left to make said reference point x_left be on a principal axis of said first large format optical lens;

symmetrically mounting a second of said twin photogrammetric CCD cameras on said platform adjacent to said first of said twin photogrammetric CCD cameras, said second of said twin photogrammetric CCD cameras comprises a second CCD imager and a nadir pointing second large format optical lens;

attaching a second lens shift mount between a body of said second of said twin photogrammetric CCD cameras and said second large format optical lens for shifting said second large format optical lens to a mirrored reference point x_right to make said reference point x_right be on a focal point of said second large format optical lens;

capturing a left scene image on a left side of said panoramic imaging area of said twin CCD cameras by said first of said twin photogrammetric CCD cameras having a predetermined focal point offset;

capturing a right scene image on a right side of a panoramic imaging area of twin CCD cameras by said second of said twin photogrammetric CCD cameras having a predetermined focal point offset; and

aligning in parallel said first principal axis of said first large format optical lens and said second principal axis of said second large format optical lens to obtain seamlessly stitched panoramic image;

providing for the direct geo-referencing of the dual-swath left scene and right scene images onto an earth geographic information system (GIS);

providing every frame of left scene and right scene images with DEMs data from either a predetermined USGS data base or a center strip width lookup table derived AGL of a dual-swath panorama;

providing every frame of said left scene and right scene images with IMU/GPS exterior orientation data, said DEMs, AGL, and interior imaging system orientation data forming geo-spatial frame metadata files in KML format;

combining said frame metadata files with said DEMs data;

solving a left collinearity condition (x, y)=f(x_LEFT;

X, Y, Z) using required parameter inputs from said frame metadata files and said DEMs data to generate a KML file for said left scene image;

solving a right collinearity condition (x, y)=f(x_RIGHT;

X, Y, Z) using required parameter inputs from said frame metadata files and said DEMs data to generate a KML file for said right scene image;

converting said left scene and right scene images into a left image JPEG file and a right image JPEG file;

geo-tagging said KML file for left image with said left image JPEG file;

registering said geo-tagged left image on said GIS;

geo-tagging said KML file for right image with said right image JPEG file; and

registering said geo-tagged right image on said GIS superimposed on said left image to form a said overlapped and mosaicked panorama image.