Multispectral imaging system with spatial resolution enhancement

• US 20020012071A1
• Filed: 04/19/2001
• Published: 01/31/2002
• Est. Priority Date: 04/21/2000
• Status: Active Grant

• Current Assignee: KEYW Corporation
• Original Assignee: Flight Landata, Inc. (The Keyw Corporation)
• Inventors: Sun, Xiuhong
• Granted Patent: US 7,019,777 B2
• Time in Patent Office: Days
• US Class Current: 348/578
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A spatial resolution enhancement and dynamic range extension for a Computerized Airborne Multicamera Imaging System (CAMIS). CAMIS is a multispectral imaging system for diverse manned and unmanned aerial vehicles to fly along flexible paths and altitudes for a wide variety of applications. CAMIS comprises four spectral bands of progressive scan CCD video cameras with 782 x 576 square pixels each, giving a total of 1.82 million effective pixels. These cameras are synchronized and aligned in parallel with sub-pixel-accurate spatial offsets over a common field of view. A software procedure interpolates the original four-band 782×576 captures into 1564×1152 ones using a bi-linear algorithm, and then performs geometric correction and band-to-band pixel registration. The result is a more precisely registered, spatial resolution enhanced multispectral image, sized 1540(H) ×1140(V) ×4(Bytes). The CAMIS CCD cameras include a controllable electronic shutter, which permits the system to acquire a desirable range of signals by a computed exposure, and then bracket it with two additional up/down-stepped exposures into computer memory. The integrated data set of the multiple stepped exposures results in effectively extending the dynamic range of the measurement.

Claims

1. An imaging system having spatial resolution enhancement comprising:

means for providing multispectral bands of images;

a computer connected to said multispectral bands of images for receiving said multispectral bands of images;

means within said computer for resampling-up said multispectral bands of spectral;

means for storing said resampled-up multispectral bands of images; and

means for performing multispectral band-to-band pixel registration of said resampled-up images.

2. The imaging system as recited in claim 1 wherein said computer comprises a memory for storing said resampled-up, registered images.

3. The imaging system of claim 1 wherein said means for producing multispectral bands of images comprises a plurality of cameras.

4. The imaging system of claim 3 wherein each of said plurality of cameras comprises an interline transfer, black and white, progressive scan, CCD video cameras.

5. The imaging system as recited in claim 3 wherein each of said plurality of cameras comprises 782×

576 square pixels.

6. The imaging system as recited in claim 3 wherein said plurality of cameras are optically aligned with a fractional pixel offset to each other.

7. The imaging system as recited in claim 3 wherein each ID of said plurality of cameras comprises a narrow band interference filter in front of the lens of each of said plurality of cameras.

8. The imaging system as recited in claim 7 wherein said narrow band interference filter in front of each of said plurality of cameras comprises user selectable spectral bands within a spectral range covering blue, green, red and near infrared.

9. The imaging system as recited in claim 1 wherein said means for resampling-up said multispectral bands of images comprises a routine for performing a neighbor average interpolation.

10. The imaging system as recited in claim 1 wherein said computer comprises means for acquiring a bracket of computer controlled multiple exposures of said multispectral bands of images for extending dynamic range.

11. An imaging system having spatial resolution enhancement comprising:

a sensor head including a plurality of cameras, each of said cameras having an interchangeable filter producing multispectral bands of images;

a computer for receiving said multispectral bands of images from said plurality of cameras;

said computer comprises means for resampling-up said multispectral bands of images; and

means for performing multispectral band-to-band pixel registration of said resampled-up images.

12. The imaging system as recited in claim 11 wherein said computer comprises means for storing said resampled-up, registered images.

13. The imaging system as recited in claim 11 wherein said cameras are optically aligned with a fractional pixel offset to each other.

14. The imaging system as recited in claim 11 wherein said means for resampling-up said multispectral bands of images comprises a routine for performing a neighbor average interpolation.

15. The imaging system as recited in claim 11 wherein said sensor head comprises progressive scan, interline transfer, CCD video cameras.

16. The imaging system as recited in claim 11 wherein each of said plurality of cameras comprises 782 x 576 square pixels.

17. The imaging system as recited in claim 11 wherein said narrow band interference filter in front of the lens of each of said cameras comprises user selectable spectral bands within a spectral range for blue, green, red, and near infrared.

18. The imaging system as recited in claim 11 wherein said means for resampling-up said multispectral bands of images comprises a routine for performing a neighbor average interpolation.

19. The imaging system as recited in claim 11 wherein said computer comprises means for acquiring a bracket of computer controlled multiple exposures of said multispectral bands of images for extending dynamic range.

20. A method of providing an imaging system with spatial resolution enhancement comprising the steps of:

providing means for generating multispectral bands of images;

digitizing said multispectral bands of images;

performing a resampling-up operation on said multispectral bands of images in a computer connected to said means for generating multispectral bands of images; and

performing band-to-band pixel registration of said bands of images.

21. The method as recited in claim 20 wherein the method comprises the step of storing said resampled-up, registered images.

22. The method as recited in claim 20 wherein said step of generating multispectral bands of images comprises the step of using a plurality of interline transfer, black and white, progressive scan, CCD video cameras.

23. The method as recited in claim 22 wherein said step of using a plurality of cameras comprises the step of optically aligning said cameras with a fractional pixel offset to each other.

24. The method as recited in claim 22 wherein said step of using a plurality of cameras comprises the step of providing each of said cameras with a narrow band interference filter in front of the lens of each of said cameras.

25. The method as recited in claim 24 wherein said step of providing each of said cameras with a narrow band interference filter comprises the step of said narrow band interference filter having user selectable spectral bands within a spectral range covering blue, green, red and near infrared.

26. The method as recited in claim 22 wherein said step of using a plurality of video cameras comprises the step of providing each of said plurality of cameras with 782×

576 square pixels.

27. The method as recited in claim 20 wherein said step of performing a resampling-up operation on said multispectral bands of images comprises the step of performing a neighbor average interpolation routine.

28. The method as recited in claim 20 wherein said step of performing said resampling-up operation comprises the steps of:

redistributing the sensed data of each of said bands of images by filling odd columns with existing data and even rows/columns with zeros;

calculating an average value of the pixel data at every two adjacent odd columns;

copying said average value into an even column between said adjacent odd columns;

redistributing the sensed data of each of said bands of images by filling odd rows with existing pixel data and even rows with zeros;

calculating an average value of the pixel data at every two adjacent rows; and

copying said average value into an even row between said adjacent odd rows.

29. The method as recited in claim 20 wherein said step of performing band-to-band pixel registration of said bands of images comprises the step of cropping edges of the bands to produce a 1532×

1150×

4 bands format.

30. The method as recited in claim 20 wherein said method comprises the step of acquiring a bracket of computer controlled multiple exposures of said multispectral bands of images for extending dynamic range.

31. A method of providing an imaging system with spatial resolution enhancement comprising the steps of:

providing a sensor head including a plurality of cameras, each of said cameras having a narrow band interference filter producing multispectral bands of images;

digitizing said multispectral bands of images from said plurality of cameras;

storing said digitized multispectral bands of images;

performing a resampling-up operation on said bands of images; and

performing band-to-band pixel registrations of said bands of images.

32. The method as recited in claim 31 wherein said method comprises the step of storing said resampled-up, registered images in a memory.

33. The method as recited in claim 31 wherein said method comprises the step of said cameras being optically aligned with a fractional pixel offset to each other.

34. The method as recited in claim 31 wherein said step of performing said resampling-up operation on said bands of images comprises the step of performing a neighbor average interpolation.

35. The method as recited in claim 31 wherein said step of providing a sensor head including a plurality of cameras comprises the step of providing progressive scan, interline transfer, CCD video cameras.

36. The method as recited in claim 31 wherein said step of providing a sensor head including a plurality of cameras each of said cameras having a narrow band interference filter comprises the step of said narrow band interference filter having user selectable spectral bands within a spectral range covering blue, green, red and near infrared.

37. The method as recited in claim 31 wherein said step of providing a sensor head including a plurality of cameras comprises the step of providing each of said plurality of cameras with 782×

576 square pixels.

38. The method as recited in claim 31 wherein said step of performing band-to-band pixel registration of said bands of images comprises the step of cropping edges of the bands to produce a 1532×

1150×

4 bands format.

39. The method as recited in claim 31 wherein said step of performing said resampling-up operation comprises the steps of:

redistributing the sensed data of each of said bands of images by filling odd columns with existing data and even rows/columns with zeros;

calculating an average value of the pixel data at every two adjacent odd columns;

copying said average value into an even column between said adjacent odd columns;

redistributing the sensed data of each of said bands of images by filling odd rows with existing pixel data and even rows with zeros;

calculating an average value of the pixel data at every two adjacent rows; and

copying said average value into an even row between said adjacent odd rows.

40. The method as recited in claim 31 wherein said method comprises the step of acquiring a bracket of computer controlled, multiple exposures of said multispectral bands of images for extending dynamic range.