Multi-bank TDI approach for high-sensitivity scanners

• US 20100046853A1
• Filed: 08/22/2008
• Published: 02/25/2010
• Est. Priority Date: 08/23/2007
• Status: Active Grant

• Current Assignee: Lockheed Martin Missiles & Fire Control (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Missiles & Fire Control (Martin Marietta Corporation)
• Inventors: Goodnough, Mark A., Tener, Gene D.
• Granted Patent: US 8,463,078 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/711 Time delay and integration [TDI] registers; TDI shift registers

Abstract

An apparatus and method for acquiring image data from a scanned, multi-bank time-delay and integrate (TDI) focal plane array (FPA) detector. Specifically a method and apparatus for warping and combining sequentially-acquired image data of a scene portion from multiple TDI detector banks into a single image having improved image quality, thereby providing improved FPA sensitivity. Also, a method and apparatus for enabling sensitivity and areal rate trade-offs in a multi-bank, scanning TDI FPA based on the number of TDI banks being used for sequential imaging of the same scene portion.

Claims

1. A method of capturing image data from a scene using a scanning focal plane array (FPA) device that contains at least two banks of Time-Delay and Integration (TDI) detectors, the method comprising:

imaging a first portion of the scene with the first TDI bank to create first image data;

imaging a second portion of the scene with the second TDI bank to create second image data;

warping the second image data to a frame of reference of the first image data; and

generating composite image data by combining the first image data and the warped second image data.

2. The method of claim 1, said imaging the portion of the scene with the second TDI bank comprising filtering incoming electro-optical radiation before it is detected by said second TDI bank.

3. The method of claim 1, further comprising increasing imaging sensitivity of the FPA and reducing imaging areal rate of the FPA by selecting the second portion of the scene to be the same as the first portion of the scene and performing the imaging a first portion and imaging a second portion steps sequentially during a scanning pass.

4. The method of claim 1, said warping comprising carrying out a projective transform on said second image data to change a frame of reference of the second image data into the frame of reference of the first image data.

5. The method of claim 1, said warping comprising carrying out an affine transform on said second image data to remove the effects of uniform distortions present in the second image data relative to the first image data.

6. The method of claim 1, said warping comprising removing the effects of optics distortion from said second image data.

7. The method of claim 1, said warping comprising mapping pixel coordinates from the second image data to pixels in the first image data.

8. The method of claim 1, further comprising generating an average image from overlapping portions of the first and second image data.

9. The method of claim 2, said filtering comprising band-pass filtering a predetermined spectral band of incoming electro-optical radiation.

10. The method of claim 2, said filtering comprising polarization filtering a predetermined polarization of incoming electro-optical radiation.

11. The method of claim 7, said mapping pixel coordinates including using an interpolation function to produce an effective pixel value for fractional pixel coordinates.

12. The method of claim 1, said combining filtering comprising unsharp mask filtering to de-focus said second image data, where the first and second portions of the scene are the same portion.

13. The method of claim 1, said combining the first image data and the second image data comprising temporal differencing the first image data and the second image data to detect moving objects, where the first and second portions of the scene are the same portion.

14. The method of claim 1, further comprising performing correlation for a scene region that overlaps between the first and second TDI bank in order to resample and integrate said region.

15. The method of claim 1, further comprising:

imaging the first portion of the scene with a third TDI bank to create third image data;

imaging the second portion of the scene with a fourth TDI bank to create fourth image data;

warping the third image data to the frame of reference of the first image data;

warping the fourth image data to the frame of reference of the second image data;

generating first composite image data by combining first and warped third image data;

generating second composite image data by combining second and warped fourth image data;

warping the second composite image data to the frame of reference of the first composite image data; and

combining first composite image data and warped second composite image data into third composite image data.

16. The method of claim 1, further comprising

imaging a third portion of the scene with a third TDI bank to create third image data;

imaging a fourth portion of the scene with a fourth TDI bank to create fourth image data;

warping the third and fourth image data to a frame of reference of the first image data;

wherein generating composite image data further comprises and combining the warped third and warped fourth image data with the first and warped second image data.

17. The method of claim 16, wherein the third portion of the scene is the same as the first portion of the scene and the fourth portion of the scene is the same as the second portion of the scene.

19. The method of claim 16, further comprising increasing imaging sensitivity and reducing imaging areal rate by selecting the second, third, and fourth portions of the scene to be the same as the first portion of the scene.

18. A method of adaptively setting areal rate and sensitivity levels for a scanned multi-bank TDI imaging detector, the method comprising adjusting areal rate and adjusting sensitivity by:

changing the time delay of a first TDI bank in the scanned multi-bank TDI detector such that it no longer images the same portion of a scene as a second TDI bank in the multi-bank TDI detector during a scanning pass, thereby increasing areal rate and reducing sensitivity;

or

changing the time delay of the first TDI bank such that it images the same portion of the scene as the second TDI bank during a scanning pass, thereby decreasing areal rate and increasing senstivity,

wherein increases in sensitivity for a portion of a scene are directly proportional to the number of TDI banks simultaneously imaging the same portion of a scene and wherein increases in areal rate for the detector are inversely proportion to the number of TDI banks simultaneously imaging the same portion of a scene.

20. The method of claim 18, further comprising changing the time delay of a third TDI bank in concert with the time delay of the first TDI bank such that:

when the time delay of the first TDI bank is changed to sequentially image the same portion of the scene as the second TDI bank during the scanning pass, the time delay of the third TDI bank is also changed to sequentially image that portion of the scene during the scanning pass, and

when the time delay of the first TDI bank is changed to image said different portion of a scene than said second TDI bank, the time delay of the third TDI bank is changed to image a third portion of the scene.

21. An apparatus for capturing image data from a scene using a multi-bank scanning Time-Delay and Integration (TDI) focal plane array (FPA) device that contains at least two banks of TDI detectors, the apparatus comprising:

a first read-out integrated circuit (ROIC) that integrates and reads out the contents the first TDI bank to create first image data, wherein first image data corresponds to a first scene portion;

a second ROIC that integrates and reads out the contents of the second TDI bank to create second image data, wherein second image data corresponds to a second scene portion; and

an image data processor that produces composite image data, where the image data processor includes;

an image warper that warps the second image data to a frame of reference of the first image data; and

an image combiner that combines the first image data and the warped second image data into composite image data.

22. The apparatus of claim 21, further comprising at least one filter disposed on at least one TDI bank.

23. The apparatus of claim 21, further comprising a sensitivity controller that increases imaging sensitivity of the FPA while decreasing imaging areal rate of the FPA by selecting the second scene portion to be the same as the first scene portion and coordinating the first and second TDI banks to capture imaging data sequentially during a scanning pass.

24. The apparatus of claim 21, the image warper including a projective transform section that changes a frame of reference of the second image data into the frame of reference of the first image data by carrying out a projective transform on said second image data.

25. The apparatus of claim 21, said image warper including an affine transform section that removes the effects of uniform distortions present in the second image data relative to the first image data by carrying out an affine transform on said second image data.

26. The apparatus of claim 21, said image warper including an optical distortion removal section that removes the effects of optics distortion from said second image data.

27. The apparatus of claim 21, said image warper including a mapping unit, that maps pixel coordinates from the second image data to pixels in the first image data.

28. The apparatus of claim 21, said image combiner generating an average image of overlapping portions of the first and second image data.

29. The apparatus of claim 21, said filter including a spectral filter that filters a predetermined spectral band of incoming electro-optical radiation.

30. The apparatus of claim 21, said filter including a polarization filter that filters a predetermined polarization range of incoming electro-optical radiation.

31. The apparatus of claim 27, said image warper including an interpolation engine that produces an effective pixel value for fractional pixel coordinates.

32. The apparatus of claim 21, said image combiner including an unsharp masking engine that performs unsharp mask filtering of the first image data using the second image data, where the first and second portions of the scene are the same portion.

33. The apparatus of claim 21, said image combiner including a time difference engine that performs temporal differencing the first image data and the second image data to detect moving objects.

34. The apparatus of claim 21, said image warper including a scene correlator that correlates a scene region from said first and second image data that overlaps between the first and second TDI bank in order to resample and integrate said region.

35. The apparatus of claim 21, further comprising:

a third TDI bank operably coupled to a third ROIC, said third TDI bank imaging the first portion of the scene to create third image data;

a fourth TDI bank operably coupled to a fourth ROIC, said fourth TDI bank imaging the second portion of the scene to create fourth image data;

the image warper warping the third image data to the frame of reference of the first image data and warping the fourth image data to a frame of reference of the second image data;

the image combiner combining the first and warped third image data into first composite image data and combining the second and warped fourth image data into second composite image data;

the image warper warping the second composite image data to a frame of reference of the first composite image data; and

the image combiner combining said first and second composite image data into third composite image data.

36. The apparatus of claim 21, further comprising

a third TDI bank operably coupled to a third ROIC, said third TDI bank imaging a third portion of the scene to create third image data;

a fourth TDI bank operably coupled to a fourth ROIC, said fourth TDI bank imaging a fourth portion of the scene to create fourth image data;

the image warper warping the third and fourth image data to a frame of reference of the first image data; and

the image combiner combining the first, warped second, warped third, and warped fourth image data into composite image data.

37. The apparatus of claim 35, further comprising a sensitivity controller that increases imaging sensitivity of the apparatus while decreasing imaging areal rate of the apparatus by configuring the third TDI bank to image, as the third portion of the scene, the first portion of the scene after said first portion is imaged by said first TDI bank in a scanning pass, and configuring the fourth TDI bank to image, as the fourth portion of the scene, the second portion of the scene after said second portion is imaged by said second TDI bank in a scanning pass.

39. The apparatus of claim 36, further comprising a sensitivity controller that increases imaging sensitivity of the apparatus while decreasing imaging areal rate of the apparatus by configuring the second, third, and fourth TDI banks such that the second, third, and fourth portions of the scene are the same as the first portion and such that the first, second, third, and fourth portions are captured sequentially in a scanning pass.

38. An apparatus that adjusts areal rate and sensitivity levels for a scanned multi-bank TDI focal plane array (FPA) detector by trading off increases in one for proportional decreases in the other, the apparatus comprising:

a sensitivity controller that increases or decreases the time delay associated with a first TDI bank in the detector such that;

when time delay is decreased, said first TDI bank images a different portion of a scene than a second TDI bank in the detector during a scanning pass, thereby increasing the areal rate and reducing the sensitivity of the detector, and

when time delay is increased said first TDI bank sequentially images the same portion of the scene as the second TDI bank during a scanning pass, thereby decreasing areal rate and increasing sensitivity;

where increases in sensitivity of the detector are directly proportional to the number of TDI banks on the detector that are sequentially imaging the same portion of the scene and increases in areal rate for the detector are directly proportional to the number of TDI banks on the detector imaging different portions of the scene.

40. The apparatus of claim 38, said second TDI bank being the second TDI bank arranged on the FPA in the scanning direction and said first TDI bank being the third TDI bank arranged on the FPA in the scanning direction.

41. The apparatus of claim 38, the first TDI bank having at least two read-out integrated circuits (ROICS), the first ROIC being associated with even pixel rows of the first TDI bank and the second ROIC being associated with odd pixel rows of the first TDI bank.

42. The Apparatus of claim 41, the first TDI bank having at least one high-speed output that combines, delays, and outputs data from the first and second ROICs.

43. The apparatus of claim 38, said sensitivity controller increasing or decreasing the time delay associated with a third TDI bank in the detector in concert with the time delay of the first TDI bank, such that:

when said controller changes the time delay of the first TDI bank to sequentially image the same portion of the scene as the second TDI bank during the scanning pass, said controller also changes the time delay of the third TDI bank to sequentially image that portion of the scene during the scanning pass, thereby further increasing the sensitivity of the detector, and

when said controller changes the time delay of the first TDI bank to image said different portion of a scene than said second TDI bank, said controller changes the time delay of the third TDI bank to image a third portion of the scene, thereby further increasing the areal rate of the detector.

44. A computer-readable medium having embodied thereon a program that, when executed by a computing device, causes the computing device to perform a method of capturing image data from a scene using a scanning focal plane array (FPA) device that contains at least two banks of Time-Delay and Integration (TDI) detectors, the method comprising:

imaging a first portion of the scene with the first TDI bank to create first image data;

imaging a second portion of the scene with the second TDI bank to create second image data;

warping the second image data to a frame of reference of the first image data; and

generating composite image data by combining the first image data and the warped second image data.

45. A computer-readable medium, having embodied thereon a program that, when executed by a computing device, causes the computing device to perform a method of adaptively setting areal rate and sensitivity levels for a scanned multi-bank TDI imaging detector, the method comprising adjusting areal rate and adjusting sensitivity by:

changing the time delay of a first TDI bank in the scanned multi-bank TDI detector such that it no longer images the same portion of a scene as a second TDI bank in the multi-bank TDI detector during a scanning pass, thereby increasing areal rate and reducing sensitivity;

or

changing the time delay of the first TDI bank such that it images the same portion of the scene as the second TDI bank during a scanning pass, thereby decreasing areal rate and increasing senstivity,

wherein increases in sensitivity for a portion of a scene are directly proportional to the number of TDI banks simultaneously imaging the same portion of a scene and wherein increases in areal rate for the detector are inversely proportion to the number of TDI banks simultaneously imaging the same portion of a scene.