Method and system for enhancing short wave infrared images using super resolution (SR) and local area processing (LAP) techniques

• US 8,200,046 B2
• Filed: 04/11/2008
• Issued: 06/12/2012
• Est. Priority Date: 04/11/2008
• Status: Active Grant

• Current Assignee: DRS Network & Imaging Systems, LLC (Leonardo S.p.A.)
• Original Assignee: DRS RSTA Incorporated (Leonardo S.p.A.)
• Inventors: Cloud, Eugene L., Lee, Harry C., Manville, Richard A.
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US12/101,636
• Publication Number:
  US 20090257683A1
• Time in Patent Office: 1,523 Days
• Field of Search: 345/589, 348/297, 375/E7.011, 375/E7.182, 382/205, 382/254, 382/274, 382/276, 382/282, 382/284, 382/294, 382/299
• US Class Current: 382/299
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution

Abstract

A high resolution image of a scene or object is generated by collecting a plurality of images, enhancing the plurality of images to produce a sequence of enhanced images, registering the sequence of enhanced images, accumulating the intensities of the registered sequence of enhanced images to produce a composite image, and enhancing the composite image.

Claims

1. A method, comprising:

providing a processor;

collecting, using the processor, a plurality of low-intensity infrared images;

enhancing, using the processor, the plurality of low-intensity infrared images to produce a sequence of enhanced images;

registering, using the processor, the sequence of enhanced images;

accumulating, using the processor, the intensities of the registered sequence of enhanced images to produce a composite image;

enhancing, using the processor, the composite image; and

outputting, using the processor, the enhanced composite image,

wherein enhancing the plurality of low-intensity infrared images includes;

selecting a first subset of low-intensity infrared images from the plurality of low-intensity infrared images;

selecting a first low-intensity infrared image from the first subset of low-intensity infrared images;

determining a region of interest in the first low-intensity infrared image;

extracting a first window image from the first low-intensity infrared image which corresponds to the determined region of interest;

up-sampling and sharpening the extracted first window image;

selecting a second low-intensity infrared image from the first subset of low-intensity infrared images;

extracting a second window image based on the region of interest of the first low-intensity infrared image;

up-sampling and sharpening the extracted second window image;

generating a composite window based on the first and second extracted window images;

enhancing the generated composite window; and

combining the enhanced composite window with the selected first low-intensity infrared image.

2. The method of claim 1, wherein registering the sequence of enhanced images comprises:

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

3. The method of claim 1, wherein enhancing the composite image comprises:

reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

4. A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to:

collect a plurality of low-intensity infrared images;

enhance each of the plurality of low-intensity infrared images to produce a sequence of enhanced images;

register the sequence of enhanced images;

accumulate the intensities of the registered sequence of enhanced images to produce a composite image;

enhance the composite image; and

output the enhanced composite image,

wherein enhancing each of the plurality of low-intensity infrared images includes;

selecting a first subset of low-intensity infrared images from the plurality of low-intensity infrared images;

selecting a first low-intensity infrared image from the first subset of low-intensity infrared images;

determining a region of interest in the first low-intensity infrared image;

extracting a first window image from the first low-intensity infrared image which corresponds to the determined region of interest;

up-sampling and sharpening the extracted first window image;

selecting a second low-intensity infrared image from the first subset of low-intensity infrared images;

extracting a second window image based on the region of interest of the first low-intensity infrared image;

up-sampling and sharpening the extracted second window image;

generating a composite window based on the first and second extracted window images;

enhancing the generated composite window; and

combining the enhanced composite window with the selected first low-intensity infrared image.

5. The non-transitory computer-readable medium of claim 4, wherein registering the sequence of enhanced images comprises:

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

6. The non-transitory computer-readable medium of claim 4, wherein enhancing the composite image comprises:

reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

7. An apparatus, comprising:

an image detection unit; and

an image acquisition unit in communication with the image detection unit, the image acquisition unit comprising a processor and memory storing instructions that, when executed by the processor, cause the image acquisition unit to;

collect a plurality of low-intensity infrared images;

enhance each of the plurality of low-intensity infrared images to produce a sequence of enhanced images;

register the sequence of enhanced images;

accumulate the intensities of the registered sequence of enhanced images to produce a composite image;

enhance the composite image; and

output the enhanced composite image,

wherein enhancing each of the plurality of low-intensity infrared images includes;

selecting a first subset of low-intensity infrared images from the plurality of low-intensity infrared images;

selecting a first low-intensity infrared image from the first subset of low-intensity infrared images;

determining a region of interest in the first low-intensity infrared image;

extracting a first window image from the first low-intensity infrared image which corresponds to the determined region of interest;

up-sampling and sharpening the extracted first window image;

selecting a second low-intensity infrared image from the first subset of low-intensity infrared images;

extracting a second window image based on the region of interest of the first low-intensity infrared image;

up-sampling and sharpening the extracted second window image;

generating a composite window based on the first and second extracted window images;

enhancing the generated composite window; and

combining the enhanced composite window with the selected first low-intensity infrared image.

8. The apparatus of claim 7, wherein registering the sequence of enhanced images comprises:

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

9. The apparatus of claim 7, wherein enhancing the composite image comprises:

reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

10. The apparatus of claim 7, further comprising a data recording and storage unit.

11. The apparatus of claim 7, further comprising a display unit.

12. The apparatus of claim 7, further comprising a communication interface unit.

13. A method, comprising:

providing a processor;

collecting, using the processor, a plurality of low-intensity infrared images;

enhancing, using the processor, the plurality of low-intensity infrared images to produce a sequence of enhanced images;

registering, using the processor, the sequence of enhanced images;

accumulating, using the processor, the intensities of the registered sequence of enhanced images to produce a composite image;

enhancing, using the processor, the composite image; and

outputting, using the processor, the enhanced composite image,

wherein registering the sequence of enhanced images comprises;

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

14. The method of claim 13, wherein enhancing the composite image comprises reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

15. A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to:

collect a plurality of low-intensity infrared images;

enhance each of the plurality of low-intensity infrared images to produce a sequence of enhanced images;

register the sequence of enhanced images;

accumulate the intensities of the registered sequence of enhanced images to produce a composite image;

enhance the composite image; and

output the enhanced composite image,

wherein registering the sequence of enhanced images comprises;

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

16. The non-transitory computer-readable medium of claim 15, wherein enhancing the composite image comprises reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

17. An apparatus, comprising:

an image detection unit; and

an image acquisition unit in communication with the image detection unit, the image acquisition unit comprising a processor and memory storing instructions that, when executed by the processor, cause the image acquisition unit to;

collect a plurality of low-intensity infrared images;

enhance each of the plurality of low-intensity infrared images to produce a sequence of enhanced images;

register the sequence of enhanced images;

accumulate the intensities of the registered sequence of enhanced images to produce a composite image;

enhance the composite image; and

output the enhanced composite image,

wherein registering the sequence of enhanced images comprises;

selecting a reference image from the sequence of enhanced images;

determining pixel shifts between the reference image and each of remaining images of the sequence of enhanced images; and

mapping the remaining images of the sequence of enhanced images onto the reference image based on the pixel shifts.

18. The apparatus of claim 17, wherein enhancing the composite image comprises reducing noise in the composite image via local area processing to improve image interpretability for an observer of the composite image.

19. The apparatus of claim 17, further comprising a data recording and storage unit.

20. The apparatus of claim 17, further comprising a display unit.

21. The apparatus of claim 17, further comprising a communication interface unit.