Infrared resolution and contrast enhancement with fusion

• US 8,565,547 B2
• Filed: 05/11/2011
• Issued: 10/22/2013
• Est. Priority Date: 04/23/2010
• Status: Active Grant

• Current Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Original Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Inventors: Strandemar, Katrin
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US13/105,765
• Publication Number:
  US 20110261207A1
• Time in Patent Office: 895 Days
• Field of Search: 382/284, 382/294, 382/299, 382/263, 382/264, 348/164, 348/135
• US Class Current: 382/263
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The present disclosure relates to combination of images. A method according to an embodiment comprises: receiving a visual image and an infrared (IR) image of a scene; extracting high spatial frequency content from said visual image; and combining said extracted high spatial frequency content from said visual image with said IR image, wherein a resolution for the visual image and the IR image are substantially the same, to generate a combined image.

Claims

1. A method, comprising:

receiving a visual image of a scene captured using a visual image sensor;

receiving an infrared (IR) image of at least a portion of the scene captured using an IR image sensor;

extracting, from the visual image, pixel data representing high spatial frequency content of said visual image, wherein the high spatial frequency content includes contours and/or edges that show no or low contrast in the IR image; and

combining, by a processor, luminance information of said extracted pixel data from said visual image with luminance information of corresponding pixels in said IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.

2. The method of claim 1, further comprising altering at least one of a resolution of said visual image and a resolution of said IR image such that the visual image and the IR image have substantially the same resolution.

3. The method of claim 1, further comprising up-sampling a resolution of said IR image to substantially a resolution of said visual image and/or down-sampling the resolution of said visual image to substantially the resolution of said IR image.

4. The method of claim 1, further comprising altering a resolution of said visual image and a resolution of said IR image to be substantially the same as a third resolution.

5. The method of claim 1, wherein a resolution of the received IR image is 64×

64 pixels or less.

6. The method of claim 1, wherein a number of pixels in the received IR image is less than a number of pixels in the received visual image by more than two orders of magnitude.

7. The method of claim 1, wherein the extracting the pixel data representing the high spatial frequency content from said visual image is performed by high pass filtering using at least one spatial filter.

8. The method of claim 1, further comprising processing said IR image to reduce noise in the IR image and/or smooth the IR image.

9. The method of claim 1, further comprising low pass filtering said IR image prior to the combining.

10. The method of claim 9, wherein the low pass filtering said IR image is by use of a spatial filter.

11. The method of claim 1, wherein the combining the luminance information further comprises:

multiplying respective luminance values of the extracted pixel data by a factor; and

adding the luminance values of the extracted pixel data to luminance values of the corresponding pixels in said IR image.

12. The method of claim 11, wherein said factor is variable based on a control parameter received from an input from a user.

13. The method of claim 1, further comprising adding high resolution noise to said combined image.

14. The method of claim 1, further comprising capturing the visual image and the IR image of the scene, and wherein the method of claim 1 is performed by an imaging device comprising the IR image sensor and the visual image sensor.

15. The method of claim 1, wherein the method is performed by a computer adapted to receive the visual image and the IR image of the scene.

20. The method of claim 1, wherein the visual image sensor comprises a CCD imaging device, an EMCCD imaging device, a CMOS imaging device, or an sCMOS imaging device.

21. The method of claim 1, wherein the IR image sensor is adapted to detect electromagnetic radiation having a wavelength between 0.7 and 20 μ

m, and wherein the IR image is represented in one or more colors from a palette according to temperature.

22. The method of claim 21, wherein the combining the luminance information maintains the palette associated with the IR image.

23. The method of claim 1, wherein the visual image is a first visual image, further comprising obtaining a difference image between the first visual image and a second visual image captured by the visual image sensor, wherein the extracting the pixels from the visual image comprises extracting the pixels from the difference image.

24. The method of claim 23, wherein the difference image is obtained by a change in the scene and/or a change in a position of the visual imaging sensor between the first and second visual images.

25. The method of claim 23, wherein the second visual image is intentionally de-focused to obtain the difference image.

16. A non-transitory computer-readable medium storing computer program code which, when executed by a processor, causes the processor to perform a method comprising:

receiving a visual image of a scene captured using a visual image sensor;

receiving an infrared (IR) image of at least a portion of the scene captured using an IR image sensor;

extracting, from the visual image, pixel data representing high spatial frequency content of said visual image, wherein the high spatial frequency content includes contours and/or edges that show no or low contrast in the IR image; and

combining luminance information of said extracted pixel data from said visual image with luminance information of corresponding pixels in said IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.

17. The computer-readable medium of claim 16, wherein the method further comprises:

checking whether a resolution for the visual image and the IR image are substantially the same; and

wherein, if the resolution for the visual image and the IR image are not substantially the same, altering a resolution of the visual image and/or a resolution of the IR image such that the resolution for the visual image and the IR image are substantially the same.

18. An imaging device, comprising:

a visual image sensor for capturing a visual image;

an infrared (IR) image sensor for capturing an IR image; and

a processor adapted to;

extract, from the visual image, pixel data representing high spatial frequency content of said visual image, wherein the high spatial frequency content includes contours and/or edges that show no or low contrast in the IR image, and

combine luminance information of said extracted pixel data from said visual image with luminance information of corresponding pixels in said IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.

19. The imaging device of claim 18, wherein the processor is further adapted to ensure that a resolution for the visual image and a resolution for the IR image are substantially the same.