Infrared resolution and contrast enhancement with fusion

• US 9,171,361 B2
• Filed: 04/02/2012
• Issued: 10/27/2015
• Est. Priority Date: 04/23/2010
• Status: Active Grant

• Current Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Original Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Inventors: Strandemar, Katrin
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US13/437,645
• Publication Number:
  US 20120262584A1
• Time in Patent Office: 1,303 Days
• Field of Search: 382/284, 348/164
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The present disclosure relates to combination of images. A method according to an embodiment comprises: receiving a visual image and an infrared (IR) image of a scene and for a portion of said IR image extracting high spatial frequency content from a corresponding portion of said visual image. The method according to the embodiment further comprises combining said extracted high spatial frequency content from said portion of the visual image with said portion of the IR image, to generate a combined image, wherein the contrast and/or resolution in the portion of the IR image is increased compared to the contrast and/or resolution of said received IR image.

Claims

1. A method, comprising:

receiving, by a processor, a visual image of a scene captured using a visual image sensor;

receiving, by the processor, an infrared (IR) image of the scene captured using an IR image sensor;

for a portion of said IR image;

extracting, by the processor from a corresponding portion of the visual image, pixel data representing high spatial frequency content, wherein the high spatial content includes contours and/or edges that show low or no contrast in the IR image; and

combining, by the processor, luminance information of said extracted pixel data from said portion of the visual image with luminance information of corresponding pixels in said portion of the IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.

2. The method of claim 1, wherein the corresponding portion of the visual image is the portion that shows the same part of the observed real world scene as the portion of the IR image.

3. The method of claim 1, wherein the portion of said IR image is the entire IR image.

4. The method of claim 1, wherein said portion of the IR image and said corresponding portion of the visual image are scaled to a predetermined size.

5. The method of claim 4, wherein said predetermined size is a selection of:

the size of the captured IR image;

the size of the captured visual image; and

/or

the size of a display onto which the augmented IR image is to be displayed.

6. The method of claim 1, further comprising receiving a control signal indicating a manual selection of a portion of said IR image.

7. The method of claim 1, wherein said portion of the IR image is a predetermined area in the IR image.

8. The method of claim 1, wherein the resolution of the visual image and the resolution of the IR image are substantially the same.

9. The method of claim 1, further comprising ensuring that a resolution for said portion of the visual image and said portion of the IR image are substantially the same.

10. The method of claim 1, further comprising checking whether a resolution for said portion of the visual image and said portion of the IR image are substantially the same;

and wherein, if the resolution for said portion of the visual image and said portion of the IR image are not substantially the same, altering a resolution of said portion of the visual image and/or a resolution of said portion of the IR image such that the resolution for said portion of the visual image and said portion of the IR image are substantially the same.

11. The method of claim 1, further comprising altering at least one of a resolution of said portion of said visual image and a resolution of said portion of said IR image such that said portion of the visual image and said portion of the IR image have substantially the same resolution.

12. The method of claim 1, further comprising up-sampling a resolution of said portion of said IR image to substantially a resolution of said portion of said visual image and/or down-sampling the resolution of said portion of said visual image to substantially the resolution of said portion of said IR image.

13. The method of claim 1, further comprising altering a resolution of said portion of said visual image and a resolution of said portion of said IR image to be substantially the same as a third resolution.

14. The method of claim 1, wherein a resolution of said portion of the IR image is 64×

64 pixels or less.

15. The method of claim 1, wherein a resolution of said portion of the IR image is 32×

32 pixels or less.

16. The method of claim 1, wherein the extracting high spatial frequency content from said portion of said visual image is performed by high pass filtering using at least one spatial filter.

17. The method of claim 1, further comprising processing said IR image to reduce noise in the IR image and/or smooth the IR image.

18. The method of claim 1, further comprising low pass filtering said IR image prior to the combining.

19. The method of claim 1, further comprising low pass filtering said IR image by use of a spatial filter prior to the combining.

20. The method of claim 1, wherein the combining comprises superimposing said extracted high spatial frequency content from said portion of said visual image on said portion of said IR image.

21. The method of claim 1, wherein the combining comprises superimposing said IR image on said extracted high spatial frequency content from said visual image.

22. The method of claim 1, wherein the combining comprises adding only a luminance component of said portion of said visual image to said portion of said IR image.

23. The method of claim 22, wherein the combining comprises adding a luminance component of said portion of said visual image multiplied by a factor to said portion of said IR image.

24. The method of claim 23, wherein said factor is variable based on a control parameter received from an input from a user.

25. The method of claim 1, further comprising adding high resolution noise to said augmented IR image.

26. The method of claim 1, further comprising maintaining an IR palette for said IR image and/or for said augmented IR image.

27. The method of claim 1, further comprising capturing a visual image and an IR image of a scene, and wherein the method of claim 1 is performed by an imaging device comprising an IR image capturing device and a visual image capturing device.

28. The method of claim 1, wherein the method is performed by a computer adapted to receive a visual image and an IR image of a scene.

32. An imaging device, comprising:

a visual image sensor for capturing a visual image;

an IR image sensor for capturing an IR image; and

a processing unit adapted to;

for a portion of said IR image;

extract, from a corresponding portion of the visual image, pixel data representing high spatial frequency content from a corresponding portion of said visual image, wherein the high spatial frequency content includes contours and/or edges that show low or no contrast in the IR image; and

combine luminance information of said extracted pixel data from said portion of the visual image with luminance information of corresponding pixels in said portion of the IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining, wherein the processing unit is further adapted to perform the method of claim 10.

29. A non-transitory computer-readable medium storing a computer program for combining an IR image with a visual image, the computer program comprising computer program code portions devised to control a data processing system to perform operations comprising:

receiving a visual image of a scene captured using a visual image sensor;

receiving an infrared (IR) image of the scene captured using an IR image sensor;

for a portion of said IR image;

extracting, from a corresponding portion of the visual image, pixel data representing high spatial frequency content, wherein the high spatial frequency content includes contours and/or edges that show low or no contrasts in the IR image; and

combining luminance information of said extracted pixel data from said portion of the visual image with luminance information of corresponding pixels in said portion of the IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.

30. The non-transitory computer-readable medium of claim 29, wherein the operations further comprise:

checking whether a resolution for said portion of the visual image and said portion of the IR image are substantially the same; and

wherein, if the resolution for said portion of the visual image and said portion of the IR image are not substantially the same, altering a resolution of said portion of the visual image and/or a resolution of said portion of the IR image such that the resolution for said portion of the visual image and said portion of the IR image are substantially the same.

31. An imaging device, comprising:

a visual image sensor for capturing a visual image;

an IR image sensor for capturing an IR image; and

a processing unit adapted to;

for a portion of said IR image;

extract, from a corresponding portion of the visual image, pixel data representing high spatial frequency content from a corresponding portion of said visual image, wherein the high spatial frequency content includes contours and/or edges that show low or no contrast in the IR image; and

combine luminance information of said extracted pixel data from said portion of the visual image with luminance information of corresponding pixels in said portion of the IR image to augment the IR image with the contours and/or edges from the visual image, wherein the augmented IR image comprises temperature information that is unaltered by the combining.