Infrared resolution and contrast enhancement with fusion

• US 10,249,032 B2
• Filed: 09/15/2016
• Issued: 04/02/2019
• Est. Priority Date: 04/23/2010
• Status: Active Grant

• Current Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Original Assignee: Flir Systems AB (Teledyne Technologies Incorporated)
• Inventors: Strandemar, Katrin
• Primary Examiner(s): Couso, Yon J
• Application Number:
  US15/267,133
• Publication Number:
  US 20170004609A1
• Time in Patent Office: 929 Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

The present disclosure relates to combination of images. A method according to an embodiment comprises: receiving a visual image and an infrared (IR) image of a scene and for a portion of said IR image extracting high spatial frequency content from a corresponding portion of said visual image. The method according to the embodiment further comprises combining said extracted high spatial frequency content from said portion of the visual image with said portion of the IR image, to generate a combined image, wherein the contrast and/or resolution in the portion of the IR image is increased compared to the contrast and/or resolution of said received IR image.

Claims

1. An imaging system, comprising:

an infrared (IR) imaging device configured to capture an IR image representing a scene;

a visual imaging device configured to capture a visual image of at least a portion of the scene, wherein optical axes of the IR imaging device and the visual imaging device are located at a distance and an angle from each other; and

a processor configured to;

align the IR image with the visual image at least by compensating for a parallax and/or pointing errors due to the optical axes being located at the distance and the angle;

locate contours and/or edges from the visual image to obtain image data representing the located contours and/or edges; and

modify a luminance component of the IR image based on the image data representing the located contours and/or edges from the visual image to enhance the IR image.

2. The imaging system of claim 1, wherein:

the IR image and the visual image after the aligning show a same part of the scene; and

the processor is configured to determine the parallax and/or pointing errors by performing feature detection and/or based on a pre-stored relationship between the IR and visual imaging devices known from design, production, and/or calibration of the IR and visual imaging devices.

3. The imaging system of claim 1, wherein:

the IR image represents the scene by colors or greyscale levels according to temperature; and

the colors or the greyscale levels representing the scene for at least a portion of the IR image are unaltered by the modifying.

4. The imaging system of claim 1, wherein:

the IR image further comprises at least one color component; and

temperature variations in the scene is represented only by the at least one color component of the IR image, such that the temperature variations in the scene are shown unaltered in the IR image after the modification by the processor.

5. The imaging system of claim 1, wherein the IR image is a greyscale IR image representing temperature variations in the scene by the luminance component.

6. The imaging system of claim 1, wherein:

the image data representing the located contours and/or edges from the visual image comprises luminance data for the located contours and/or edges; and

the processor is configured to modify the luminance component at least by adding the luminance data for the located contours and/or edges to the luminance component of the IR image.

7. The imaging system of claim 1, wherein the contours and/or edges located from the visual image comprise lines, visual patterns, contours, and/or edges of one or more objects in the scene.

8. The imaging system of claim 1, wherein the processor is configured to locate the contours and/or edges from the visual image at least by high-pass filtering the visual image to extract high-spatial frequency content comprising the contours and/or edges.

18. The imaging system of claim 1, wherein the optical axes of the IR imaging device and the visual imaging device are not parallel.

9. A method, comprising:

receiving an infrared (IR) image representing a scene, wherein the IR image is captured by an IR imaging device;

receiving a visual image of at least a portion of the scene, wherein the visual image is captured by a visual imaging device;

aligning the IR image with the visual image, wherein the aligning compensates for a parallax and/or pointing errors caused by optical axes of the IR and visual imaging devices that are located at a distance and an angle from each other;

locating contours and/or edges from the visual image to obtain image data representing the located contours and/or edges; and

modifying the IR image based on the image data representing the located contours and/or edges from the visual image to enhance the IR image.

10. The method of claim 9, wherein:

the IR image and the visual image after the aligning show a same part of the scene; and

the aligning comprises determining the parallax and/or pointing errors by performing feature detection and/or based on a pre-stored relationship between the IR and visual imaging devices known from design, production, and/or calibration of the IR and visual imaging devices.

11. The method of claim 9, wherein:

the IR image represents the scene by colors or greyscale levels according to temperature; and

the colors or the greyscale levels representing the scene for at least a portion of the IR image are unaltered by the modifying.

12. The method of claim 11, wherein:

the IR image comprises a luminance component; and

the modifying of the IR image comprises modifying the luminance component of the IR image based on the image data representing the located contours and/or edges from the visual image.

13. The method of claim 12, wherein:

the IR image further comprises at least one color component; and

temperature variations in the scene is represented only by the at least one color component of the IR image, such that the temperature variations in the scene are shown unaltered in the IR image after the modifying.

14. The method of claim 12, wherein the IR image is a greyscale IR image representing temperature variations in the scene by the luminance component.

15. The method of claim 12, wherein:

the image data representing the located contours and/or edges from the visual image comprises luminance data for the located contours and/or edges; and

the modifying of the luminance component comprises adding the luminance data for the located contours and/or edges to the luminance component of the IR image.

16. The method of claim 9, wherein the contours and/or edges located from the visual image comprise lines, visual patterns, contours, and/or edges of one or more objects in the scene.

17. The method of claim 9, wherein the locating of the contours and/or edges comprises high-pass filtering the visual image to extract high-spatial frequency content comprising the contours and/or edges.

19. The method of claim 9, wherein the optical axes of the IR imaging device and the visual imaging device are not parallel.