Visible light image with edge marking for enhancing IR imagery

• US 9,990,730 B2
• Filed: 03/21/2014
• Issued: 06/05/2018
• Est. Priority Date: 03/21/2014
• Status: Active Grant

• Current Assignee: Fluke Corporation (Fortive Corp.)
• Original Assignee: Fluke Corporation (Fortive Corp.)
• Inventors: Bergstrom, Peter A., Johnson, Kirk R., Pickett, James T.
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Hasan, Syed
• Application Number:
  US14/222,153
• Publication Number:
  US 20150269742A1
• Time in Patent Office: 1,537 Days
• Field of Search: 348164, 348143, 382144, 382106, 382209, 345 55, 345 87, 250330
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G09G 3/20 for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/83 on disks or drums

Abstract

The invention relates generally to edge detection and presentation in thermal images. Infrared and visible light images comprising at least partially overlapping target scenes are analyzed. An edge detection process is performed on the visible light image to determine which pixels represent edges in the target scene. A display image is generated in which some pixels include infrared image data and in which pixels corresponding to edges in the visible light image include a predetermined color and do not include corresponding infrared image data to emphasize edges. Edge pixels in the display image can include exclusively the predetermined color, or in some examples, a blend of a predetermined color and visible light image data. Methods can include replacing one or the other of visible light edge pixels or corresponding infrared pixels with the predetermined color before combining the visible light and infrared image data to create a display image.

Claims

1. A non-transitory computer readable medium containing executable instructions causing one or more processors to perform a method of creating an infrared image of a scene that emphasizes the locations of visible edges, the method comprising:

receiving visible light (VL) image data associated with a plurality of VL pixels indicative of a VL image of a target scene;

receiving infrared (IR) image data associated with a plurality of IR pixels indicative of an IR image of the target scene, each IR pixel having a corresponding one or more VL pixels indicative of the same portion of the target scene;

determining which VL pixels correspond to edges in the VL image data and considering such pixels as VL edge pixels; and

generating display image data associated with a plurality of display pixels, each display pixel having corresponding VL and IR pixels that correspond to the same portion of the target scene as such display pixel;

wherein

some of the display pixels include IR image data associated with corresponding IR pixels, and display pixels corresponding to VL edge pixels include a predetermined color that is independent of the received VL and IR image data and do not include IR image data associated with corresponding IR pixels in order to emphasize the location of edges in the display image data.

2. The non-transitory computer-readable medium of claim 1, wherein the predetermined color is selectable by a user.

3. The non-transitory computer-readable medium of claim 1, wherein, for display pixels corresponding to VL pixels that are the VL non-edge pixels, the generated display image data of each such display pixel comprises a blend of between 0% and 100% IR image data from the associated IR pixel corresponding to such display pixel.

4. The non-transitory computer-readable medium of claim 3, wherein the amount of blending is selectable by a user.

5. The non-transitory computer-readable medium of claim 1,

further including, for IR pixels corresponding to the VL edge pixels, replacing the IR image data of each such IR pixel with the predetermined color, creating modified IR image data, and

wherein, for display pixels corresponding to the VL edge pixels, including the predetermined color includes, for each such display pixel, blending the modified IR image data associated with the corresponding IR pixel and the corresponding VL image data.

6. The non-transitory computer-readable medium of claim 1, further configured to receive visible light and corresponding infrared image data and generate a display image data in substantially real time.

7. The non-transitory computer-readable medium of claim 6, wherein generating a display image data in substantially real time comprises generating a display video comprising a frame rate of approximately 60 frames per second.

8. The non-transitory computer-readable medium of claim 1, wherein determining which VL pixels correspond to the edges in the VL image data of the target scene:

(i) measuring the variance in a kernel of VL pixels; and

(ii) if the variance is above a predetermined threshold value, comparing a target pixel within the kernel to the mean pixel within the kernel.

9. A display system for generating and displaying a display image comprising:

a processor;

a display; and

wherein the display system is configured to

receive a visible light (VL) image and a corresponding infrared (IR) image;

detect pixels in the VL image representative of detected edges in the VL image and consider such pixels as VL edge pixels; and

generate a display image corresponding to the VL and IR images and comprising a plurality of display pixels, wherein

at least one display pixel corresponding to a VL pixel not considered to be a VL edge pixel comprises corresponding IR image data; and

display pixels corresponding to the VL pixels that are considered to be VL edge pixels comprise a predetermined color that is independent of the received VL and IR images and do not comprise corresponding IR image data in order to emphasize the location of edges in the display image data.

10. The system of claim 9, further comprising

a portable, hand-held thermal imaging camera including:

an infrared (IR) camera module comprising an IR lens assembly and an associated IR sensor for detecting IR images of a target scene; and

a visible light (VL) camera module comprising a VL lens assembly having an associated VL sensor for detecting VL images of the target scene;

the system being further configured to;

capture the VL image of the target scene via the VL camera module; and

capture the corresponding IR image of a target scene via the IR camera module.

11. The system of claim 10, wherein the visible light camera module comprises a second processor, and wherein the second processor is configured to detect pixels in the VL image representative of detected edges in the VL image and considering such pixels as VL edge pixels and considering the VL pixels not corresponding to the detected edges as VL non-edge pixels.

12. The system of claim 11, wherein the second processor is configured to replace the pixels in the VL image considered to be VL edge pixels with pixels of exclusively the predetermined color, creating a modified VL image.

13. The system of claim 12, wherein the processor is configured to:

(i) receive the modified VL image from the second processor; and

(ii) blend at least a portion of the modified VL image with a corresponding portion of the IR image to generate a portion of the display image;

wherein

the pixels in the modified VL image having exclusively the predetermined color are unaffected by the blending.

14. The system of claim 13, wherein the display image forms a part of a picture-in-picture image, wherein

a first portion of the picture-in-picture image comprises the display image;

and

a second portion of the picture-in-picture image comprises an unblended portion of the VL image.

15. The system of claim 9, further configured to

receive VL and IR images and generate display images in substantially real time;

and

present the display images in substantially real time on a display.

16. The system of claim 9, wherein

detecting pixels in the VL image representative of detected edges in the VL image comprises executing an algorithm for detecting edges in the VL image;

and wherein

the algorithm is adjustable by the user via a user interface such that the sensitivity of the edge detection can be adjusted by a user.

17. The system of claim 16, wherein the algorithm comprises:

(i) measuring the variance in a kernel of pixels; and

(ii) if the variance is above a predetermined threshold value, comparing a target pixel within the kernel to the mean pixel within the kernel.

18. The system of claim 16, wherein the algorithm comprises generating a color histogram of the VL pixels in the VL image and detecting VL edge pixels based on a user-defined threshold of color frequency in the color histogram.

19. The system of claim 9, wherein generating the display image comprises replacing pixels in the IR image corresponding to the VL pixels considered to be VL edge pixels with the predetermined color.

20. A method of enhancing infrared images comprising:

receiving visible light (VL) image data associated with a plurality of VL pixels representative of a target scene;

receiving infrared (IR) image data associated with a plurality of 1R pixels representative of at least a portion of the target scene, and at least a portion of the IR pixels corresponding to one or more VL pixels;

detecting edge pixels in the VL image data representative of edges in the VL image data; and

generating display image data associated with a plurality of display pixels, wherein at least one display pixel includes IR image data from at least one corresponding IR pixel, and display pixels corresponding to edge pixels in the VL image include a predetermined color that is unrelated to the received VL and IR image data and do not include corresponding IR image data in order to emphasize the location of edges in the display image data.

21. The method of claim 20, wherein generating display image data comprises:

for detected edge pixels, replacing one of (a) the detected edge pixel in the VL image data, (b) one or more IR pixels corresponding to the detected edge pixel, or (c) one or more display pixels corresponding to the detected edge pixel with the predetermined color.