Infrared and visible-light image registration

• US 7,924,312 B2
• Filed: 08/22/2008
• Issued: 04/12/2011
• Est. Priority Date: 08/22/2008
• Status: Active Grant

• Current Assignee: Fluke Corporation (Fortive Corp.)
• Original Assignee: Fluke Corporation (Fortive Corp.)
• Inventors: Packard, Corey D.
• Primary Examiner(s): WHIPKEY, JASON T
• Application Number:
  US12/196,433
• Publication Number:
  US 20100045809A1
• Time in Patent Office: 963 Days
• Field of Search: 348/164, 348/262, 348/263, 348/265, 348/239, 348/33, 250/330, 250/332
• US Class Current: 348/164
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Methods and thermal imaging cameras are provided for registering visible-light and infrared images within a thermal imaging camera to reduce a parallax error in the images. Registration generally includes detecting a first feature set in the visible-light image data, detecting a second feature set in the infrared image data and generating a plurality of overlapping alignments of the infrared image data and the visible-light image data. A similarity figure is calculated as a function of the relative alignment of the first feature set and the second feature set for each alignment and the alignment with the desired similarity figure is selected. The images are aligned and the camera displays at least a portion of the visible-light image data and/or at least a portion of the infrared image data.

Claims

1. A method of registering a visible-light image and an infrared image in a thermal imaging camera, comprising:

receiving visible-light image data corresponding to a first view of a scene and infrared image data corresponding to a second view of the scene, the first view and the second view, and the corresponding visible-light image data and infrared image data, having a parallax error;

detecting a first feature set in the visible-light image data;

detecting a second feature set in the infrared image data;

generating a plurality of overlapping alignments of the infrared image data and the visible-light image data;

calculating a similarity figure for each overlapping alignment, the similarity figure being a function of the relative alignment of the first feature set and the second feature set;

selecting one of the overlapping alignments having a desired similarity figure;

aligning the visible-light image data and the infrared image data according to the selected overlapping alignment; and

displaying at least a portion of the visible-light image data and/or at least a portion of the infrared image data.

2. The method of claim 1, wherein the visible-light image data comprises color data and further comprising converting the visible-light image data to visible-light intensity data.

3. The method of claim 1, wherein the infrared image data comprises color data and further comprising converting the infrared image data to infrared intensity data.

4. The method of claim 1, wherein the first and second feature sets comprise edges in the visible-light image data and the infrared image data, respectively.

5. The method of claim 4, further comprising enhancing the edges in the first and second feature sets.

6. The method of claim 1, further comprising thresholding the first feature set and the second feature set and generating binary representations of the visible-light image data and the infrared image data.

7. The method of claim 1, wherein the desired similarity figure represents the greatest degree of similarity between the first feature set and the second feature set among the overlapping alignments.

8. The method of claim 7, further comprising comparing the desired similarity figure to a similarity threshold and setting the selected overlapping alignment to a default alignment if the desired similarity is less than the similarity threshold.

9. The method of claim 8, wherein the default alignment is a manual alignment.

10. The method of claim 1, further comprising employing a subset of at least one of the visible-light image data and the infrared image data in generating the plurality of overlapping alignments.

11. The method of claim 10, wherein the subset is determined from a physical configuration of the thermal imaging camera.

12. A thermal imaging camera, comprising:

a visible-light camera module having a visible-light sensor for generating visible-light image data corresponding to a first view of a scene;

an infrared camera module having an infrared sensor for generating infrared image data corresponding to a second view of the scene, the infrared sensor offset from the visible-light sensor such that the first view and the second view, and the corresponding visible-light image data and infrared image data, have a parallax error;

a display for displaying at least a portion of the visible-light image data and/or at least a portion of the infrared image data; and

a programmable processor coupled with the display and the infrared and visible-light camera modules, the processor programmed with instructions for registering the visible-light image data and the infrared image data, the registering comprising

detecting a first feature set in the visible-light image data,

detecting a second feature set in the infrared image data,

generating a plurality of overlapping alignments of the infrared image data and the visible light image data,

calculating a similarity figure for each overlapping alignment, the similarity figure being a function of the relative alignment of the first feature set and the second feature set,

selecting one of the overlapping alignments having a desired similarity figure, and

aligning the visible-light image data and the infrared image data according to the selected overlapping alignment.

13. The thermal imaging camera of claim 12, the registering further comprising employing a subset of at least one of the visible-light image data and the infrared image data in generating the plurality of overlapping alignments, wherein the subset is determined from a position of the first view relative to the second view.

14. The thermal imaging camera of claim 13, wherein the visible-light camera module and the infrared camera module are configured such that the first view is substantially horizontally aligned with the second view.

15. The thermal imaging camera of claim 12, the registering further comprising thresholding the first feature set and the second feature set and generating binary representations of the visible-light image data and the infrared image data.

16. The thermal imaging camera of claim 12, wherein the first and second feature sets comprise edges in the visible-light image data and the infrared image data, respectively.

17. A thermal imaging camera, comprising:

a visible-light camera module for generating visible-light image data corresponding to a first view of a scene;

an infrared camera module for generating infrared image data corresponding to a second view of the scene, the infrared camera module offset from the visible-light camera module such that the first view and the second view, and the corresponding visible-light image data and infrared image data, have a parallax error;

a display for displaying at least a portion of the visible-light image data and/or at least a portion of the infrared image data;

a manual adjustment mechanism for reducing the parallax error; and

a programmable processor coupled with the display and the infrared and visible-light camera modules, the processor programmed with instructions for registering the visible-light image data and the infrared image data, the registering comprising

detecting a first feature set in the visible-light image data,

detecting a second feature set in the infrared image data,

generating a plurality of overlapping alignments of the infrared image data and the visible light image data,

calculating a similarity figure for each overlapping alignment, the similarity figure being a function of the relative alignment of the first feature set and the second feature set,

selecting one of the plurality of overlapping alignments having a desired similarity figure, and

aligning the visible-light image data and the infrared image data according to the selected overlapping alignment.

18. The thermal imaging camera of claim 17, wherein the registering further comprises employing a subset of at least one of the visible-light image data and the infrared image data in generating the plurality of overlapping alignments, the subset corresponding to a setting of the manual adjustment mechanism.

19. The thermal imaging camera of claim 18, wherein the manual adjustment mechanism comprises focusable optics for adjusting the first view of the scene or the second view of the scene.

20. The thermal imaging camera of claim 17, further comprising comparing the desired similarity figure to a similarity threshold and setting the selected overlapping alignment to a manual alignment determined by the manual adjustment mechanism if the desired similarity is less than the similarity threshold.