INFRARED IMAGING ENHANCEMENT WITH FUSION

• US 20150334315A1
• Filed: 12/21/2013
• Published: 11/19/2015
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hgasten, Nicholas, Scott, Jeffrey S., Strandemar, Katrin, Nussmeier, Mark, Kurth, Eric A., Hoelter, Theodore R., Boulanger, Pierre, Sharp, Barbara, Teich, Andrew C.
• Granted Patent: US 9,635,285 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2218/04 Denoising
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/21 for transforming thermal infrared radiation into image signals
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 25/674 based on the scene itself, e.g. defocusing
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Triple fusion processing, including selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.

Claims

1. A system comprising:

a memory adapted to receive a visible spectrum image of a scene and an infrared image of the scene; and

a processor in communication with the memory and configured to;

receive control parameters,

derive color characteristics of the scene from at least one of the images,

derive high spatial frequency content from at least one of the images, and

generate a combined image comprising relative contributions of the color characteristics and the high spatial frequency content, wherein the relative contributions are determined by the control parameters.

2. The system of claim 1, wherein:

the infrared image is an unblurred infrared image of the scene corrected by a plurality of non-uniform correction (NUC) terms used to remove noise from the unblurred infrared image; and

the NUC terms are determined based on an intentionally blurred infrared image of the scene.

3. The system of claim 2, wherein:

the processor is configured to selectively apply the NUC terms to the unblurred infrared image to selectively remove noise from the unblurred infrared image; and

the selective application of the NUC terms is determined by the control parameters.

4. The system of claim 2, further comprising an infrared imaging module in communication with the memory and configured to:

capture the intentionally blurred infrared image of the scene; and

determine the NUC terms.

5. The system of claim 1, wherein:

one or more of the control parameters are determined by one or more threshold values associated with at least one of the images.

6. The system of claim 1, further comprising a user interface in communication with the memory and the processor, wherein:

the processor is configured to receive user input from the user interface; and

one or more of the control parameters are determined from the user input.

7. The system of claim 1, wherein:

the color characteristics of the scene are derived from a chrominance component of the visible spectrum image;

the relative contributions include a luminance component of the infrared image; and

the relative contribution of the luminance component of the infrared image substantially matches the relative contribution of the color characteristics.

8. The system of claim 7, wherein:

the luminance component of the infrared image is blended with a luminance component of the visible spectrum image before being contributed to the combined image.

9. The system of claim 1, wherein:

the high spatial frequency content is derived from a luminance component of the visible spectrum image;

the relative contributions include a chrominance component of the infrared image; and

the relative contribution of the chrominance component of the infrared image substantially matches the relative contribution of the high spatial frequency content.

10. The system of claim 9, wherein:

the high spatial frequency content is blended with a luminance component of the infrared image before being contributed to the combined image.

11. The system of claim 1, wherein:

the system further comprises an infrared imaging module comprising a focal plane array (FPA) configured to capture the infrared image of the scene; and

the FPA comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts.

12. A method comprising:

receiving a visible spectrum image of a scene and an infrared image of the scene;

receiving control parameters;

deriving color characteristics of the scene from at least one of the images;

deriving high spatial frequency content from at least one of the images; and

generating a combined image comprising relative contributions of the color characteristics and the high spatial frequency content, wherein the relative contributions are determined by the control parameters.

13. The method of claim 12, wherein:

the infrared image is an unblurred infrared image of the scene corrected by a plurality of non-uniform correction (NUC) terms used to remove noise from the unblurred infrared image; and

the NUC terms are determined based on an intentionally blurred infrared image of the scene.

14. The method of claim 13, further comprising selectively applying the NUC terms to the unblurred infrared image.

15. The method of claim 13, wherein an infrared imaging module is configured to capture the intentionally blurred infrared image of the scene and determine the NUC terms.

16. The method of claim 12, wherein:

one or more of the control parameters are determined by one or more threshold values associated with at least one of the images.

17. The method of claim 12, further comprising:

receiving user input from a user interface, wherein one or more of the control parameters are determined by the user input.

18. The method of claim 12, further comprising:

receiving information from a sensor, wherein one or more of the control parameters are determined by the information, wherein the information is associated with an amount of available ambient light.

19. The method of claim 12, wherein:

the color characteristics of the scene are derived from a chrominance component of the visible spectrum image;

the relative contributions include a luminance component of the infrared image; and

the relative contribution of the luminance component of the infrared image substantially matches the relative contribution of the color characteristics.

20. The method of claim 19, wherein:

the luminance component of the infrared image is blended with a luminance component of the visible spectrum image before being contributed to the combined image.

21. The method of claim 12, wherein:

the high spatial frequency content is derived from a luminance component of the visible spectrum image;

the relative contributions include a chrominance component of the infrared image; and

the relative contribution of the chrominance component of the infrared image substantially matches the relative contribution of the high spatial frequency content.

22. The method of claim 21, wherein:

the high spatial frequency content is blended with a luminance component of the infrared image before being contributed to the combined image.

23. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors of a system are adapted to cause the system to perform a method comprising:

receiving a visible spectrum image of a scene and an infrared image of the scene;

receiving control parameters;

deriving color characteristics of the scene from at least one of the images;

deriving high spatial frequency content from at least one of the images; and

generating a combined image comprising relative contributions of the color characteristics and the high spatial frequency content, wherein the relative contributions are determined by the control parameters.