Time spaced infrared image enhancement

• US 9,451,183 B2
• Filed: 12/21/2013
• Issued: 09/20/2016
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hgasten, Nicholas, Dumpert, Dwight, Hoelter, Theodore R., Scott, Jeffrey S., Strandemar, Katrin, Nussmeier, Mark, Kurth, Eric A., Boulanger, Pierre, Sharp, Barbara
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US14/138,040
• Publication Number:
  US 20140240512A1
• Time in Patent Office: 1,004 Days
• Field of Search: 382/264
• US Class Current: 1/1
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Triple fusion processing, including selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.

Claims

1. A system comprising:

a visible spectrum imaging module comprising a plurality of visible spectrum sensors configured to capture a visible spectrum image of a scene at a first time;

an infrared imaging module comprising a plurality of infrared sensors configured to capture an infrared image of the scene at a second time different from the first time, wherein the infrared image comprises an infrared data component; and

a processor configured to communicate with the visible spectrum imaging module and the infrared imaging module and to process the visible spectrum image and the infrared image to generate a combined image comprising visible spectrum characteristics of the scene derived from the visible spectrum image and infrared characteristics of the scene derived from the infrared data component of the infrared image.

2. The system of claim 1, wherein:

high spatial frequency content is derived from the visible spectrum image or the infrared image; and

the combined image comprises the high spatial frequency content encoded into one or more components of the combined image.

3. The system of claim 2, wherein:

the infrared data component comprises thermal data;

the high spatial frequency content is derived from the visible spectrum image; and

the combined image comprises the infrared data component encoded into a chrominance component of the combined image and the high spatial frequency content encoded into a luminance component of the combined image.

4. The system of claim 2, wherein:

the high spatial frequency content is derived by high pass filtering and/or difference imaging performed on the visible spectrum image and/or the infrared image.

5. The system of claim 2, wherein:

the high spatial frequency content is blended with one or more components of the infrared image to produce blended image data; and

the combined image comprises the blended image data encoded into the one or more components of the combined image.

6. The system of claim 1, wherein:

the infrared data component comprises thermal data;

the combined image comprises a chrominance component of the visible spectrum image encoded into a chrominance component of the combined image and blended image data encoded into a luminance component of the combined image; and

the blended image data comprises the infrared data component blended with a luminance component of the visible spectrum image.

7. The system of claim 1, wherein:

the visible spectrum image is captured by the visible spectrum imaging module while the scene is visibly illuminated; and

the infrared image is captured by the infrared imaging module while the scene is not visibly illuminated.

8. The system of claim 1, wherein:

the visible spectrum image is captured by the visible spectrum imaging module while the scene is visibly illuminated by ambient light or artificial illumination; and

the second time is before the first time.

9. The system of claim 1, wherein:

a first resolution of the infrared image or the visible spectrum image is scaled to approximate a second resolution of a display, a corresponding image, or a specified resolution.

10. The system of claim 1, wherein:

the visible spectrum imaging module and the infrared imaging module are adapted to have approximately a same field of view.

11. The system of claim 1, further comprising:

a display configured to present the combined image to a user.

12. The system of claim 1, wherein:

the infrared imaging module comprises a focal plane array (FPA) configured to capture the infrared image of the scene; and

the FPA comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts.

13. The system of claim 12, wherein:

a size of the array of microbolometers is approximately 80 by 60; and

the infrared imaging module is adapted to be inserted into a socket having a size less than approximately 8.5 mm by 8.5 mm.

14. The system of claim 1, wherein:

the infrared image is an unblurred infrared image of the scene;

the infrared imaging module is configured to capture an intentionally blurred infrared image of the scene; and

the processor is configured to determine a plurality of non-uniform correction (NUC) terms based on the intentionally blurred infrared image and apply the NUC terms to the unblurred infrared image to remove noise from the unblurred infrared image.

15. A method comprising:

receiving a visible spectrum image of a scene captured at a first time by a visible spectrum imaging module comprising a plurality of visible spectrum sensors;

receiving an infrared image of the scene captured at a second time, different from the first time, by an infrared imaging module comprising a plurality of infrared sensors, wherein the infrared image comprises an infrared data component; and

processing the visible spectrum image and the infrared image to generate a combined image comprising visible spectrum characteristics of the scene derived from the visible spectrum image and infrared characteristics of the scene derived from the infrared data component of the infrared image.

16. The method of claim 15, wherein:

high spatial frequency content is derived from the visible spectrum image or the infrared image; and

the combined image comprises high spatial frequency content encoded into one or more components of the combined image.

17. The method of claim 16, wherein:

the infrared data component comprises thermal data;

the high spatial frequency content is derived from the visible spectrum image; and

the combined image comprises the infrared data component encoded into a chrominance component of the combined image and the high spatial frequency content encoded into a luminance component of the combined image.

18. The method of claim 16, wherein:

the high spatial frequency content is derived by high pass filtering and/or difference imaging performed on the visible spectrum image and/or the infrared image.

19. The method of claim 16, wherein:

the high spatial frequency content is blended with one or more components of the infrared image to produce blended image data; and

the combined image comprises the blended image data encoded into the one or more components of the combined image.

20. The method of claim 15, wherein:

the infrared data component comprises thermal data;

the combined image comprises a chrominance component of the visible spectrum image encoded into a chrominance component of the combined image and blended image data encoded into a luminance component of the combined image; and

the blended image data comprises the infrared data component blended with a luminance component of the visible spectrum image.

21. The method of claim 15, wherein:

the visible spectrum image is captured by the visible spectrum imaging module while the scene is visibly illuminated; and

the infrared image is captured by the infrared imaging module while the scene is not visibly illuminated.

22. The method of claim 15, wherein:

the visible spectrum image is captured by the visible spectrum imaging module while the scene is visibly illuminated by ambient light or artificial illumination; and

the second time is before the first time.

23. The method of claim 15, wherein:

a first resolution of the infrared image or the visible spectrum image is scaled to approximate a second resolution of a display, a corresponding image, or a specified resolution.

24. The method of claim 15, wherein:

the visible spectrum imaging module and the infrared imaging module are adapted to have approximately a same field of view.

25. The method of claim 15, further comprising:

displaying the combined image to a user.

26. The method of claim 15, wherein:

the infrared imaging module comprises a focal plane array (FPA) configured to capture the infrared image of the scene; and

the FPA comprises an array of microbolometers, the method further comprising;

providing a bias voltage to the microbolometers selected from a range of approximately 0.2 volts to approximately 0.7 volts.

27. The method of claim 26, wherein:

a size of the array of microbolometers is approximately 80 by 60; and

the infrared imaging module is adapted to be inserted into a socket having a size less than approximately 8.5 mm by 8.5 mm.

28. The method of claim 15, wherein:

the infrared image is an unblurred infrared image of the scene, the method further comprising;

receiving an intentionally blurred infrared image of the scene from the infrared imaging module;

determining a plurality of non-uniform correction (NUC) terms based on the intentionally blurred infrared image; and

applying the NUC terms to the unblurred infrared image to remove noise from the unblurred infrared image.

29. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors of a system are adapted to cause the system to perform a method comprising:

receiving a visible spectrum image of a scene captured at a first time by a visible spectrum imaging module comprising a plurality of visible spectrum sensors;

receiving an infrared image of the scene captured at a second time, different from the first time, by an infrared imaging module comprising a plurality of infrared sensors, wherein the infrared image comprises an infrared data component; and

processing the visible spectrum image and the infrared image to generate a combined image comprising visible spectrum characteristics of the scene derived from the visible spectrum image and infrared characteristics of the scene derived from the infrared data component of the infrared image.