Time spaced infrared image enhancement

• US 10,033,944 B2
• Filed: 09/19/2016
• Issued: 07/24/2018
• Est. Priority Date: 03/02/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hogasten, Nicholas, Dumpert, Dwight, Hoelter, Theodore R., Scott, Jeffrey S., Strandemar, Katrin, Nussmeier, Mark, Kurth, Eric A., Boulanger, Pierre, Sharp, Barbara
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US15/269,905
• Publication Number:
  US 20170078590A1
• Time in Patent Office: 673 Days
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Triple fusion processing, including selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.

Claims

1. A system comprising:

a first imaging module comprising a first plurality of sensors configured to capture a first image of a scene;

a second imaging module comprising a second plurality of sensors configured to capture a second image of the scene; and

a processor configured to communicate with the first imaging module and the second imaging module and to process the first image and the second image to generate a combined image comprising first characteristics of the scene derived from the first image and second characteristics of the scene derived from the second image;

wherein;

the first plurality of sensors comprises a plurality of visible spectrum sensors configured to capture a visible spectrum image of the scene and the first image of the scene comprises the visible spectrum image of the scene; and

the processor and/or the first imaging module are configured to transmit, receive, store, and/or process the visible spectrum image of the scene according to an RGB color space, a YCbCr color space, a YUV color space, a color space derived through conversion from the RGB, YCbCr, and/or YUV color spaces, and/or a color space comprising an intensity component, a brightness component, and/or other color space components.

2. The system of claim 1, wherein:

the second plurality of sensors comprises a plurality of infrared sensors configured to capture a short wave infrared (SWIR) image of the scene and the second image of the scene comprises an SWIR image of the scene; and

the first characteristics of the scene comprise visible spectrum characteristics of the scene and the second characteristics of the scene comprise SWIR characteristics of the scene.

3. The system of claim 1, further comprising a display and a third imaging module comprising a plurality of infrared sensors configured to capture a thermal image of the scene, wherein:

the second imaging module comprises a plurality of non-thermal sensors configured to capture a non-thermal image of the scene and the second image of the scene comprises the non-thermal image of the scene;

the combined image comprises a first combined image; and

the processor is configured to communicate with the third imaging module, and to process the second image and the thermal image to generate a second combined image comprising non-thermal characteristics of the scene derived from the non-thermal image and thermal characteristics of the scene derived from the thermal image, wherein the display is configured to present the first combined image or the second combined image to a user.

4. The system of claim 3, wherein:

the plurality of non-thermal sensors of the second imaging module comprises a plurality of infrared sensors configured to capture a short wave infrared (SWIR) image of the scene and the second image of the scene comprises an SWIR image of the scene; and

the non-thermal characteristics of the scene comprise SWIR characteristics of the scene.

5. The system of claim 1, wherein:

the first imaging module is configured to capture the first image of the scene at a first time while the scene is sufficiently illuminated with visible light to capture visible light images of the scene including visible spectrum image detail of an object in the scene; and

the second image module is configured to capture the second image of the scene at a second time, different from the first time, while the scene is not sufficiently illuminated with visible light to capture visible light images of the scene including visible spectrum image detail of the object in the scene.

6. The system of claim 5, wherein:

the first and/or second imaging modules comprise one or more pan, tilt, and/or zoom features.

7. The system of claim 5, wherein:

the first plurality of visible spectrum sensors are configured to capture a first visible spectrum image of the scene according to a first optical axis of the first imaging module;

the second plurality of sensors comprises a second plurality of non-visible spectrum sensors configured to capture a non-visible spectrum image of the scene according to a second optical axis of the second imaging module;

the system further comprises a third imaging module comprising a second plurality of visible spectrum sensors configured to capture a second visible spectrum image of the scene according to a third optical axis of the third imaging module, wherein the first optical axis is different from the second optical axis or the second optical axis is different from the third optical axis.

8. The system of claim 7, wherein:

the first, second, and/or third imaging modules comprise one or more pan, tilt, and/or zoom features.

9. The system of claim 1, wherein:

the second plurality of sensors comprises a plurality of infrared sensors configured to capture a thermal image of the scene, the second image of the scene comprises the thermal image of the scene, and the thermal image is an unblurred thermal image of the scene;

the second imaging module is configured to capture an intentionally blurred infrared image of the scene; and

the processor is configured to determine a plurality of non-uniform correction (NUC) terms based on the intentionally blurred infrared image and apply the NUC terms to the unblurred infrared image to remove noise from the unblurred infrared image.

10. A method comprising:

receiving a first image of a scene captured by a first imaging module comprising a first plurality of sensors;

receiving a second image of the scene captured by a second imaging module comprising a second plurality of sensors;

processing the first image and the second image to generate a combined image comprising first characteristics of the scene derived from the first image and second characteristics of the scene derived from the second image, wherein the first plurality of sensors comprises a plurality of visible spectrum sensors configured to capture a visible spectrum image of the scene and the first image of the scene comprises the visible spectrum image of the scene; and

transmitting, receiving, storing, and/or processing the visible spectrum image of the scene according to an RGB color space, a YCbCr color space, a YUV color space, a color space derived through conversion from the RGB, YCbCr, and/or YUV color spaces, and/or a color space comprising an intensity component, a brightness component, and/or other color space components.

11. The method of claim 10, wherein:

the second plurality of sensors comprises a plurality of infrared sensors configured to capture a short wave infrared (SWIR) image of the scene and the second image of the scene comprises an SWIR image of the scene; and

the first characteristics of the scene comprise visible spectrum characteristics of the scene and the second characteristics of the scene comprise SWIR characteristics of the scene.

12. The method of claim 10, wherein:

the second imaging module comprises a plurality of non-thermal sensors configured to capture a non-thermal image of the scene and the first second image of the scene comprises the non-thermal image of the scene;

the combined image comprises a first combined image; and

the method further comprises;

processing the second image and a thermal image of the scene captured by a third imaging module comprising a plurality of infrared sensors configured to capture the thermal image of the scene to generate a second combined image comprising non-thermal characteristics of the scene derived from the non-thermal image and thermal characteristics of the scene derived from the thermal image; and

displaying the first combined image or the second combined image to a user.

13. The method of claim 12, wherein:

the plurality, of non-thermal sensors of the second imaging module comprises a plurality of infrared sensors configured to capture a short wave infrared (SWIR) image of the scene and the second image of the scene comprises an SWIR image of the scene; and

the non-thermal characteristics of the scene comprise SWIR characteristics of the scene.

14. The method of claim 10, wherein:

the first imaging module is configured to capture the first image of the scene at a first time while the scene is sufficiently illuminated with visible light to capture visible light images of the scene including visible spectrum image detail of an object in the scene; and

the second image module is configured to capture the second image of the scene at a second time, different from the first time, while the scene is not sufficiently illuminated with visible light to capture visible light images of the scene including visible spectrum image detail of the object in the scene.

15. The method of claim 14, wherein:

the first and/or second imaging modules comprise one or more pan, tilt, and/or zoom features.

16. The method of claim 14, wherein:

the first plurality of visible spectrum sensors are configured to capture a first visible spectrum image of the scene according to a first optical axis of the first imaging module;

the second plurality of sensors comprises a second plurality of non-visible spectrum sensors configured to capture a non-visible spectrum image of the scene according to a second optical axis of the second imaging module;

the method further comprises receiving a second visible spectrum image of the scene captured by a third imaging module comprising a second plurality of visible spectrum sensors configured to capture the second visible spectrum image of the scene according to a third optical axis of the third imaging module, wherein the first optical axis is different from the second optical axis or the second optical axis is different from the third optical axis.

17. The method of claim 16, wherein:

the first, second, and/or third imaging modules comprise one or more pan, tilt, and/or zoom features.

18. The method of claim 10, wherein:

the second plurality of sensors comprises a plurality of infrared sensors configured to capture a thermal image of the scene, the second image of the scene comprises the thermal image of the scene, and the thermal image is an unblurred thermal image of the scene; and

the method further comprises;

receiving an intentionally blurred thermal image of the scene from the second imaging module;

determining a plurality of non-uniform correction (NUC) terms based on the intentionally blurred infrared image; and

applying the NUC terms to the unblurred infrared image to remove noise from the unblurred infrared image.

19. A system comprising:

a display;

a first imaging module comprising a first plurality of sensors configured to capture a first image of a scene;

a second imaging module comprising a second plurality of sensors configured to capture a second image of the scene;

a third imaging module comprising a plurality of infrared sensors configured to capture a thermal image of the scene; and

a processor configured to communicate with the first imaging module and the second imaging module and to process the first image and the second image to generate a combined image comprising first characteristics of the scene derived from the first image and second characteristics of the scene derived from the second image, wherein;

the first imaging module comprises a plurality of non-thermal sensors configured to capture a non-thermal image of the scene and the first image of the scene comprises the non-thermal image of the scene;

the combined image comprises a first combined image; and

the processor is configured to communicate with the third imaging module and to process the first image and the thermal image to generate a second combined image comprising the first characteristics of the scene derived from the non-thermal image and thermal characteristics of the scene derived from the thermal image, wherein the display is configured to present the first or the second combined images to a user.