Infrared camera systems and methods for dual sensor applications

• US 8,749,635 B2
• Filed: 06/03/2009
• Issued: 06/10/2014
• Est. Priority Date: 06/03/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hgasten, Nicholas, Scott, Jeffrey S., Richardson, Patrick B., Frank, Jeffrey D., Richards, Austin A., Woolaway, James T.
• Primary Examiner(s): TRUONG, LAN DAI T
• Application Number:
  US12/477,828
• Publication Number:
  US 20100309315A1
• Time in Patent Office: 1,833 Days
• Field of Search: 709/164, 709/175, 709/177, 709/179, 709/191, 709/211.1, 709/224.1, 348/164, 348/175, 348/177, 348/179, 348/191, 348/211.1, 348/224.1
• US Class Current: 348/164
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods disclosed herein provide for infrared camera systems and methods for dual sensor applications. For example, in one embodiment, an enhanced vision system comprises an image capture component having a visible light sensor to capture visible light images and an infrared sensor to capture infrared images. The system comprises a first control component adapted to provide a plurality of selectable processing modes to a user, receive a user input corresponding to a user selected processing mode, and generate a control signal indicative of the user selected processing mode, wherein the plurality of selectable processing modes includes a visible light only mode, infrared only mode, and a combined visible-infrared mode. The system comprises a processing component adapted to receive the generated control signal from the control component, process the captured visible light images and the captured infrared images according to the user selected processing mode, and generate processed images based on the processing mode selected by the user. The system comprises a display component adapted to display the processed images based on the processing mode selected by the user.

Claims

1. An enhanced vision system comprising:

a visible light sensor adapted to capture a visible light image;

an infrared sensor adapted to capture an infrared image, wherein the infrared image is a thermal infrared image comprising thermal information from an infrared wavelength spectrum;

a control component adapted to receive a user input corresponding to a user selected control parameter;

a processing component adapted to extract a luminance part from the captured visible light image, extract a luminance part from the captured infrared image, scale the luminance parts based on the control parameter, and merge the scaled luminance parts to generate an output image; and

a display component adapted to display the output image.

2. The system of claim 1, wherein:

the visible light sensor is adapted to capture one or more visible light images and provide visible light sensor data related to the captured visible light images,

the infrared sensor is adapted to capture one or more infrared images and provide infrared sensor data related to the captured infrared images,

the processing component is adapted to receive and process the visible light sensor data and the infrared sensor data to generate output image data, and

the display component is adapted to display one or more output images based on the output image data generated by the processing component.

3. The system of claim 2, wherein the processing component is adapted to extract a luminance data part from the visible light sensor data, extract a luminance data part from the infrared sensor data, scale the luminance data parts, and merge the scaled luminance data parts to generate the output image data.

4. The system of claim 1, further comprising:

a control component adapted to provide a plurality of selectable processing modes and generate a control signal indicative of a selected processing mode, wherein the plurality of selectable processing modes includes a night cruising mode, a day cruising mode, and a man overboard mode; and

wherein the processing component is adapted to receive the generated control signal from the control component and process the captured infrared image according to the selected processing mode.

5. The system of claim 4, further comprising:

a memory component adapted to store one or more of the captured visible light image, the captured infrared image, and the output image;

wherein the plurality of selectable processing modes includes a hazy conditions mode that causes the processing component to apply a non-linear low pass filter on the captured infrared image, and then histogram equalize and scale the captured infrared image; and

wherein the display component is adapted to display the output image in a red color palette or a green color palette in a night display mode.

6. The system of claim 1, further comprising:

a control component adapted to provide a plurality of selectable processing modes to a user, receive user input corresponding to a user selected processing mode, and generate a control signal indicative of the user selected processing mode;

wherein the plurality of selectable processing modes includes a visible light only mode, infrared only mode, and at least one combined visible-infrared mode;

wherein the processing component is adapted to receive the generated control signal from the first control component, process the captured visible light images and the captured infrared images according to the user selected processing mode, and generate one or more output images based on the processing mode selected by the user; and

wherein the display component is adapted to display the output images based on the processing mode selected by the user.

7. The system of claim 6, wherein the plurality of selectable processing modes includes the visible light only mode that causes the processing component to generate the processed images from only the captured visible light images, and wherein the visible light images comprise color information or monochrome information from the visible light wavelength spectrum.

8. The system of claim 6, wherein the plurality of selectable processing modes includes the infrared only mode that causes the processing component to generate the processed images from only the captured infrared images.

9. The system of claim 6, wherein the plurality of selectable processing modes includes the combined visible-infrared mode that causes the processing component to generate the output image by combining the scaled luminance part of the captured visible light image with the scaled luminance part from the captured infrared image.

10. The system of claim 1, wherein the control component is adapted to generate a control signal indicative of the user selected control parameter.

11. The system of claim 10, wherein the selectable control parameter provides a selectable range of values, and wherein a first value causes the processing component to generate the processed image from only the captured visible light images, and wherein a second value causes the processing component to generate the processed images from only the captured infrared images, and wherein one or more other values between the first and second values causes the processing component to generate the processed images from proportional values of the luminance parts of the captured visible light images and the captured infrared images, and wherein the selectable control parameter is adjusted to increase or decrease a gain value of the luminance parts.

12. The system of claim 1, wherein the display component is adapted to display the output image as at least one of a visible light only image, an infrared only image, and a combined visible-infrared image having portions of both the visible light only image and the infrared only image.

13. The system of claim 1, wherein the captured visible light image comprises the luminance part and a chrominance part, and wherein the captured infrared image comprises only a luminance part, and wherein the processing component is adapted to extract the luminance part and the chrominance part from the captured visible light image, extract only the luminance part from the captured infrared image, scale the luminance parts, and merge the luminance parts and the chrominance part to generate the output image.

14. A method comprising:

obtaining visible light sensor data;

obtaining infrared sensor data, wherein the infrared sensor data comprises thermal information from an infrared wavelength spectrum;

receiving a user input corresponding to a user selected control parameter;

extracting a luminance data part from the visible light sensor data;

extracting a luminance data part from the infrared sensor data;

scaling the luminance data parts based on the control parameter;

merging the scaled luminance data parts to generate output image data; and

displaying an output image based on the output image data.

15. The method of claim 14, further comprising:

capturing one or more visible light images and providing the visible light sensor data related to the captured visible light images;

capturing one or more infrared images and providing the infrared sensor data related to the captured infrared images;

providing a plurality of selectable processing modes to a user, wherein the plurality of selectable processing modes includes a visible light only mode, infrared only mode, and at least one combined visible-infrared mode;

receiving a user input corresponding to a user selected processing mode;

processing the captured visible light images and the captured infrared images based on the user selected processing mode;

generating one or more output images based on the user selected processing mode; and

displaying the one or more output images based on the user selected processing mode.

16. The method of claim 15, wherein:

the one or more output images comprise visible light only images when the visible light only mode is selected by the user,

the one or more output images comprise infrared only images when the infrared only mode is selected by the user, and

the one or more output images comprise combined visible-infrared images when the combined visible-infrared mode is selected by the user.

17. The method of claim 16, wherein:

the visible light images comprise color information or monochrome information from the visible light wavelength spectrum, and

the combined visible-infrared images comprise blended luminance parts of the captured visible light images and the captured infrared images.

18. The method of claim 15, further comprising:

generating a control signal indicative of the user selected control parameter,

wherein the selectable control parameter comprises a value in the range of 0 (zero) to 1,

wherein a value of 1 is used to generate the one or more output images from only the captured visible light images,

wherein a value of 0 (zero) is used to generate the one or more output images from only the captured infrared images,

wherein values between 0 (zero) and 1 is used to generate the one or more output image from proportional parts of both the captured visible light images and the captured infrared images, and

wherein the selectable control parameter is adjustable to increase or decrease a gain value of the captured visible light images and the captured infrared images.

19. The method of claim 14, wherein displaying the output image comprises displaying at least one of a visible light only image, an infrared only image, and a combined visible-infrared image having portions of both the visible light only image and the infrared only image.

20. The method of claim 14, further comprising:

storing one or more of the visible light sensor data, the infrared sensor data, and the output image data;

providing a plurality of selectable processing modes, wherein the plurality of selectable processing modes includes a night docking mode, a man overboard mode, and a night cruising mode; and

processing the infrared sensor data according to one of the selectable processing modes.

21. The method of claim 14, wherein the visible light sensor data comprises a luminance data part and a chrominance data part, and wherein the infrared sensor data comprises only a luminance data part, and wherein the method further comprises extracting the luminance data part and the chrominance data part from the visible light sensor data, extracting the luminance data part from the infrared sensor data, scaling the luminance parts, and merging the luminance parts and the chrominance part to generate the output image data.

22. A non-transitory computer-readable medium on which is stored information for performing a method, the method comprising:

obtaining visible light sensor data;

obtaining infrared sensor data, wherein the infrared sensor data comprises thermal information from an infrared wavelength spectrum;

receiving a user input corresponding to a user selected control parameter;

extracting a luminance data part from the visible light sensor data;

extracting a luminance data part from the infrared sensor data;

scaling the luminance data parts based on the control parameter;

merging the scaled luminance data parts to generate output image data; and

displaying an output image based on the output image data.

23. The non-transitory computer-readable medium of claim 22, wherein the visible light sensor data comprises color information or monochrome information from the visible light wavelength spectrum, and wherein the infrared sensor data comprises thermal information from the infrared wavelength spectrum, and wherein the output image is generated by combining a part of the visible light sensor data with a part from the infrared sensor data.

24. The non-transitory computer-readable medium of claim 22, wherein the visible light sensor data comprises the luminance data part and a chrominance data part, and wherein the infrared image data comprises only the luminance data part, and wherein the method further comprises extracting the chrominance data part from the visible light sensor data, and merging the scaled luminance data parts and the chrominance data part to generate the output image based on a processing mode selected by the user.