Systems and methods of suppressing sky regions in images

• US 9,819,880 B2
• Filed: 12/20/2013
• Issued: 11/14/2017
• Est. Priority Date: 06/03/2009
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Hgasten, Nicholas, Nussmeier, Mark, Kurth, Eric A., Hoelter, Theodore R., Strandemar, Katrin, Boulanger, Pierre, Sharp, Barbara
• Primary Examiner(s): Itskovich, Mikhail
• Application Number:
  US14/136,557
• Publication Number:
  US 20140168445A1
• Time in Patent Office: 1,425 Days
• Field of Search: 348164
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/671 for non-uniformity detection or correction
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Various techniques are provided for systems and methods to process images to reduce consumption of an available output dynamic range by the sky in images. For example, according to one or more embodiments of the disclosure, a region or area in images that may correspond to the sky may be identified based on the location of the horizon in the images. A distribution of irradiance levels in the identified sky region may be analyzed to determine a dynamic range attributable to the sky region. A transfer function that compresses the dynamic range attributable to the sky region may be generated and applied so that the sky in the images may be suppressed, thereby advantageously preserving more dynamic range for terrestrial objects and other objects of interest in the images.

Claims

1. A system comprising:

a memory configured to store an image of a scene including a sky region and a ground region; and

a processor configured to communicate with the memory, the processor configured to;

detect an approximate location of a horizon in the image based on one or more of;

a tilt measurement from an orientation sensor, an edge detection information from processing the image, or pixel coordinates provided by a user;

identify the sky region in the image based on the detected location of the horizon;

assign pixels corresponding to the sky region as sky region pixels and assign remaining pixels as ground region pixels;

receive pixel values indicative of irradiance levels from each of the assigned sky region pixels;

count the number of occurrences of each pixel value of the received pixel values;

select one or more ranges of pixel values having the number of occurrences greater than a predetermined threshold number based on the counted number of occurrences to determine one or more dynamic ranges attributable to the sky region;

generate a transfer function to apply to the entire image including the sky region and the ground region, the generated transfer function being configured to compress the one or more dynamic ranges attributable to the sky region and to expand a dynamic range attributable to the ground region; and

apply the transfer function to the image.

2. The system of claim 1, wherein

the transfer function is a nonlinear gamma curve;

and

the processor is configured to determine an exponent of the gamma curve such that the gamma curve compresses the one or more dynamic ranges attributable to the sky region.

3. The system of claim 1, wherein

the transfer function is a piece-wise linear function having a plurality of line sections;

and

the processor is configured to determine slopes of the line sections such that the piece-wise linear function compresses the one or more dynamic ranges attributable to the sky region.

4. The system of claim 1, wherein the processor is configured to:

automatically detect the approximate location of the horizon in the image.

5. The system of claim 1, further comprising:

an imaging sensor device configured to capture the image of the scene and to provide the captured image to the memory and/or the processor; and

the orientation sensor configured to detect an orientation of the imaging sensor device relative to the horizon, wherein the processor is further configured to automatically detect the approximate location of the horizon in the image based on the orientation detected using the orientation sensor.

6. The system of claim 5, wherein

the image is a thermal image of the scene;

and

the imaging sensor device is a thermal imager comprising a focal plane array (FPA) of infrared sensors and adapted to capture the thermal image.

7. The system of claim 6, wherein the FPA comprises:

a low-dropout regulator (LDO) integrated with the FPA and adapted to provide a regulated voltage in response to an external supply voltage; and

a bias circuit adapted to provide a bias voltage to the infrared sensors in response to the regulated voltage from the LDO.

8. The system of claim 6, wherein:

the thermal imager is configured to capture an intentionally blurred image of the scene, the blurred image comprising noise introduced by the thermal imager; and

the processor is further configured to;

use the blurred image to determine a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, and

apply the NUC terms to the thermal image.

17. The system of claim 1, wherein the selecting of the one or more ranges of pixel values having the number of occurrences greater than the predetermined threshold comprises selecting one or more ranges of pixel values that are dominant in the sky region based on a comparison of the number of occurrences of the pixel values in the sky region with an absolute or relative threshold number.

9. A method comprising:

receiving an image of a scene including a sky region and a ground region;

detecting an approximate location of a horizon in the image based on one or more of;

a tilt measurement from an orientation sensor, an edge detection information from processing the image, or pixel coordinates provided by a user;

identifying the sky region in the image based on the detected location of the horizon;

assigning pixels corresponding to the sky region as sky region pixels and assigning remaining pixels as ground region pixels;

counting the number of occurrences of each pixel value of the received pixel values;

selecting one or more ranges of pixel values having the number of occurrences greater than a predetermined threshold number based on the counted number of occurrences to determine one or more dynamic ranges attributable to the sky region;

generating a transfer function to apply to the entire image including the sky region and the ground region, the generated transfer function compressing the one or more dynamic ranges attributable to the sky region and to expand a dynamic range attributable to the ground region; and

applying the transfer function to the image.

10. The method of claim 9, wherein:

the transfer function is a nonlinear gamma curve; and

the generating the transfer function comprises determining an exponent of the gamma curve such that the gamma curve compresses the one or more dynamic ranges attributable to the sky region.

11. The method of claim 9, wherein

the transfer function is a piece-wise linear function having a plurality of line sections;

and

the generating the transfer function comprises determining slopes of the line sections such that the piece-wise linear function compresses the one or more dynamic ranges attributable to the sky region.

12. The method of claim 9, further comprising automatically detecting the approximate location of the horizon in the image.

13. The method of claim 9, further comprising:

capturing the image of the scene using an imaging sensor device;

detecting an orientation of the imaging sensor device relative to the horizon using the orientation sensor fixed relative to the imaging sensor device; and

automatically detecting the approximate location of the horizon in the image based on the orientation relative to the horizon detected using the orientation sensor.

14. The method of claim 13, wherein

the imaging sensor device is a thermal imager comprising a focal plane array (FPA) of infrared sensors;

and

capturing the image comprises capturing a thermal image of the scene using the thermal imager.

15. The method of claim 14, further comprising:

receiving an external supply voltage at the FPA;

providing a regulated voltage in response to the external supply voltage using a low-dropout regulator (LDO) integrated with the FPA; and

providing a bias voltage from a bias circuit of the FPA to the infrared sensors of the FPA, the bias voltage provided in response to the regulated voltage from the LDO.

16. The method of claim 14, further comprising:

capturing an intentionally blurred image of the scene, the blurred image comprising noise introduced by the thermal imager;

determining a plurality of non-uniformity correction (NUC) terms to reduce at least a portion of the noise, using the blurred image; and

applying the NUC terms to the thermal image.

18. The method of claim 9, wherein the selecting of the one or more ranges of pixel values having the number of occurrences greater than the predetermined threshold comprises selecting one or more ranges of pixel values that are dominant in the sky region based on a comparison of the number of occurrences of the pixel values in the sky region with an absolute or relative threshold number.