Imaging through aerosol obscurants

• US 9,659,237 B2
• Filed: 10/04/2013
• Issued: 05/23/2017
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Micro Biz Ventures LLC
• Original Assignee: Micro USA, Inc.
• Inventors: Puetter, Richard C., Junkkarinen, Vesa
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/046,835
• Publication Number:
  US 20160217346A1
• Time in Patent Office: 1,327 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

A method for determining a level of haze within a field of view of an image sensor adapted for detecting a plurality of color components includes collecting a first image of the field of view at a first point in time, defining regions of the first image as a reference and determining reference intensities for each color component. At least one second image is collected at different points in time and intensities for each color component are determined and compared against the reference intensities. If the intensities of second image differ from the reference intensities by more than a predetermined threshold, all or a portion of the second image is designated as being obscured by haze and requiring correction.

Claims

1. A method for determining a level of haze within a field of view of an image sensor adapted for detecting one or more color components, the method comprising:

(a) collecting a first image of one or more objects within the field of view at a first point in time at which a first set of haze conditions are present;

(b) defining one or more regions of the first image as a reference;

(c) determining reference intensities for each of the one or more color components for the reference;

(d) collecting at least one second image of one or more objects within the field of view at one or more second points in time at which a different second set of haze conditions are present;

(e) determining intensities for each of the one or more color components within the at least one second image;

(f) comparing the intensities for each of the one or more color components of the at least one second image against the reference intensities;

(g) if the intensities of the at least one second image differs from the reference intensity by more than a predetermined threshold, designating all or a portion of the at least one second image as being obscured by haze; and

(h) generating an output identifying the designated portions of the at least second image as requiring correction.

2. The method of claim 1, wherein the one or more regions of the first image correspond to one or more stationary objects within the field of view, and wherein optical properties of the one or more stationary objects and distances between the image sensor and the one or more stationary objects are at least approximately known.

3. The method of claim 2, wherein the step of determining reference intensities further comprises calculating an optical depth to each of the one or more stationary objects.

4. The method of claim 3, wherein the step of determining intensities for each of the one or more color components within the at least one second image comprises calculating an optical depth to each of the one or more stationary objects at the one or more second points in time, and the step of comparing includes determining a change in the optical depth to determine a level of obscuration at the one or more second points in time.

5. The method of claim 1, wherein the first set of haze conditions comprise ideal conditions or correctable to near-ideal conditions.

6. The method of claim 1, wherein the one or more color components comprise a plurality of colors of a standard color model.

7. The method of claim 6, wherein the standard color model is RGB.

8. The method of claim 1, further comprising processing the output to remove effects of the haze from the image.

9. The method of claim 1, wherein the step of defining one or more regions of the first image comprises segmenting the first image into a plurality of tiles to define a plurality of reference tiles.

10. The method of claim 9, wherein the step of determining reference intensities comprises normalizing intensity values for each of the one or more color components for each region and generating a cumulative histogram of intensity for each of the one or more color components to define a reference set of histograms.

11. The method of claim 9, wherein the step of determining intensities for each of the one or more color components within the at least one second image comprises segmenting the at least one second image into the plurality of tiles, normalizing intensity values for each color component for each tile and generating a cumulative histogram of intensity for each color component to define a second set of histograms, wherein the second set of histograms is compared against the reference set of histograms.

12. The method of claim 11, further comprising replacing the reference set of histograms with the second set of histograms if the step of comparing indicates decreased haze conditions in the at least one second image.

13. The method of claim 12, wherein decreased haze conditions correspond to a broader histogram.

14. A non-transitory machine-readable medium comprising a plurality of instructions, that in response to be executed, result in a processor executing the steps of:

(a) collecting a first image of one or more objects within the field of view of an image sensor at a first point in time at which a first set of haze conditions are present, wherein the image sensor is adapted to detect one or more color components;

(b) defining one or more regions of the first image as a reference;

(c) determining reference intensities for each of the one or more color components for the reference;

(d) collecting at least one second image of one or more objects within the field of view at one or more second points in time at which a different second set of haze conditions are present;

(e) determining intensities for each of the one or more color components within the at least one second image;

(f) comparing the intensities for each of the one or more color components of the at least one second image against the reference intensities;

(g) if the intensities of the at least one second image differs from the reference intensity by more than a predetermined threshold, designating all or a portion of the at least one second image as being obscured by haze; and

(h) generating an output identifying the designated portions of the at least second image as requiring correction.

15. The non-transitory machine readable medium of claim 14, wherein the one or more regions of the first image correspond to one or more stationary objects within the field of view, and wherein optical properties of the one or more stationary objects and distances between the image sensor and the one or more stationary objects are at least approximately known.

16. The non-transitory machine readable medium of claim 15, wherein the step of determining reference intensities further comprises calculating an optical depth to each of the one or more stationary objects.

17. The non-transitory machine readable medium of claim 16, wherein the step of determining intensities for each color component within the at least one second image comprises calculating an optical depth to each of the one or more stationary objects at the one or more second points in time, and the step of comparing includes determining a change in the optical depth to determine a level of obscuration at the one or more second points in time.

18. The non-transitory machine readable medium of claim 14, wherein the first set of haze conditions comprise ideal conditions or correctable to near-ideal conditions.

19. The non-transitory machine readable medium of claim 14, wherein the one or more color components comprise a plurality of colors of a standard color model.

20. The non-transitory machine readable medium of claim 19, wherein the standard color model is RGB.

21. The non-transitory machine readable medium of claim 14, further comprising processing the output to remove effects of the haze from the image.

22. The non-transitory machine readable medium of claim 14, wherein the step of defining one or more regions of the first image comprises segmenting the first image into a plurality of tiles to define a plurality of reference tiles.

23. The non-transitory machine readable medium of claim 22, wherein the step of determining reference intensities comprises normalizing intensity values for each color component for each region and generating a cumulative histogram of intensity for each color component to define a reference set of histograms.

24. The non-transitory machine readable medium of claim 22, wherein the step of determining intensities for each of the one or more color components within the at least one second image comprises segmenting the at least one second image into the plurality of tiles, normalizing intensity values for each of the one or more color components for each tile and generating a cumulative histogram of intensity for each of the one or more color components to define a second set of histograms, wherein the second set of histograms is compared against the reference set of histograms.

25. The non-transitory machine readable medium of claim 24, further comprising replacing the reference set of histograms with the second set of histograms if the step of comparing indicates decreased haze conditions in the at least one second image.

26. The non-transitory machine readable medium of claim 25, wherein decreased haze conditions correspond to a broader histogram.

27. A method for determining a level of haze within a field of view of an image sensor, the method comprises:

detecting one or more color components by collecting a first image of one or more objects within the field of view at a first point in time at which a first set of haze conditions are present;

defining one or more regions of the first image by segmenting the first image into a plurality of tiles to define a plurality of reference tiles;

determining reference intensities for each color component by normalizing intensity values for each of the one or more color components for each reference tile and generating a cumulative histogram of intensity for each of the one or more color components to define a reference set of histograms;

collecting at least one second image of one or more objects within the field of view at one or more second points in time at which a different second set of haze conditions are present;

determining intensities for each of the one or more color components within the at least one second image by segmenting the at least one second image into the plurality of tiles, normalizing intensity values for each of the one or more color components for each tile and generating a cumulative histogram of intensity for each of the one or more color components to define a second set of histograms;

comparing the second set of histograms against the reference set of histograms;

if the second set of histograms differs from the reference set of histograms by more than a predetermined threshold, designating one or more tiles within the at least one second image as being obscured by haze; and

generating an output identifying the one or more tiles of the at least second image as requiring correction.

28. The method of claim 27, further comprising replacing the reference set of histograms with the second set of histograms if the step of comparing indicates decreased haze conditions in the at least one second image, wherein decreased haze conditions are determined based on the relative widths of the histograms.