Method and System for Removal of Fog, Mist, or Haze from Images and Videos

• US 20140140619A1
• Filed: 02/02/2012
• Published: 05/22/2014
• Est. Priority Date: 08/03/2011
• Status: Active Grant

• Current Assignee: Indian Institute of Technology Kharagpur
• Original Assignee: Indian Institute of Technology Kharagpur
• Inventors: Mukhopadhyay, Sudipta, Tripathi, Abhishek Kumar
• Granted Patent: US 9,197,789 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence

Abstract

A method of removing fog from the images/videos independent of the density or amount of the fog and free of user intervention and a system for carrying out such method of fog removal from images/videos are disclosed. The removal of fog from images and video involve airlight estimation and airlight map refinement based restoration of foggy images and videos. Advantageously, removal of fog from images and videos of this invention would require less execution time and yet achieve high perceptual image quality with reduced noise and enhanced contrast. The proposed method is adapted for RGB Colour model and advantageously also for HSI colour model involving reduced computational requirements and be user friendly and supposed to have wide application and use.

Claims

1. A method for removing fog from images and/or videos comprising:

(a) carrying out a histogram equalization such that a foggy image is equalized to increase contrast;

(b) providing an airlight map of the foggy image and/or video;

(c) refining the airlight map involving anisotropic diffusion and/or bilateral filtering; and

(d) restoring the foggy image based on the refined airlight map involving a restoration process having flexibility to focus on the quality of the image and/or video at a required depth involving an extinction co-efficient for attenuation and airlight and performing a histogram stretching of the restored image if the restored image has low contrast.

2. The method for removing fog from images and/or videos according to claim 1, wherein the images and/or videos include multi component color images and/or videos and/or single component color images, wherein the multi component color images and/or videos includes RGB and Hue, Saturation and Intensity (HSI) images and/or videos, and wherein the single component color images and/or videos include grayscale images and/or videos.

3. The method for removing fog from images and/or videos as according to claim 1, wherein after refined airlight map A is estimated by carrying out restoration of each color component of de-foggy image following:

4. The method for removing fog from images and/or videos according to claim 1, wherein after refined airlight map A is estimated, since fog has no effect on the hue, carrying out restoration based on a HSI color model involving processing carried out only over the saturation and intensity plane with reduced computation following:

5. The method for removing fog from images and/or videos according to claim 1, wherein the airlight map is estimated iteratively involving anisotropic diffusion or non iterative bilateral filter.

6. The method for removing fog from images and/or videos according to claim 5, wherein the refined airlight estimation involving anisotropic diffusion or bilateral filter is carried out perform the smoothing of the intra region and edges remain stable over a very long time whereby diffusion and edge detection is done under a single process adapted to reduce the diffusivity at those locations which have a larger likelihood to be edges.

7. The method for removing fog from images and/or videos according to claim 1, wherein the foggy image estimated refined airlight map depends upon the distance of the scene points from a camera.

8. The method for removing fog from images and/or videos according to claim 1 comprising flexibility to focus on the quality of the image at required depth involving the factor ‘

q’

adapted to control the visual quality of the restored image such that for a lower value of q(q<

1), objects at a lower depth are more enhanced, and on the other hand for a higher value of q(q>

1), the images at a further end are more enhanced.

9. The method for removing fog from images and/or videos according to claim 1 involving different extinction co-efficients (k₁ &

k₂) for attenuation and airlight.

10. The method for removing fog from images and/or videos according to claim 1 comprising involving temporal redundancy, wherein I frame is spatially encoded and P and/or B frame is temporally coded, the temporal coding carried out using motion vectors.

11. The method for removing fog from images and/or videos according to claim 10, wherein the airlight for the I frame is estimated by one or more suitable methods including the anisotropic diffusion and airlight of the P and/or B frame is essentially estimated using previously calculated motion vectors and the airlight estimate(s) of the reference frame(s).

12. The method for removing fog from images and/or videos according to claim 10, comprising regularizing motion vectors for inaccuracies stemming from 3D to 2D mapping of real motion, finite aperture, motion discontinuities and temporal discontinuities.

13. The method for removing fog from images and/or videos according to claim 10, wherein for faster computation, when the frame rate is high and the relative motion between the scene and camera is low, the temporal correlation between constitutive frames is high, the airlight map of the previous frame is used as an estimate of the airlight of the frame under consideration.

14. A system for removing fog from images and/or videos according to the method of claim 1, comprising:

(a) means for carrying out histogram equalization whereby the foggy image is equalized to increase the contrast;

(b) means for generating the airlight map of the foggy image;

(c) means adapted for refining the airlight map involving anisotropic diffusion and/or bilateral filtering; and

(d) means for restoring the foggy image based on the refined airlight map and, optionally, if required, when the restored image has a low contrast adapted to perform histogram stretching of the restored image.

15. (canceled)