Joint contrast enhancement and turbulence mitigation method

• US 9,355,439 B1
• Filed: 11/19/2014
• Issued: 05/31/2016
• Est. Priority Date: 07/02/2014
• Status: Active Grant

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Gibson, Kristofor B.
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US14/547,759
• Time in Patent Office: 559 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method involves generating a denoised image by performing a denoising operation on an input image of an object, the input image having turbulence and a contrast reducing scattering medium therein, generating a contrast enhanced image from the denoised image using a contrast enhancement algorithm, estimating a global motion vector of the contrast enhanced image using an image alignment algorithm, generating a contrast enhanced aligned image by globally aligning the contrast enhanced image to the global motion vector and locally aligning the contrast enhanced image with an optical flow method, temporally averaging the contrast enhanced aligned image, and generating an output image of the object by performing a deblur operation on the temporally averaged contrast enhanced aligned image. The method may further include measuring turbulence using sequences of input images and output images.

Claims

1. A method comprising the steps of:

generating a denoised image by performing a denoising operation on an input image of an object ô

_k(x), the input image having turbulence and a contrast reducing scattering medium therein;

generating a contrast enhanced image from the denoised image using a contrast enhancement algorithm;

estimating a global motion vector {circumflex over (Δ

)}_g,k of the contrast enhanced image using an image alignment algorithm;

generating a contrast enhanced aligned image by globally aligning the contrast enhanced image to {circumflex over (Δ

)}_g,k and locally aligning the contrast enhanced image with an optical flow method;

temporally averaging the contrast enhanced aligned image with K frames;

generating an output image of the object ô

_k(x) by performing a deblur operation on the temporally averaged contrast enhanced aligned image; and

measuring turbulence using a turbulence mitigation metric (TMM).

2. The method of claim 1, wherein the step of temporally averaging the contrast enhanced aligned image with K frames includes the step of automatically determining K using a previously measured TMM.

3. The method of claim 1, wherein the TMM is a measurement between an input image sequence i and an output image sequence o, where each of i and o are of spatial dimension M₁×

M₂, have three color channels, and are of length K frames.

4. The method of claim 1, wherein the denoising operation is performed by a two-dimensional median filter.

5. The method of claim 1, wherein the contrast enhancement algorithm is a locally adaptive wiener filter.

6. The method of claim 1, wherein the image alignment algorithm is a minimum output sum of squared error adaptive correlation filter.

7. The method of claim 1, wherein the step of temporally averaging the contrast enhanced aligned image comprises determining a lower bound for, K_min, according to the equation

8. The method of claim 1, wherein the optical flow method is a dense optical flow block-based method, wherein every pixel has a motion vector associated with it, wherein the motion vectors are used to warp the images to be aligned with each other.

9. A system comprising:

a processor operatively connected to an imaging device, wherein the processor is configured, via programming instructions accessible thereto, to perform the steps of;

generating a denoised image by performing a denoising operation on an input image of an object ô

_k(x), the input image having turbulence and a contrast reducing scattering medium therein;

generating a contrast enhanced image from the denoised image using a contrast enhancement algorithm;

estimating a global motion vector {circumflex over (Δ

)}_g,k of the contrast enhanced image using an image alignment algorithm;

generating a contrast enhanced aligned image by globally aligning the contrast enhanced image to {circumflex over (Δ

)}_g,k and locally aligning the contrast enhanced image with an optical flow method;

temporally averaging the contrast enhanced aligned image with K frames;

generating an output image of the object ô

_k(x) by performing a deblur operation on the temporally averaged contrast enhanced aligned image; and

measuring turbulence using a turbulence mitigation metric (TMM).

10. The system of claim 9, wherein the step of temporally averaging the contrast enhanced aligned image with K frames includes the step of automatically determining K using a previously measured TMM.

11. The system of claim 9, wherein the TMM is a measurement between an input image sequence i and an output image sequence o, where each of i and o are of spatial dimension M₁×

M₂, have three color channels, and are of length K frames.

12. The system of claim 9, wherein the denoising operation is performed by a two-dimensional median filter.

13. The system of claim 9, wherein the contrast enhancement algorithm is a locally adaptive wiener filter.

14. The system of claim 9, wherein the image alignment algorithm is a minimum output sum of squared error adaptive correlation filter.

15. The system of claim 9, wherein the step of temporally averaging the contrast enhanced aligned image comprises determining a lower bound for, K_min, according to the equation

16. The system of claim 9, wherein the optical flow method is a dense optical flow block-based method, wherein every pixel has a motion vector associated with it, wherein the motion vectors are used to warp the images to be aligned with each other.