System for recovery of degraded images

• US 6,459,818 B1
• Filed: 03/01/1999
• Issued: 10/01/2002
• Est. Priority Date: 03/01/1999
• Status: Expired due to Term

• Current Assignee: University of Rochester
• Original Assignee: University of Rochester
• Inventors: George, Nicholas
• Primary Examiner(s): Lee, Thomas D
• Assistant Examiner(s): BRINICH, STEPHEN M
• Application Number:
  US09/259,774
• Time in Patent Office: 1,310 Days
• Field of Search: 382/254, 382/260-264, 382/265-266
• US Class Current: 382/254
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

A system for recovering degraded images captured through atmospheric turbulence, or other atmospheric inhomogeneities, such as snow, rain, smoke, fog, or underwater fluctuations, is provided having an imager for capturing through such turbulence both a degraded image of a scene having at least one object, and an image of a point source associated with the object. The imager converts the degraded image into first image data signals representing the degraded image, and converts the image of the point source into second image data signals representing a point spread function. A computer of the system receives the first and second image data signals and produces third image data signals representing a recovered image of the object of the degraded image in accordance with the first and second image data signals. In another embodiment, the imager captures a degraded image through atmospheric turbulence of a scene having a known reference object and an unknown object, and converts the degraded image into first image data signals. After receiving the first image data signals, the computer identifies in the first image data signals such image data signals representing the reference object. The computer produces image data signals representing a recovered image of the degraded image in accordance with the first image data signals, the image data signals representing the reference in the first image data signals, and image data signals representing an undegraded image of the reference. The computer may output the image data signals representing a recovered image to an output device to display or print the recovered image.

Claims

1. A system for recovering images of high quality from degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having at least one object and second image data signals representing a point spread function of an image of a point source associated with said object in the degraded image;

means for producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals, wherein said capturing means comprises;

optics for receiving light representing an image having both said degraded image and said image of said point source;

means for separating said light representing said image of said point source image from said light representing said degraded image;

means for detecting said light representing said degraded image to provide said first image data signals; and

means for detecting said light representing said image of said point source to provide said second image data signals.

2. A system for recovering images of high quality from degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having at least one object and second image data signals representing a point spread function of an image of a point source associated with said object in the degraded image;

means for producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals, wherein said first image data signals defines u₂(x, y), said second image data signals define z(x,y), and said producing means comprises;

means for determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y) and the Fourier transform of said z(x,y) to provide Z(f_x,f_y);

means for dividing said U₂(f_x,f_y) by said Z(f_x,f_y) to provide U₁(f_x,f_y); and

.

means for determining the inverse Fourier transform of said U₁(f_x,f_y) to provide u₁(x,y) in which said u₁(x,y) represents said third data signals.

3.The system according to claim 2 wherein said dividing means comprising means for dividing said U₂(f_x,f_y) by said Z(f_x,f_y) to provide an estimate U_1e(f_x,f_y) of said U₁(f_x,f_y) in accordance with the equation $U_{1<br><br>e}<br><br>\left(f_x,f_y\right)=U_2<br><br>\left(f_x,f_y\right)<br><br>\frac{Z^{*}}{Z<br><br>\text{\hspace{1em}<br><br>}<br><br>Z^{*}+\frac{1}{256}}$

in which Z* is the complex conjugate of Z(f_x,f_y) and ZZ* is the product of Z(f_x,f_y) and Z*, and said means for determining the inverse Fourier transform determines the inverse Fourier transform of said U_1e(f_x,f_y) to provide u_1e(x,y) in which said u_1e(x,y) represents said third data signals.

3. A system for recovering images of high quality from degraded images comprising.

means for capturing an image to provide first image data signals representing a degraded image of a scene having at least one object and second image data signals representing a point spread function of an image of a point source associated with said object in the degraded image;

means for producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals; and

multiple ones of said second image data signals represent a point spread function of an image of a different point source associated with different parts of the degraded image, wherein said producing means further comprises means for producing, for each of said multiple ones of said second image data signals, third image data signals each representing at least part of the recovered image from said degraded image in accordance with said first image data signals and the second image data signals.

4. The system according to claim 4 further comprising means for producing a composite recovered image from each of the third image data signals produced in accordance with each of said multiple ones of second image data signals.

5. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference; and

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said capturing means further comprises means for capturing the undegraded image of the reference to provide said third image data signals.

6. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference; and

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said first, second, third, and fourth image data signals are two-dimensional and in one or more color channels.

7. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference;

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals; and

means for determining the part of said captured image representing said image of said reference.

8. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference; and

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said first image data signals define u₂(x,y), said second image data signals define U₂^r(X,y), and said third image data signals define u₁^r(x,y), and said producing means comprises;

means for determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y), the Fourier transform of said u₂^r(X,y) to provide U₂^r(f_x,f_y), and said u₁^r(x,y) to provide U₁^r(f_x,f_y);

means for calculating U₁^r(f_x,f_y) multiplied by U₂(f_x,f_y) divided by U₂^r( f_x,f_y) to provide U_1e(f_x,f_y); and

means for determining the inverse Fourier transform of said U_1e(f_x,f_y) to provide U_1e(x,y) in which said u_1e(x,y) represents said fourth data signals.

9. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference;

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals; and

an output device for outputting an image in accordance with said fourth image data signals representing a recovery of the degraded image.

10. A system for recovering degraded images comprising:

means for capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

means for storing third image data signals representing an undegraded image of the reference;

means for producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals; and

multiple ones of said second image data signals each representing an image of a reference object in a different part of the degraded image, and multiple ones of said third image data signals representing an undegraded image of the reference object associated with each of the multiple ones of said second image data signals, wherein said producing means further comprises means for producing, for each of said multiple ones of second image data signals, fourth image data signals each representing a recovered image from said degraded image in accordance with said first image data signals, the second image data signals, and the third image data signals associated with the second image data signals.

11. A method for recovering images of high quality from degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having at least one object;

capturing an image to provide second image data signals representing a point spread function of an image of a point source associated in said scene with said object in the degraded image; and

producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals, wherein said capturing step comprises the steps of;

receiving light representing an image having both said degraded image and said image of said point source;

separating said light representing said image of said point source image from said light representing said degraded image;

detecting said light representing said degraded image to provide said first image data signals; and

detecting said light representing said image of said point source to provide said second image data signals.

12. A method for recovering images of high quality from degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having at least one object;

capturing an image to provide second image data signals representing a point spread function of an image of a point source associated in said scene with said object in the degraded image; and

producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals, wherein said first image data signals defines u₂(x,y), said second image data signals define z(x,y), and said producing step comprises the steps of;

determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y) and the Fourier transform of said z(x,y) to provide Z(f_x,f_y);

dividing said U₂(f_x,f_y) by said Z(f_x,f_y) to provide at least an estimate of U₁(f_x, f_y); and

determining the inverse Fourier transform of said U₁(f_x,f_y) to provide u₁(x,y) in which said u₁(x,y) represents said third data signals.

13. A method for recovering images of high quality from degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having at least one object;

capturing an image to provide second image data signals representing a point spread function of an image of a point source associated in said scene with said object in the degraded image; and

producing third image data signals representing a recovered image of said object from said degraded image in accordance with said first image data signals and said second image data signals, wherein multiple ones of said second image data signals represent a point spread function of an image of a different point source associated with different parts of the degraded image, and said producing step further comprises the step of producing, for each of said multiple ones of second image data signals, third image data signals each representing at least part of the recovered image from said degraded image in accordance with said first image data signals and the second image data signals.

14. The method according to claim 14 further comprising the step of producing a composite recovered image from each of the third image data signals produced in accordance with each of said multiple ones of second image data signals.

15. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference; and

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said capturing step further comprises the step of capturing the undegraded image of the reference to provide said third image data signals.

16. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference; and

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said first, second, third, and fourth image data signals are two-dimensional and in one or more color channels.

17. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference;

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals; and

determining the part of said captured image representing said image of said reference.

18. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference; and

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein said first image data signals define u₂(x,y), said second image data signals define U₂^r(x,y), and said third image data signals define U₁^r(x,y), and said producing step comprises the steps of;

determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y), the Fourier transform of said u₂^r(x,y) to provide U₂^r(f_x,f_y), and the Fourier transform u₁^r(x,y) to provide U₁^r(f_x,f_y);

calculating U₁^r(f_x,f_y) multiplied by U₂(f_x,f_y) divided by U₂^r(f_x,f_y) to provide U_1e(f_x,f_y); and

determining the inverse Fourier transform of said U_1e(f_x,f_y) to provide u_1e(x,y) in which said u_1e(x,y) represents said fourth image data signals.

19. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference;

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals; and

outputting an image in accordance with said fourth image data signals representing a recovery of the degraded image.

20. A method for recovering degraded images comprising the steps of:

capturing an image to provide first image data signals representing a degraded image of a scene having an unknown object and second image data signals representing a degraded image of a reference object in the scene of the degraded image;

storing third image data signals representing an undegraded image of the reference; and

producing fourth image data signals representing a recovered image from the degraded image in accordance with said first image data signals, said second image data signals, and said stored third image data signals, wherein multiple ones of said second image data signals each represent an image of a reference object in a different part of the degraded image, and multiple ones of said third image data signals representing an undegraded image of the reference object associated with each of the multiple ones of said second image data signals, and said producing step further comprises the step of producing, for each of said multiple ones of second image data signals, fourth image data signals each representing a recovered image from said degraded image in accordance with said first image data signals, the second image data signals, and the third image data signals associated with the second image data signals.

21. A method for recovering a degraded image represented by first image data signals u₂(x,y), in which second image data signals z(x,y) represent a point source function associated with an object in said first image data signals, said method comprising the steps of:

determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y) and the Fourier transform of said z(x,y) to provide Z(f_x,f_y);

dividing said U₂(f_x,f_y) by said Z(f_x,f_y) to provide at least an estimate of U₁( f_x,f_y); and

determining the inverse Fourier transform of said U₁(f_x,f_y) to provide u₁(x,y) in which said u₁(x,y) represents the recovered image of at least the object in said u₂(x,y).

22. A method for recovering a degraded image represented by first image data signals u₂(x,y), in which second image data signals u₂^r(x,y) represent a degraded reference in the degraded image, and u₁^r(x,y) represent an undegraded image of the reference, said method comprising the steps of:

determining the Fourier transform of said u₂(x,y) to provide U₂(f_x,f_y), the Fourier transform of said U₂^r(X,y) to provide U₂^r(f_x,f_y), and the Fourier transform u₁^r(x,y) to provide U₁^r(f_x,f_y);

calculating U₁^r(f_x,f_y) multiplied by U₂(f_x,f_y) divided by U₂^r(f_x,f_y) to provide at least an estimate of U₁(f_x,f_y); and

determining the inverse Fourier transform of U₁(f_x,f_y) to provide u₁(x,y) in which u₁(x,y) represents the recovered image of u₂(x,y)