REMOVING BLUR FROM AN IMAGE

• US 20100074552A1
• Filed: 09/24/2008
• Published: 03/25/2010
• Est. Priority Date: 09/24/2008
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Yuan, Lu, Quan, Long, Sun, Jian, Shum, Heung-Yeung
• Granted Patent: US 8,406,564 B2
• Time in Patent Office: Days
• US Class Current: 382/264
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

Embodiments related to the removal of blur from an image are disclosed. One disclosed embodiment provides a method of performing an iterative non-blind deconvolution of a blurred image to form an updated image. The method comprises downsampling the blurred image to form a blurred image pyramid comprising images of two or more different resolution scales, downsampling a blur kernel to form a blur kernel pyramid comprising kernels of two or more different sizes, and deconvoluting a selected image in the blurred image pyramid according to a Richardson-Lucy deconvolution process in which a bilateral range/spatial filter is employed.

Claims

1. A computer-readable storage medium comprising instructions thereon that are executable by a computing device to perform a method of performing an iterative non-blind deconvolution of a blurred image to form an updated image comprising:

downsampling the blurred image to form a blurred image pyramid comprising images of two or more different resolution scales;

downsampling a blur kernel to form a blur kernel pyramid comprising kernels of two or more different sizes; and

deconvoluting a selected image in the blurred image pyramid according to a Richardson-Lucy deconvolution process in which a bilateral range/spatial filter is employed.

2. The computer-readable storage medium of claim 1, wherein the instructions are executable to employ a spatial filter reliant on an external guide image in the Richardson-Lucy deconvolution process in addition to a bilateral range/spatial filter.

3. The computer-readable storage medium of claim 2, wherein the guide image is an upsampled image from a previously deconvoluted next-lower resolution scale in the blurred image pyramid.

4. The computer-readable storage medium of claim 2, wherein the guide image is a previously updated image from a same resolution scale in the blurred image pyramid as a current image.

5. The computer-readable storage medium of claim 2, wherein the instructions are executable to perform one or more inter-scale deconvolutions in which the guide image at a selected scale in the blurred image pyramid is an upsampled image from a next-lower scale in the blurred image pyramid.

6. The computer-readable storage medium of claim 5, wherein the instructions are executable to perform one or more intra-scale deconvolutions in which the guide image is a previously deconvoluted image from a same scale in the blurred image pyramid.

7. The computer-readable storage medium of claim 6, wherein the instructions are executable to perform the Richardson-Lucy deconvolution process in which the bilateral range/spatial filter is employed and in which the spatial filter reliant on the guide image is employed as follows:

$I^{t+1}=\frac{I^t}{1+\mathrm{\lambda <br><br>}<br><br>\nabla <br><br>E_B<br><br>\left(I^t\right)}<br><br>\left[K^{*}\otimes <br><br>\frac{B}{\left(I^t\otimes <br><br>K\right)}\right]$

wherein I^t+1 is a value of a pixel in the updated image, I^t is a value of the pixel in a current image, B is a value of the pixel in the original degraded image, K is a blur kernel, K* is an adjoint of K, λ

is a regularization strength, and ∇

E_B comprises the bilateral range/spatial filter and the spatial filter reliant on the guide image; and

wherein the regularization strength λ

is reduced after each intra-scale deconvolution process within an image scale.

8. The computer-readable storage medium of claim 7, wherein ∇

E_B comprises a high pass filter as an additional filter for the Richardson-Lucy deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.

9. A method of deconvoluting a blurred image via a non-blind deconvolution, the method comprising:

iteratively downsampling the blurred image to form a blurred image pyramid comprising images of a plurality of different resolution scales;

at a lower resolution scale in the blurred image pyramid, deconvoluting a lower resolution image according to a Joint Bilateral Richardson-Lucy (JBRL) deconvolution process in which a bilateral range/spatial filter is employed and in which a spatial filter reliant on a guide image upsampled from a next-lowest resolution scale is also employed;

upsampling the lower resolution image to form an upsampled image; and

at a higher resolution scale in the blurred image pyramid, deconvoluting the higher resolution image according to the JBRL deconvolution process in which the upsampled image is utilized as the guide image.

10. The method of claim 9, further performing a Bilateral Richardson-Lucy (BRL) deconvolution process at a lowest resolution scale in the blurred image pyramid in which the spatial filter employing the guide image is not used.

11. The method of claim 9, further comprising performing one or more intra-scale deconvolutions in one or more resolution scales, in which the guide image is a previously deconvoluted image from a same scale in the blurred image pyramid.

12. The method of claim 11, further comprising reducing a regularization strength after each intra-scale deconvolution within an image scale.

13. The method of claim 9, wherein the JBRL deconvolution process proceeds as following:

$I^{t+1}=\frac{I^t}{1+\mathrm{\lambda <br><br>}<br><br>\nabla <br><br>E_B<br><br>\left(I^t\right)}<br><br>\left[K^{*}\otimes <br><br>\frac{B}{\left(I^t\otimes <br><br>K\right)}\right]$

wherein I^t+1 is a value of a pixel in the updated image, I^t is a value of the pixel in a current image, B is a value of the pixel in the original degraded image, K is a blur kernel, K* is an adjoint of K, λ

is a regularization strength, and ∇

E_B comprises the bilateral range/spatial filter and the spatial filter reliant on the guide image.

14. The method of claim 13, wherein ∇

E_B comprises a high pass filter as an additional filter for the JBRL deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.

15. A method of deconvoluting an original blurred image to form an updated image, comprising:

iteratively downsampling the original blurred image to form a blurred image pyramid comprising i layers of blurred images of progressively lower resolution scales, wherein i=0 is a lowest-resolution scale;

for each resolution scale 1 to i;

performing an inter-scale Joint Bilateral Richardson-Lucy (JBRL) deconvolution process in which a bilateral range/spatial filter is employed and in which a spatial filter reliant on a guide image upsampled from a next-lower resolution scale in the blurred image pyramid is employed;

performing one or more intra-scale JBRL deconvolution processes in which the bilateral range/spatial filter is employed and in which a spatial filter reliant on an intra-scale guide image is employed, thereby forming an intra-scale deconvoluted image; and

upsampling the intra-scale deconvoluted image to form a guide image for another inter-scale Richardson-Lucy deconvolution process at a next-highest scale in the blurred image pyramid.

16. The method of claim 15, further comprising, for scale zero, deconvoluting a lowest resolution image in the blurred image pyramid according to a Bilateral Richardson-Lucy (BRL) deconvolution process in which a bilateral range/spatial filter, but no spatial filter reliant on a guide image, is employed.

17. The method of claim 15, wherein upsampling the intra-scale deconvoluted image comprises performing a bicubic or bilinear interpolation to form an intermediate upscaled image, and then performing a deconvolution of the intermediate upscaled image via a Bilateral Richardson-Lucy (BRL) deconvolution process in which a bilateral range/spatial filter, but no spatial filter reliant on a guide image, is employed.

18. The method of claim 15, wherein performing one or more intra-scale JBRL deconvolution processes comprises determining a blurred residual image by

Δ

B=B−

I^g{circle around (×

)}K,

wherein Δ

B is a value of a pixel in the blurred residual image, B is a value of a pixel of the original blurred image, I^g is a value of a pixel in the intra-scale guide image, and K is a blur kernel, then determining a detail layer Δ

I by

Δ

B=Δ

I{circle around (×

)}K,

and then updating I^g with the values in the detail layer Δ

I to produce a guide image for a next intra-scale deconvolution process.

19. The method of claim 18, further comprising reducing a regularization strength in the next intra-scale deconvolution process.

20. The method of claim 15, further comprising employing high pass filter as an additional filter for the intra-scale JBRL deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.