Removing blur from an image

• US 8,406,564 B2
• Filed: 09/24/2008
• Issued: 03/26/2013
• Est. Priority Date: 09/24/2008
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, Shum, Heung-Yeung, Yuan, Lu, Quan, Long
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US12/237,126
• Publication Number:
  US 20100074552A1
• Time in Patent Office: 1,644 Days
• Field of Search: None
• US Class Current: 382/279
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

Embodiments related to the removal of blur from an image are disclosed. One disclosed embodiment provides a method of performing an iterative non-blind deconvolution of a blurred image to form an updated image. The method comprises downsampling the blurred image to form a blurred image pyramid comprising images of two or more different resolution scales, downsampling a blur kernel to form a blur kernel pyramid comprising kernels of two or more different sizes, and deconvoluting a selected image in the blurred image pyramid according to a Richardson-Lucy deconvolution process in which a bilateral range/spatial filter is employed.

Claims

1. A computer-readable memory comprising instructions thereon that are executable by a computing device to perform an iterative non-blind deconvolution of a blurred image to form an updated image, the deconvolution comprising:

downsampling the blurred image to form a blurred image pyramid comprising images of two or more different resolution scales;

downsampling a blur kernel to form a blur kernel pyramid comprising kernels of two or more different sizes; and

deconvoluting a selected image in the blurred image pyramid according to a bilateral Richardson-Lucy deconvolution process including a bilateral range/spatial filter and an additional range filter reliant on an external guide image.

2. The computer-readable memory of claim 1, wherein the guide image is an upsampled image from a previously deconvoluted next-lower resolution scale in the blurred image pyramid.

3. The computer-readable memory of claim 1, wherein the guide image is a previously updated image from a same resolution scale in the blurred image pyramid as a current image.

4. The computer-readable memory of claim 1, wherein the instructions are executable to perform one or more inter-scale deconvolutions in which the guide image at a selected scale in the blurred image pyramid is an upsampled image from a next-lower scale in the blurred image pyramid.

5. The computer-readable memory of claim 4, wherein the instructions are executable to perform one or more intra-scale deconvolutions in which the guide image is a previously deconvoluted image from a same scale in the blurred image pyramid.

6. The computer-readable memory of claim 5, wherein the instructions are executable to perform the bilateral Richardson-Lucy deconvolution process in which the bilateral range/spatial filter is employed and in which the spatial filter reliant on the guide image is employed as follows:

7. The computer-readable memory of claim 6, wherein ∇

E_B comprises a high pass filter as an additional filter for the bilateral Richardson-Lucy deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.

8. A method of deconvoluting a blurred image via a non-blind deconvolution, the method comprising:

iteratively downsampling the blurred image to form a blurred image pyramid comprising images of a plurality of different resolution scales;

at a lower resolution scale in the blurred image pyramid, deconvoluting a lower resolution image according to a Joint Bilateral Richardson-Lucy (JBRL) deconvolution process in which a bilateral range/spatial filter is employed and in which an additional range filter reliant on a guide image upsampled from a next-lowest resolution scale is also employed;

upsampling the lower resolution image to form an upsampled image; and

at a higher resolution scale in the blurred image pyramid, deconvoluting the higher resolution image according to the JBRL deconvolution process in which the upsampled image is utilized as the guide image for the additional range filter.

9. The method of claim 8, further performing a Bilateral Richardson-Lucy (BRL) deconvolution process at a lowest resolution scale in the blurred image pyramid in which the spatial filter employing the guide image is not used.

10. The method of claim 8, further comprising performing one or more intra-scale deconvolutions in one or more resolution scales, in which the guide image is a previously deconvoluted image from a same scale in the blurred image pyramid.

11. The method of claim 10, further comprising reducing a regularization strength after each intra-scale deconvolution within an image scale.

12. The method of claim 8, wherein the JBRL deconvolution process proceeds as following:

13. The method of claim 12, wherein ∇

E_B comprises a high pass filter as an additional filter for the JBRL deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.

14. A method of deconvoluting an original blurred image to form an updated image, the method comprising:

iteratively downsampling the original blurred image to form a blurred image pyramid comprising i layers of blurred images of progressively lower resolution scales, wherein i=0 is a lowest-resolution scale;

for each resolution scale 1 to i;

performing an inter-scale Joint Bilateral Richardson-Lucy (JBRL) deconvolution process in which a bilateral range/spatial filter is employed and in which an additional range filter reliant on a guide image upsampled from a next-lower resolution scale in the blurred image pyramid is employed;

performing one or more intra-scale JBRL deconvolution processes in which the bilateral range/spatial filter is employed and in which a spatial filter reliant on an intra-scale guide image is employed, thereby forming an intra-scale deconvoluted image; and

upsampling the intra-scale deconvoluted image to form a guide image for another inter-scale Richardson-Lucy deconvolution process at a next-highest scale in the blurred image pyramid.

15. The method of claim 14, further comprising, for scale zero, deconvoluting a lowest resolution image in the blurred image pyramid according to a Bilateral Richardson-Lucy (BRL) deconvolution process in which a bilateral range/spatial filter, but no spatial filter reliant on a guide image, is employed.

16. The method of claim 14, wherein upsampling the intra-scale deconvoluted image comprises performing a bicubic or bilinear interpolation to form an intermediate upscaled image, and then performing a deconvolution of the intermediate upscaled image via a Bilateral Richardson-Lucy (BRL) deconvolution process in which a bilateral range/spatial filter, but no spatial filter reliant on a guide image, is employed.

17. The method of claim 14, wherein performing one or more intra-scale JBRL deconvolution processes comprises determining a blurred residual image by

Δ

B=B−

I^g{circle around (×

)}K,

wherein Δ

B is a value of a pixel in the blurred residual image, B is a value of a pixel of the original blurred image, I^g is a value of a pixel in the intra-scale guide image, and K is a blur kernel, then determining a detail layer Δ

I by

Δ

B=Δ

I{circle around (×

)}K,

and then updating I^g with the values in the detail layer Δ

I to produce a guide image for a next intra-scale deconvolution process.

18. The method of claim 17, further comprising reducing a regularization strength in the next intra-scale deconvolution process.

19. The method of claim 14, further comprising employing high pass filter as an additional filter for the intra-scale JBRL deconvolution process in a final intra-scale deconvolution at each scale in the blurred image pyramid.