IMAGE DEBLURRING USING A COMBINED DIFFERENTIAL IMAGE

• US 20110102642A1
• Filed: 11/04/2009
• Published: 05/05/2011
• Est. Priority Date: 11/04/2009
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Wang, Sen, Miller, Rodney L., Hou, Tingbo
• Granted Patent: US 8,379,120 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

A method for determining a deblurred image comprising: receiving a blurred image of a scene; receiving a blur kernel; initializing a candidate deblurred image; determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image; determining a combined differential image by combining the differential images; repeatedly updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image until a convergence criterion is satisfied; and storing the final candidate deblurred image in a processor-accessible memory system.

Claims

1. A method for determining a deblurred image, the method implemented at least in part by a data processing system and comprising:

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image;

g) repeating steps d)-f) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in a processor-accessible memory system.

2. The method of claim 1 wherein the blur kernel is determined from the blurred image.

3. The method of claim 2 wherein the blur kernel is determined by analyzing only a subset of the image pixels in the blurred image, wherein the subset is determined to contain significant image detail.

4. The method of claim 3 wherein the subset of the image pixels in the blurred image is selected by:

determining a Laplacian image from the blurred image; and

selecting the subset of image pixels in the blurred image by analyzing the Laplacian image to determine a subset of image pixels containing significant image detail.

5. The method of claim 1, wherein the blur kernel is determined based on analysis of one or more additional images captured of same scene.

6. The method of claim 5, wherein at least one of the additional images is an image captured with a shorter exposure time than the blurred image.

7. The method of claim 6, wherein the at least one of the additional images has a lower level of motion blur than the blurred image.

8. The method of claim 1 wherein the each of the differential images is calculated using a specified direction and distance interval for the neighboring pixels.

9. The method of claim 1 wherein the plurality of differential images include a horizontal differential image representing differences between neighboring pixels in the horizontal direction and a vertical differential image representing differences between neighboring pixels in the vertical direction.

10. The method of claim 1 wherein the candidate deblurred image is updated using a Bayesian inference method.

11. The method of claim 10 wherein the Bayesian inference method includes evaluating candidate deblurred images using an energy function.

12. The method of claim 11 wherein the energy function includes:

an image fidelity term which is a function of a difference between the blurred image and a candidate blurred image determined by convolving the candidate deblurred image with the blur kernel; and

an image differential term which is a function of the combined differential image.

13. The method of claim 11 wherein the energy function is optimized using a conjugate gradient algorithm.

14. The method of claim 1 wherein the blurred image is captured using a digital camera.

15. The method of claim 14 wherein the blurred image has been blurred due to motion blur resulting from motion of the digital camera during the time that the blurred image was captured.

16. The method of claim 1 wherein the blurred image is a frame of a digital video.

17. The method of claim 1 wherein the combined differential image is determined by a weighted combination of functions of the differential images.

18. The method of claim 1 wherein the functions of the differential images are weighted by a distance weighting factor and a pixel-dependent weighting factor.

19. A digital camera system comprising:

an image sensor for capturing an image of a scene;

an lens system for imaging the scene onto the image sensor;

a processor-accessible memory system. and

a data processing system for performing the steps of;

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image;

g) repeating steps d)-f) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in the processor-accessible memory system.

20. A computer program product for determining a deblurred image comprising an executable software application for causing a data processing system to perform the steps of:

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image;

g) repeating steps d)-t) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in a processor-accessible memory system.