Image deblurring using a combined differential image

• US 8,379,120 B2
• Filed: 11/04/2009
• Issued: 02/19/2013
• Est. Priority Date: 11/04/2009
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Wang, Sen, Hou, Tingbo, Miller, Rodney L.
• Primary Examiner(s): Fan, Chieh M
• Assistant Examiner(s): Trehan, Akshay
• Application Number:
  US12/612,135
• Publication Number:
  US 20110102642A1
• Time in Patent Office: 1,203 Days
• Field of Search: 348/78, 348/208.4, 348/241, 348/248, 348/208.6, 348/208.12, 348/208.13, 382/255, 382/275, 382/264
• US Class Current: 348/248
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

A method for determining a deblurred image comprising: receiving a blurred image of a scene; receiving a blur kernel; initializing a candidate deblurred image; determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image; determining a combined differential image by combining the differential images; repeatedly updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image until a convergence criterion is satisfied; and storing the final candidate deblurred image in a processor-accessible memory system.

Claims

1. A method for determining a deblurred image, the method implemented at least in part by a data processing system and comprising:

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image, wherein each pixel of a particular differential image is determined by computing a difference between the corresponding pixel in the candidate deblurred image and a nearby pixel in the candidate deblurred image at a predefined relative position, wherein each differential image is associated with a different predefined relative position;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image, wherein the candidate deblurred image is updated by using a Bayesian inference method that evaluates candidate deblurred images using an energy function that includes an image differential term which is a function of the combined differential image;

g) repeating steps d)-f) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in a processor-accessible memory system.

2. The method of claim 1 wherein the blur kernel is determined from the blurred image.

3. The method of claim 2 wherein the blur kernel is determined by analyzing only a subset of the image pixels in the blurred image, wherein the subset is determined to contain significant image detail.

4. The method of claim 3 wherein the subset of the image pixels in the blurred image is selected by:

determining a Laplacian image from the blurred image; and

selecting the subset of image pixels in the blurred image by analyzing the Laplacian image to determine a subset of image pixels containing significant image detail.

5. The method of claim 1, wherein the blur kernel is determined based on analysis of one or more additional images captured of same scene.

6. The method of claim 5, wherein at least one of the additional images is an image captured with a shorter exposure time than the blurred image.

7. The method of claim 6, wherein the at least one of the additional images has a lower level of motion blur than the blurred image.

8. The method of claim 1 wherein the predefined relative position associated with each of the differential images is specified by a direction and distance interval between the neighboring pixels.

9. The method of claim 1 wherein the plurality of differential images include a horizontal differential image corresponding to a predefined relative position with a horizontal offset and a vertical differential image corresponding to a predefined relative position with a vertical offset.

10. The method of claim 1 wherein the energy function further includes:

an image fidelity term which is a function of a difference between the blurred image and a candidate blurred image determined by convolving the candidate deblurred image with the blur kernel.

11. The method of claim 1 wherein the energy function is optimized using a conjugate gradient algorithm.

12. The method of claim 1 wherein the blurred image is captured using a digital camera.

13. The method of claim 12 wherein the blurred image has been blurred due to motion blur resulting from motion of the digital camera during the time that the blurred image was captured.

14. The method of claim 1 wherein the blurred image is a frame of a digital video.

15. The method of claim 1 wherein the combined differential image is determined by a weighted combination of functions of the differential images.

16. The method of claim 1 wherein the functions of the differential images are weighted by a distance weighting factor that depends on a distance between the pixels being differenced, and a pixel-dependent weighting factor that depends on pixel magnitudes in the differential images.

17. A digital camera system comprising:

an image sensor for capturing an image of a scene;

an lens system for imaging the scene onto the image sensor;

a processor-accessible memory system; and

a data processing system for performing the steps of;

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image, wherein each pixel of a particular differential image is determined by computing a difference between the corresponding pixel in the candidate deblurred image and a nearby pixel in the candidate deblurred image at a predefined relative position, wherein each differential image is associated with a different predefined relative position;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image, wherein the candidate deblurred image is updated by using a Bayesian inference method that evaluates candidate deblurred images using an energy function that includes an image differential term which is a function of the combined differential image;

g) repeating steps d)-f) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in the processor-accessible memory system.

18. A computer program product for determining a deblurred image comprising an executable software application stored in a non-transitory process readable medium for causing a data processing system to perform the steps of:

a) receiving a blurred image of a scene;

b) receiving a blur kernel;

c) initializing a candidate deblurred image;

d) determining a plurality of differential images representing differences between neighboring pixels in the candidate deblurred image, wherein each pixel of a particular differential image is determined by computing a difference between the corresponding pixel in the candidate deblurred image and a nearby pixel in the candidate deblurred image at a predefined relative position, wherein each differential image is associated with a different predefined relative position;

e) determining a combined differential image by combining the differential images;

f) updating the candidate deblurred image responsive to the blurred image, the blur kernel, the candidate deblurred image and the combined differential image, wherein the candidate deblurred image is updated by using a Bayesian inference method that evaluates candidate deblurred images using an energy function that includes an image differential term which is a function of the combined differential image;

g) repeating steps d)-f) until a convergence criterion is satisfied; and

h) storing the final candidate deblurred image in a processor-accessible memory system.