Removing camera shake from a single photograph

• US 20080025627A1
• Filed: 07/28/2006
• Published: 01/31/2008
• Est. Priority Date: 07/28/2006
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology, University of Toronto
• Original Assignee: Massachusetts Institute of Technology, University of Toronto
• Inventors: Roweis, Sam T., Hertzmann, Aaron Phillip, Fergus, Robert D.W., Freeman, William T., Singh, Barun
• Granted Patent: US 7,616,826 B2
• Time in Patent Office: Days
• US Class Current: 382/255
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

A computer method and system for deblurring an image is provided. The invention method and system of deblurring employs statistics on distribution of intensity gradients of a known model. The known model is based on a natural image which may be unrelated to the subject image to be deblurred by the system. Given a subject image having blur, the invention method/system estimates a blur kernel and a solution image portion corresponding to a sample area of the subject image, by applying the statistics to intensity gradients of the sample area and solving for most probable solution image. The estimation process is carried out at multiple scales and results in a blur kernel. In a last step, the subject image is deconvolved image using the resulting blur kernel. The deconvolution generates a deblurred image corresponding to the subject image.

Claims

1. A method for deblurring an image comprising the computer implemented steps of:

providing statistics on distribution of intensity gradients of a known model natural image;

receiving a single input image defining a subject image having blur;

estimating a blur kernel and a solution image portion corresponding to a sample area of the subject image, by applying the statistics to intensity gradients of the sample area and solving for most probable solution image, said estimating resulting in a blur kernel, wherein the model natural image may be unrelated to the subject image; and

deconvoluting the subject image using the resulting blur kernel, the deconvoluting generating a deblurred image corresponding to the subject image.

2. A method as claimed in claim 1 wherein the step of estimating includes performing an estimation procedure at multiple scales starting with a coarse resolution version of the sample area and increasing resolution to a final scale in which the sample area is at a resolution as it is in the input image.

3. A method as claimed in claim 2 wherein for each scale, the intensity gradients of the most probable solution image are used to interpolate an input sample area for a next scale.

4. A method as claimed in claim 1 wherein the model natural image is used for deblurring different subject images such that a same one model natural image is used across different subject images.

5. A method as claimed in claim 1 wherein blur of the subject image is due to camera shake.

6. A method as claimed in claim 5 wherein the camera is any of a digital camera and a film-utilizing camera.

7. A method as claimed in claim 1 wherein the step of estimating includes employing variational Bayesian techniques.

8. A method as claimed in claim 1 wherein the step of receiving further includes linearizing intensities of the subject image.

9. Computer apparatus for deblurring an image, comprising:

a set of statistics on distribution of intensity gradients of a model natural image;

a preprocessor for receiving a single input image defining a subject image having blur;

a motion estimator estimating a blur kernel and a solution image portion corresponding to a sample area of the subject image, by applying the statistics to intensity gradients of the sample area and solving for most probable solution image, said estimating resulting in a blur kernel, wherein the model natural image may be unrelated to the subject image; and

a deconvoluter for deconvoluting the subject image using the resulting blur kernel, the deconvoluting generating a deblurred image corresponding to the subject image.

10. Computer apparatus as claimed in claim 9 wherein the motion estimator performs an estimation procedure at multiple scales starting with a coarse resolution version of the sample area and increasing resolution to a final scale in which the sample area is at its input image resolution.

11. Computer apparatus as claimed in claim 10 wherein for each scale, the intensity gradients of the most probable solution image are used to interpolate an input sample area for a next scale.

12. Computer apparatus as claimed in claim 9 wherein the model natural image is used for deblurring different subject images such that a same one model natural image is used across different subject images.

13. Computer apparatus as claimed in claim 9 wherein blur of the subject image is due to camera shake and the motion estimator effectively estimates path of the camera shake.

14. Computer apparatus as claimed in claim 13 wherein the camera is any of a digital camera and a film-utilizing camera.

15. Computer apparatus as claimed in claim 9 wherein the motion estimator employs variational Bayesian techniques.

16. Computer apparatus as claimed in claim 9 wherein the preprocessor further linearizes the subject image so that the subject image has a linear color space.

17. A computer system for deblurring an image, comprising:

means for providing statistics on distribution of intensity gradients of a known model natural image;

means for receiving a single input image defining a subject image having blur;

motion estimation means for estimating a blur kernel and a solution image portion corresponding to a sample area of the subject image, by applying the statistics to intensity gradients of the sample area and solving for most probable solution image, said estimating resulting in a blur kernel, wherein the known model natural image may be unrelated to the subject image; and

deconvolution means for deconvoluting deconvoluting the subject image using the resulting blur kernel, the deconvoluting generating a deblurred image corresponding to the subject image.

18. A computer system as claimed in claim 17 wherein the motion estimation means performs an estimation procedure at multiple scales starting with a coarse resolution version of the sample area and increasing resolution to a final scale in which the sample area is at its input image resolution;

and

wherein for each scale, the intensity gradients of the most probable solution image are used to interpolate an input sample area for a next scale.

19. A computer system as claimed in claim 17 wherein blur of the subject image is due to camera shake and the motion estimator effectively estimates path of the camera shake, associated camera is any of a digital camera and a film-utilizing camera.

20. A computer system as claimed in claim 17 wherein the motion estimation means employs variational Bayesian techniques.

21. A computer system as claimed in claim 17 wherein the means for receiving includes a preprocessor, the preprocessor linearizing the subject image so that the subject image has a linear set of intensities.