System and methods for de-blurring motion blurred images

• US 8,547,441 B2
• Filed: 07/29/2011
• Issued: 10/01/2013
• Est. Priority Date: 06/21/2002
• Status: Expired due to Term

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Ben-Ezra, Moshe, Nayar, Shree K.
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): COWAN, EUEL K
• Application Number:
  US13/194,643
• Publication Number:
  US 20110286681A1
• Time in Patent Office: 795 Days
• Field of Search: 382254-275, 345441-443, 715700-773
• US Class Current: 348/208.4
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Systems and methods for providing a substantially de-blurred image of a scene from a motion blurred image of the scene are disclosed. An exemplary system includes a primary detector for sensing the motion blurred image and generating primary image information representing the blurred image, a secondary detector for sensing two or more secondary images of the scene and for generating secondary image information representing the two or more secondary images, and a processor for determining motion information from the secondary image information, estimating a point spread function for the motion blurred image from the motion information, and applying the estimated point spread function to the primary image information to generate information representing the substantially de-blurred image.

Claims

1. A method for interactively determining a de-blurred image corresponding to a motion blurred image of a scene, comprising:

selecting one or more features within said motion blurred image with a graphical user interface drawing tool;

forming one or more model images on selected one or more features with said graphical user interface drawing tool, each of which corresponds to one of said one or more selected features;

estimating a point spread function using said selected one or more features and said corresponding one or more model images; and

determining said de-blurred image using said estimated point spread function and said motion blurred image.

2. The method of claim 1, wherein at least one of said one of said one or more selected features comprises a point, and forming said one or more model images comprises using a point tool to form at least one of said one or more model images.

3. The method of claim 1, wherein at least one of said one of said one or more selected features comprises a line, and forming said one or more model images comprises using a line tool to form at least one of said one or more model images.

4. The method of claim 1, wherein at least one of said one of said one or more selected features comprises a ramp, and forming said one or more model images comprises using a ramp tool to form at least one of said one or more model images.

5. The method of claim 1, wherein at least one of said one of said one or more selected features comprises a corner, and forming said one or more model images comprises using a corner tool to form at least one of said one or more model images.

6. The method of claim 1, wherein selecting one or more features within said motion blurred image comprises selecting one or more points of an object in said scene.

7. The method of claim 1, wherein said point spread function is represented by a convolution kernel.

8. The method of claim 1, wherein determining said de-blurred image using said estimated point spread function and said motion blurred image comprises utilizing a Richardson Lucy algorithm.

9. A method for determining a de-blurred image corresponding to a motion blurred image of a scene comprising:

selecting, using an image processor, one or more features within said motion blurred image including selecting one or more points of an object in said scene;

estimating, using said image processor, a point spread function using said selected one or more features; and

determining using said image processor, said de-blurred image using said estimated point spread function of a model image drawn on and corresponding to selected one or more features with a graphical user interface drawing tool and said motion blurred image.

10. The method of claim 9, wherein said point spread function is represented by a convolution kernel.

11. The method of claim 9, wherein determining said de-blurred image using said estimated point spread function and said motion blurred image comprises utilizing a Richardson Lucy algorithm.