Multi-image deblurring

• US 8,229,244 B2
• Filed: 03/30/2009
• Issued: 07/24/2012
• Est. Priority Date: 03/30/2009
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Agrawal, Amit, Xu, Yi, Raskar, Ramesh
• Primary Examiner(s): Patel, Kanjibhai
• Application Number:
  US12/414,162
• Publication Number:
  US 20100246989A1
• Time in Patent Office: 1,212 Days
• Field of Search: 382/205, 382/255, 382/284, 382/285, 382/312, 382/264, 382274-276, 703/2, 348/254, 348/207.99, 348/208.4, 348/208.13
• US Class Current: 382/255
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

Embodiments of the invention describe a method for reducing a blur in an image of a scene. First, we acquire a set of images of the scene, wherein each image in the set of images includes an object having a blur associated with a point spread function (PSF) forming a set of point spread functions (PSFs), wherein the set of PSFs is suitable for null-filling operation. Next, we invert jointly the set of images and the set of PSFs to produce an output image having a reduced blur.

Claims

1. A method for reducing a blur in an image of a scene, comprising a processor for performing steps of the method, comprising the steps of:

acquiring a set of images of the scene with a single conventional camera, wherein each image in the set of images includes an object having a blur associated with a point spread function (PSF) forming a set of point spread functions (PSFs), wherein the set of PSFs is suitable for null-filling operation; and

inverting jointly the set of images and the set of PSFs to produce an output image having a reduced blur.

2. The method of claim 1, further comprising:

acquiring the set of images with exposure times, which are non-integer multiples of each other.

3. The method of claim 2, further comprising:

determining the exposure times such that a deconvolution operator is maximized.

4. The method of claim 2, further comprising:

determining the exposure times such that the signal to noise is maximized.

5. The method of claim 1, wherein the set of PSFs includes a first PSF, the first PSF at a particular frequency is zero, and a second PSF, wherein the second PSF at the particular frequency is not zero.

6. The method of claim 1, further comprising:

acquiring the set of images using multiple cameras running at different frame rates.

7. The method of claim 1, wherein the PSF is a motion PSF.

8. The method of claim 1, wherein the inverting further comprising:

determining the output image according to

9. A method for reducing blur in an image, comprising a processor for performing steps of the method, comprising the steps of:

acquiring, with a single conventional camera, repetitively a set of images of a moving object with a set of exposures, wherein the exposures are non-integer multiples of each other;

estimating a point spread function (PSF) using motion vectors computed from images acquired with the same exposure forming a set of point spread functions (PSFs);

determining an alpha map for each image in the set of images;

removing a background contribution from the images using the alpha map; and

inverting jointly the set of images and the set of PSFs to produce an output image having a reduced blur.

10. The method of claim 9, further comprising:

repeating the determining, the removing, and the inverting.

11. The method of claim 9, wherein the acquiring is performed using a camera having an auto exposure bracketing.

12. The method of claim 9, wherein for each image further comprising:

estimating a background of the image; and

generating a crude trimap by thresholding a difference between the image and the background.

13. The method of claim 9, further comprising:

aligning the object in the set of images.

14. The method of claim 9, further comprising:

refining a segmentation of the object for each motion line independently.

15. A camera configured for reducing blur in an image, comprising:

means for acquiring a set of images of a moving object with a single conventional camera, wherein each image is associated with a point spread function (PSF) forming a set of point spread functions (PSFs), wherein the set of PSFs is suitable for null-filling operation; and

means for inverting jointly the set of images and the set of PSFs to produce an output image having a reduced blur.

16. The camera of claim 15, wherein the set of PSFs includes a first PSF, the first PSF at a particular frequency is zero, and a second PSF, wherein the second PSF at the particular frequency is not zero.

17. The camera of claim 15, wherein the means for inverting further comprising:

means for determining the output image according to

18. The camera of claim 15, further comprising:

means for estimating the set of PSFs.

19. The camera of claim 15, further comprising:

means for segmenting the set of images.