Adaptive PSF estimation technique using a sharp preview and a blurred image

• US 8,208,746 B2
• Filed: 06/21/2010
• Issued: 06/26/2012
• Est. Priority Date: 06/29/2009
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Albu, Felix, Drimbarean, Alexandru, Zamfir, Adrian, Florea, Corneliu, Corcoran, Peter
• Primary Examiner(s): Henn, Timothy J
• Application Number:
  US12/820,086
• Publication Number:
  US 20110050919A1
• Time in Patent Office: 736 Days
• Field of Search: 382/255, 348/208.99, 348/208.1, 348/208.4, 348/208.6, 348/208.12
• US Class Current: 382/255
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An adaptive motion estimation and deblurring technique for acquired digital images includes acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image. And initial approximate point spread function (PSF) is estimated corresponding to the moving of the device. A different DC offset point is determined and a second PSF is calculated based on the different DC offset point.

Claims

1. A method of adaptive motion estimation and deblurring of acquired digital images, comprising:

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

choosing offset grid points from an offset grid in an offset values space based on data of the digital images acquired with the moving device;

applying an iterative process using a variable sized window;

stopping the process when an outer low importance border is detected;

estimating an approximate point spread function (PSF) one or more times corresponding to the moving of the device during the acquiring of the multiple digital images based on the choosing, applying and stopping; and

deblurring at least one of the digital images using the approximate PSF.

2. The method of claim 1, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

3. The method of claim 1, further comprising determining that said reference image is too dark, and amplifying pixel values of the reference image, including avoiding saturation.

4. The method of claim 1, wherein said reference image comprises a preview image.

5. The method of claim 1, further comprising aligning the reference image and the blurred image.

6. The method of claim 1, further comprising re-sizing the reference image to the size of the blurred image.

14. The device of claim 1, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

15. The device of claim 1, wherein the method further comprises determining that said reference image is too dark, and amplifying pixel values of the reference image, including avoiding saturation.

16. The device of claim 1, wherein said reference image comprises a preview image.

17. The device of claim 1, wherein the method further comprises aligning the reference image and the blurred image.

18. The device of claim 1, wherein the method further comprises re-sizing the reference image to the size of the blurred image.

7. One or more non-transitory computer readable media having code embedded therein for programming one or more processors to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises:

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

choosing offset grid points from an offset grid in an offset values space based on data of the digital images acquired with the moving device;

applying an iterative process using a variable sized window;

stopping the process when an outer low importance border is detected;

estimating an approximate point spread function (PSF) one or more times corresponding to the moving of the device during the acquiring of the multiple digital images based on the choosing, applying and stopping; and

deblurring at least one of the digital images using the approximate PSF.

8. The one or more computer readable media of claim 7, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

9. The one or more computer readable media of claim 7, wherein the method further comprises determining that said reference image is too dark, and amplifying pixel values of the reference image, including avoiding saturation.

10. The one or more computer readable media of claim 7, wherein said reference image comprises a preview image.

11. The one or more computer readable media of claim 7, wherein the method further comprises aligning the reference image and the blurred image.

12. The one or more computer readable media of claim 7, wherein the method further comprises re-sizing the reference image to the size of the blurred image.

13. A digital image acquisition device, comprising:

a lens and an image sensor for acquiring a digital image;

a processor; and

one or more processor-readable media having code embedded therein for programming the processor to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises;

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

choosing offset grid points from an offset grid in an offset values space based on data of the digital images acquired with the moving device;

applying an iterative process using a variable sized window;

stopping the process when an outer low importance border is detected;

estimating an approximate point spread function (PSF) one or more times corresponding to the moving of the device during the acquiring of the multiple digital images based on the choosing, applying and stopping; and

deblurring at least one of the digital images based on the approximate PSF.