Adaptive PSF estimation technique using a sharp preview and a blurred image

• US 8,649,628 B2
• Filed: 01/07/2013
• Issued: 02/11/2014
• Est. Priority Date: 06/29/2009
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Albu, Felix, Drimbarean, Alexandru, Zamfir, Adrian, Florea, Corneliu, Corcoran, Peter
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US13/735,959
• Publication Number:
  US 20130278784A1
• Time in Patent Office: 400 Days
• Field of Search: 382/255, 382/260, 382/275, 348/208.99, 348/208.1, 348/208.4, 348/208.6, 348/208.12
• US Class Current: 382/260
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/003 Deblurring; Sharpening vibration or motion blur correction for cameras comprising an electronic image sensor H04N23/682
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An adaptive motion estimation and deblurring technique for acquired digital images includes acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image. Anb initial approximate point spread function (PSF) is estimated corresponding to the moving of the device. A different DC offset point is determined and a second PSF is calculated based on the different DC offset point.

Claims

1. A method of adaptive motion estimation and deblurring of acquired digital images, comprising:

acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image;

estimating an initial approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing a second PSF for a pair of horizontal and vertical offsets;

calculating a Cepstrum and refining the second PSF to generate a third PSF based on the Cepstrum;

precisely determining a position of a first PSF point; and

deblurring at least one of the digital images using a best found PSF based on the third PSF and first PSF point.

2. The method of claim 1, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

3. The method of claim 1, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

4. The method of claim 1, wherein calculating said Cepstrum comprises measuring a linearity of the PSF.

5. The method of claim 1, wherein calculating said Cepstrum comprises computing a Radon transform.

6. The method of claim 1, further comprising changing a PSF fit that does not pass a distance test from the initial approximate PSF.

7. One or more non-transitory computer readable media having code embedded therein for programming one or more processors to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises:

acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image;

estimating an initial approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing a second PSF for a pair of horizontal and vertical offsets;

calculating a Cepstrum and refining the second PSF to generate a third PSF based on the Cepstrum;

precisely determining a position of a first PSF point; and

deblurring at least one of the digital images using a best found PSF based on the third PSF and first PSF point.

8. The one or more computer readable media of claim 7, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

9. The one or more computer readable media of claim 7, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

10. The one or more computer readable media of claim 7, wherein calculating said Cepstrum comprises measuring a linearity of the PSF.

11. The one or more computer readable media of claim 7, wherein calculating said Cepstrum comprises computing a Radon transform.

12. The one or more computer readable media of claim 7, further comprising changing a PSF fit that does not pass a distance test from the initial approximate PSF.

13. A digital image acquisition device, comprising:

a lens and an image sensor for acquiring a digital image;

a processor; and

one or more processor-readable media having code embedded therein for programming the processor to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises;

acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image;

estimating an initial approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing a second PSF for a pair of horizontal and vertical offsets;

calculating a Cepstrum and refining the second PSF to generate a third PSF based on the Cepstrum;

precisely determining a position of a first PSF point; and

deblurring at least one of the digital images using a best found PSF based on the third PSF and first PSF point.

14. The device of claim 13, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

15. The device of claim 13, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

16. The device of claim 13, wherein calculating said Cepstrum comprises measuring a linearity of the PSF.

17. The device of claim 13, wherein calculating said Cepstrum comprises computing a Radon transform.

18. The device of claim 13, further comprising changing a PSF fit that does not pass a distance test from the initial approximate PSF.