Adaptive PSF estimation technique using a sharp preview and a blurred image

• US 8,204,330 B2
• Filed: 06/21/2010
• Issued: 06/19/2012
• Est. Priority Date: 06/29/2009
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Albu, Felix, Drimbarean, Alexandru, Zamfir, Adrian, Florea, Corneliu, Corcoran, Peter
• Primary Examiner(s): Henn, Timothy J
• Application Number:
  US12/820,034
• Publication Number:
  US 20110043648A1
• Time in Patent Office: 729 Days
• Field of Search: 382/255, 348/208.99, 348/208.1, 348/208.4, 348/208.6, 348/208.12
• US Class Current: 382/255
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An adaptive motion estimation and deblurring technique for acquired digital images includes acquiring multiple digital images with a moving digital image acquisition device that includes an image sensor, including a relatively sharp, underexposed reference image and a blurred image. Anb initial approximate point spread function (PSF) is estimated corresponding to the moving of the device. A different DC offset point is determined and a second PSF is calculated based on the different DC offset point.

Claims

1. A method of adaptive motion estimation and deblurring of acquired digital images, comprising:

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

estimating an approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

generating a second DC offset point and calculating a second PSF based on the second DC offset point;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing said second PSF for a pair of horizontal and vertical offsets;

deblurring at least one of the digital images using a crop of the blurred image or convoluting with a crop of a relatively underexposed reference image, or a combination thereof.

2. The method of claim 1, further comprising computing an error and determining a minimum using a predetermined search method.

3. The method of claim 2, wherein the search method comprises an Adaptive Rood Pattern Search (ARPS) method.

4. The method of claim 2, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

5. The method of claim 1, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

6. The method of claim 1, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

7. One or more non-transitory computer readable media having code embedded therein for programming one or more processors to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises:

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

estimating an approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

generating a second DC offset point and calculating a second PSF based on the second DC offset point;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing said second PSF for a pair of horizontal and vertical offsets;

deblurring at least one of the digital images using a crop of the blurred image or convoluting with a crop of a relatively underexposed reference image, or a combination thereof.

8. The one or more computer readable media of claim 7, wherein the method further comprises computing an error and determining a minimum using a predetermined search method.

9. The one or more computer readable media of claim 8, wherein the search method comprises an Adaptive Rood Pattern Search (ARPS) method.

10. The one or more computer readable media of claim 8, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

11. The one or more computer readable media of claim 7, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

12. The one or more computer readable media of claim 7, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.

13. A digital image acquisition device, comprising:

a lens and an image sensor for acquiring a digital image;

a processor; and

one or more processor-readable media having code embedded therein for programming the processor to perform a method of adaptive motion estimation and deblurring of acquired digital images, wherein the method comprises;

acquiring multiple digital images with a moving digital image acquisition device, including a relatively sharp, underexposed reference image and a blurred image;

estimating an approximate point spread function (PSF) based on a first DC offset point corresponding to the moving of the device during the acquiring of the multiple digital images;

generating a second DC offset point and calculating a second PSF based on the second DC offset point;

choosing multiple crops from the blurred image and the reference image or an interpolated version of the reference image, or both, and equalizing a brightness between these crops;

computing said second PSF for a pair of horizontal and vertical offsets;

deblurring at least one of the digital images using a crop of the blurred image or convoluting with a crop of a relatively underexposed reference image, or a combination thereof.

14. The device of claim 13, wherein the method further comprises computing an error and determining a minimum using a predetermined search method.

15. The device of claim 14, wherein the search method comprises an Adaptive Rood Pattern Search (ARPS) method.

16. The device of claim 14, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

17. The device of claim 13, wherein the estimating comprises using a pair of gyro sensors or accelerometers, or a combination thereof.

18. The device of claim 13, wherein the offset comprises a measure of a yaw or a pitch of the image sensor, or both.