METHOD AND APPARATUS TO CORRECT DIGITAL IMAGE BLUR DUE TO MOTION OF SUBJECT OR IMAGING DEVICE

• US 20090128641A1
• Filed: 11/19/2008
• Published: 05/21/2009
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Clear Imaging Research, LLC
• Original Assignee: Fatih M. Ozluturk
• Inventors: Ozluturk, Fatih M.
• Granted Patent: US 8,064,720 B2
• Time in Patent Office: Days
• US Class Current: 348/208.600
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 5/265 Mixing

Abstract

Signal processing techniques are applied to digital image data to remove the distortion caused by motion of the camera, or the movement of the subject being photographed, or defective optics, or optical distortion from other sources. When the image is captured, the effect of relative motion between the camera and the subject is that it transforms the true image into a blurred image according to a 2-dimensional transfer function. The 2-dimensional transfer function representing the motion is derived using blind estimation techniques or by using information from sensors that detect the motion. The transfer function is inverted and used to define a corrective filter. The filter is applied to the image and the blur due to the motion is removed, restoring the correct image. Another embodiment uses the transfer function to avoid blur by combining multiple consecutive images taken at a fast shutter speed.

Claims

1. A method for use in imaging, the method comprising:

receiving an image; and

performing an iterative technique using a two dimensional transfer function to correct the image such that distortion in the image is substantially reduced.

2. The method of claim 1 wherein the iterative technique generates an inverse of the transfer function and uses the inverse transfer function to correct the image.

3. The method of claim 1 further comprising:

receiving motion information associated with the image;

deriving the two dimensional transfer function based on the motion information; and

filtering the image using an inverse of the transfer function.

4. The method of claim 3 wherein the derived transfer function is estimated using at least one blind detection technique.

5. The method of claim 3 wherein the image is generated by an imager and the motion information indicates movement of the imager.

6. The method of claim 3 wherein the image includes a subject and the motion information indicates movement of the subject.

7. The method of claim 3 wherein the image is generated by an imager, the image includes a subject, and the motion information indicates relative movement of the imager and the subject.

8. The method of claim 3 further comprising:

extracting a portion of the image, wherein the image portion includes a subject.

9. A camera, comprising:

a recording medium, configured to capture an image;

a memory, configured to record the image; and

a processor configured to perform an iterative technique using a two dimensional transfer function to correct the image such that distortion in the image is substantially reduced.

10. The camera of claim 9 wherein the processor is configured to use the iterative technique to generate an inverse of the transfer function, and configured to use the inverse transfer function to correct the image.

11. The camera of claim 9 wherein the processor is further configured to receive motion information associated with the image, to derive the two dimensional transfer function based on the motion information and an inverse of the two dimensional transfer function, and to process the image using the inverse transfer function.

12. The camera of claim 11 further comprising one or more motion detectors configured to generate the motion information, and wherein the processor is further configured to derive the two dimensional transfer function using the motion information.

13. The camera of claim 11 wherein the image includes a subject and the motion information indicates movement of the camera.

14. The camera of claim 11 wherein the image includes a subject and the motion information indicates relative movement of the camera and the subject.

15. The camera of claim 11 wherein the processor is further configured to extract a portion of the image, wherein the image portion includes a subject.

16. The camera of claim 11 wherein the processor is configured to estimate the derived transfer function using at least one blind detection technique.

17. The camera of claim 11 wherein the image is a digital image.

18. The camera of claim 11 wherein the processor is configured to filter the image using a 2-dimensional finite impulse response (2D FIR) filter.

19. A method for processing an image, comprising:

deriving a two dimensional transfer function representing distortion in the image;

inverting the two dimensional transfer function to obtain an inverse transfer function;

configuring a corrective filter to implement the inverse transfer function; and

applying the corrective filter to the image to generate a corrected image.

20. The method of claim 19 wherein the deriving of the two dimensional transfer function is performed using blind estimation.

21. The method of claim 19 wherein the image distortion is created by motion of an imager when capturing the image, and wherein the deriving of the transfer function is performed by:

measuring the motion of the imager while the image is being captured; and

using the measured motion to generate the transfer function.

22. A computing device, comprising:

a memory configured to store an image;

a processor operatively associated with the memory, the processor configured to calculate a two dimensional transfer function representing distortion in the image, to generate an inverse transfer function from the two dimensional transfer function, to generate a corrective filter based on the inverse transfer function, and to apply the corrective filter to the image.

23. The computing device of claim 22, wherein the processor is configured to calculate the two dimensional transfer function via blind estimation.

24. The computing device of claim 22, wherein the image distortion is created by movement of an imager when the image is being captured.