Method and apparatus to correct digital image blur due to motion of subject or imaging device

• US 8,154,607 B2
• Filed: 11/19/2008
• Issued: 04/10/2012
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Clear Imaging Research, LLC
• Original Assignee: Fatih M. Ozluturk
• Inventors: Ozluturk, Fatih M.
• Primary Examiner(s): HO, TUAN V
• Application Number:
  US12/274,032
• Publication Number:
  US 20090135259A1
• Time in Patent Office: 1,238 Days
• Field of Search: 348/208.2, 348/208.7, 348/208.14
• US Class Current: 348/208.7
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/615 involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
• CPC Class Codes: H04N 5/265 Mixing

Abstract

Signal processing techniques are applied to digital image data to remove the distortion caused by motion of the camera, or the movement of the subject being photographed, or defective optics, or optical distortion from other sources. When the image is captured, the effect of relative motion between the camera and the subject is that it transforms the true image into a blurred image according to a 2-dimensional transfer function. The 2-dimensional transfer function representing the motion is derived using blind estimation techniques or by using information from sensors that detect the motion. The transfer function is inverted and used to define a corrective filter. The filter is applied to the image and the blur due to the motion is removed, restoring the correct image. Another embodiment uses the transfer function to avoid blur by combining multiple consecutive images taken at a fast shutter speed.

Claims

1. A camera, comprising:

an image sensor configured to record data image data that represents an image;

a sensor configured to detect a subject of the image in the image data and to measure motion of the subject in the image data; and

an image sensor adjuster configured to move the image sensor relative to the camera in a direction opposite to a direction of the motion of the subject.

2. The camera of claim 1 wherein the image sensor adjuster is an electromagnet.

3. The camera of claim 1 wherein the image sensor adjuster is a servo.

4. The camera of claim 1 wherein the camera is a digital camera.

5. The camera of claim 1 wherein the image sensor includes a Charge-Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) sensor.

6. A method for use in a device that includes an image sensor, a subject motion sensor, and an image sensor adjuster, the method comprising:

the image sensor recording image data that represents an image;

the subject motion sensor detecting a subject of the image in the image data and measuring motion of the subject in the image data; and

the image sensor adjuster moving the image sensor relative to the device in a direction opposite to a direction of the motion of the subject.

7. The method of claim 6 wherein the image sensor adjuster is an electromagnet.

8. The method of claim 6 wherein the image sensor adjuster is a servo.

9. The method of claim 6 wherein the device is a digital camera.

10. The method of claim 6 wherein the image sensor includes a Charge-Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS) sensor.