Method and device for removing motion blur effects

• US 8,559,751 B2
• Filed: 07/03/2006
• Issued: 10/15/2013
• Est. Priority Date: 07/12/2005
• Status: Active Grant

• Current Assignee: Goodix Technology Co., Ltd. (Shenzhen Goodix Technology Co., Ltd.)
• Original Assignee: NXP B.V. (NXP Semiconductors NV)
• Inventors: Gobert, Jean
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US11/995,037
• Publication Number:
  US 20100119171A1
• Time in Patent Office: 2,661 Days
• Field of Search: 382/107, 382/255, 382/264, 382/275, 34820899-20816
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

The present invention provides an improved method and device for digital motion blur removal by using motion information extracted from a sequence of images captured immediately before the image to be corrected was captured. In particular, the invention includes estimating (56) motion information of the previous sequence of images and analyzing them based on motion estimation techniques and then extrapolating (58) the motion of the image to be corrected based on the motion estimation in order to remove (62) the motion blur effects in the desired captured image. The various types of devices which may implement the method of the present invention will thereby display (64) blur free digital images.

Claims

1. A method of removing motion blur effects of a digital image that is subject to motion correction, wherein the method comprises:

extracting motion information from a sequence of a plurality of images captured immediately before a capture of the digital image;

extracting timing information of a start time and an end time of the capture of the digital image;

generating an estimated motion information for the sequence of images by performing an image to image analysis based on a motion estimation technique using the extracted motion information, wherein the estimated motion information comprises an individual motion vector for each pixel of the image;

extrapolating motion information of the digital image in a time interval between the start time and the end time of the capture of the digital image using the estimated motion information and the extracted timing information of the start time and the end time of the capture of the digital image; and

removing motion blur effects of the digital image by compensating motion of the digital image using the extrapolated motion information.

2. The method according to claim 1, wherein generating the estimated motion information further comprises providing an estimated motion vector for every block of pixels of the image.

3. The method according to claim 2, wherein generating the estimated motion information comprises providing an estimated motion vector using a global motion estimation model.

4. The method according to claim 3, wherein using the global motion estimation model from one image to the next image in the sequence of images comprises subdividing the image into blocks, using a block matching algorithm on each block, and producing as many motion block vectors as there are blocks.

5. The method according to claim 4, further comprising using a median filter on block motion vectors and deriving a global motion vector from the median filter.

6. The method according to claim 2, wherein generating the estimated motion information comprises providing an estimated motion vector using a block based motion model.

7. The method according to claim 2, wherein generating the estimated motion information comprises providing an estimated motion vector using a pixel flow model.

8. The method according to claim 1, wherein generating the estimated motion information comprises generating motion vectors corresponding to horizontal and vertical translations of motion.

9. The method according to claim 1, wherein generating the estimated motion information comprises generating motion vectors corresponding to rotational translation of motion.

10. The method according to claim 2, wherein generating the estimated motion information comprises computing, for each motion vector, an approximate function of time, and minimizing an error criteria at estimated offsets of the sequence of images.

16. The method according to claim 2, further comprising:

using a median filter on the estimated motion vectors for the blocks; and

deriving a global motion vector from the median filter.

11. A device configured to remove motion blur effects of a digital image subject to motion correction, wherein the device comprises:

an image sensor adapted to extract motion information from a sequence of images captured immediately before a capture of the digital image;

a capture module adapted to extract timing information of a start time and an end time of image capture;

a motion estimator adapted to generate an estimated motion information of the sequence of images by performing an image to image analysis based on a motion estimation technique using the extracted motion information, wherein the estimated motion information comprises an individual motion vector for each pixel of the image;

a motion extrapolator adapted to estimate motion information of the digital image in a time interval between the start time and the end time of the image using the estimated motion information and the extracted timing information of the start time and the end time of the capture of the digital image; and

a motion deblur module configured to remove motion blur effects of the digital image by compensating motion of the digital image using the extrapolated motion information.

12. The device according to claim 11, wherein the motion estimator is further configured to use a global motion estimation model to provide the individual motion vector for each pixel of the image.

13. The device according to claim 11, wherein the motion estimator provides an estimated motion vector using a block based motion model.

14. The device according to claim 11, wherein the motion estimator provides an estimated motion vector using a pixel flow model.

15. A non-transitory computer-readable medium having a sequence of instructions stored thereon which, when executed by a processor of a device, causes the processor to:

extract motion information from a sequence of images captured immediately before a capture of the digital image;

extract timing information of a start time and an end time of image capture;

generate an estimated motion information of the sequence of images by performing an image to image analysis based on a motion estimation technique using the extracted motion information, wherein the estimated motion information comprises an individual motion vector for each pixel of the image;

extrapolate motion information of the digital image in a time interval between the start time and the end time of the capture of the digital image using the estimated motion information and the extracted timing information of the start time and the end time of the capture of the digital image; and

remove motion blur effects of the digital image by compensating motion of the digital image using the extrapolated motion information.