Motion blurred image restoring method

• US 20080232707A1
• Filed: 07/16/2007
• Published: 09/25/2008
• Est. Priority Date: 03/23/2007
• Status: Abandoned Application

• Current Assignee: Industrial Technology Research Institute, National Tsing Hua University
• Original Assignee: Industrial Technology Research Institute, National Tsing Hua University
• Inventors: Lai, Shang-Hong, Chen, Shih-Chieh, Lee, Wen-Hao, Chen, Chia-Lun
• Application Number:
  US11/826,479
• Publication Number:
  US 20080232707A1
• Time in Patent Office: Days
• US Class Current: 382/255
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/223 using block-matching

Abstract

A motion blurred image restoring method includes following steps. A blur parameter is estimated through a global motion relation between a target image and an image next to the target image, and a restored image is generated through the blur parameter. In order to avoid errors from occurring to the estimated blur parameter, the blue parameter is further adjusted according to the image quality value of the restored image, such that the restored image has a more desirable image quality.

Claims

1. A motion blurred image restoring method, comprising:

reading a target image and a reference image neighboring to the target image of the motion blurring;

comparing the target image with the reference image to obtain a global motion relation between the target image and the reference image, and generating at least one blur parameter through the global motion relation; and

restoring the target image through the blur parameter to generate a restored image.

2. The motion blurred image restoring method as claimed in claim 1, wherein the global motion relation is generated after comparing the reference image with the target image by using a robust estimation process.

3. The motion blurred image restoring method as claimed in claim 1, wherein the global motion relation comprises a translation distance, a rotation angle, and a scaling between the reference image and the target image.

4. The motion blurred image restoring method as claimed in claim 1, wherein the step of generating the blur parameter comprises:

generating at least one block motion vector of the target image through the reference image;

calculating each block motion vector through a robust estimation process to generate a global motion relation, and generating a global motion vector for describing the global motion relation; and

defining the blur parameter with the global motion vector.

5. The motion blurred image restoring method as claimed in claim 4, wherein the step of generating at least one block motion vector includes comparing the reference image and the target image through a motion estimation process.

6. The motion blurred image restoring method as claimed in claim 4, wherein the step of generating at least one block motion vector comprises:

performing an eigen-decomposition on a structure tensor corresponding to the block motion vector to generate eigenvalues; and

eliminating the block motion vector corresponding to the eigenvalues, if it is determined that the block corresponding to the eigenvalues has a homogeneous or one-dimension structure.

7. The motion blurred image restoring method as claimed in claim 4, wherein the step of generating the global motion vector further comprises:

selecting a plurality of block motion vectors, for calculating a parameter of an affine module;

predicting the other block motion vectors through the parameter of the affine module;

calculating a residual error between the predicted block motion vectors and the corresponding original block motion vectors, and determining whether the original block motion vectors are inliers or not according to the residual error;

determining the global motion relation through the affine module parameter that generates the most batches of the inliers; and

generating a motion vector for describing the global motion through a translation distance of the global motion relation, including a horizontal translation and a vertical translation.

8. The motion blurred image restoring method as claimed in claim 4, wherein the blur parameter comprises a blur angle, and the blur angle is the direction of the global motion vector.

9. The motion blurred image restoring method as claimed in claim 4, wherein the blur parameter comprises a blur extent, and the blur extent is the length of the global motion vector.

10. The motion blurred image restoring method as claimed in claim 1, wherein the step of generating the blur parameter comprises:

extracting at least one image feature of the restored image through at least one image feature extraction method;

calculating an image quality value of the restored image through the image feature; and

adjusting the blur parameter, if the image quality value does not reach a preset threshold or is not stable.

11. The motion blurred image restoring method as claimed in claim 10, wherein the image feature extraction method comprises:

determining a change of a contrast and a smoothness between the restored image and the target image; and

calculating and generating the image feature according to the change of the contrast and the smoothness.

12. The motion blurred image restoring method as claimed in claim 10, wherein the image feature extraction method comprises:

performing an edge detection on the restored image to define at least one edge point;

searching a first pixel with partially maximum strength and a second pixel with partially minimum strength along a gradient direction of the edge point;

defining a distance between the first pixel and the second pixel as an edge width corresponding to the edge point; and

generating the image feature from a probability distribution of the edge width.

13. The motion blurred image restoring method as claimed in claim 10, wherein the image feature extraction method comprises:

transforming the restored image to a Fourier image;

calculating a frequency spectrum of coordinates for each point in the Fourier image, thereby obtaining a frequency spectrum distribution diagram; and

generating the image feature through each signal strength at different frequencies in the frequency spectrum distribution diagram.

14. The motion blurred image restoring method as claimed in claim 10, wherein the step of extracting the image feature further comprises a step of integrating the image feature through a normalization process.

15. The motion blurred image restoring method as claimed in claim 10, wherein the step of calculating an image quality value of the restored image is to calculate the image quality value with a pre-trained image quality assessment module.

16. The motion blurred image restoring method as claimed in claim 15, wherein the process of establishing the pre-trained image quality assessment module comprises:

collecting representative real images with desired focusing;

defining simulative blur parameters, and blurring the real images with the simulative blur parameters, thereby generating blurred images corresponding to the simulative blur parameters;

restoring the blurred images with a blur parameter set formed by correct simulative blur parameters and at least one false simulative blue parameter, thereby generating a plurality of sample images corresponding to the blur parameter set, wherein the sample images generated by restoring the blurred image with the correct simulative blur parameter are reference images that are marked with the highest sample image quality value; and

the sample images generated by restoring the blurred image with the false simulative blur parameters are false restored images;

extracting the sample image feature of each sample image through the image feature extraction method;

calculating the sample image quality value of the false restored image through a similarity between the false restored image and the reference image; and

inputting the sample image feature and the sample image quality value to a machine learning method, such that the machine learning method learns to suitably judge the image quality value of the restored image from the image feature of the restored image.

17. The motion blurred image restoring method as claimed in claim 16, wherein the machine learning method is a neural network.

18. The motion blurred image restoring method as claimed in claim 10, wherein the step of adjusting the blur parameter comprises adjusting the blur parameter through using a numerical optimization process.

19. The motion blurred image restoring method as claimed in claim 10, further comprising defining a stop criterion, wherein if the restored image satisfies the stop criterion, the restored image is output.