Method and apparatus for robust estimation of nonuniform motion blur

0Associated
Cases 
0Associated
Defendants 
0Accused
Products 
0Forward
Citations 
0
Petitions 
1
Assignment
First Claim
1. A method of estimating a nonuniform motion blur, the method comprising:
 receiving an input image comprising a nonuniform motion blur;
estimating homographies for the received input image, the estimated homographies comprising a homograph set;
iteratively increasing a number of estimated homographies in the homography set; and
obtaining a latent image based on a change of an error value of the homography set.
1 Assignment
0 Petitions
Accused Products
Abstract
A method and apparatus for robust estimation of a nonuniform motion blur that may reduce an amount of the nonuniform motion blur information, that is, a number of homographies by estimating nonuniform motion blur information about a blur in a predetermined area, thereby reducing an amount of time needed to remove the nonuniform motion blur, and may improve accuracy and stability of the nonuniform motion blur information by estimating homographies for an input image while increasing a number of the homographies, iteratively.
26 Citations
No References
ESTIMATION OF POINT SPREAD FUNCTIONS FROM MOTIONBLURRED IMAGES  
Patent #
US 20100231732A1
Filed 03/10/2010

Current Assignee
Qualcomm Inc.

Sponsoring Entity
Qualcomm Inc.

IMAGE CAPTURING DEVICE, IMAGE CAPTURING METHOD, AND STORAGE MEDIUM HAVING STORED THEREIN IMAGE CAPTURING PROGRAM  
Patent #
US 20100209009A1
Filed 02/16/2010

Current Assignee
Casio Computer Company

Sponsoring Entity
Casio Computer Company

Image processing device, focal plane distortion component calculation method, image processing program, and recording medium  
Patent #
US 20100214423A1
Filed 02/12/2010

Current Assignee
Sony Corporation

Sponsoring Entity
Sony Corporation

Adaptive PSF Estimation Technique Using a Sharp Preview and a Blurred Image  
Patent #
US 20100329582A1
Filed 06/21/2010

Current Assignee
FotoNation Limited

Sponsoring Entity
FotoNation Limited

BLOCKBASED IMAGE STABILIZATION  
Patent #
US 20090123082A1
Filed 11/12/2007

Current Assignee
Qualcomm Inc.

Sponsoring Entity
Qualcomm Inc.

Removing camera shake from a single photograph  
Patent #
US 20080025627A1
Filed 07/28/2006

Current Assignee
University of Toronto

Sponsoring Entity
Massachusetts Institute of Technology

Estimating A Point Spread Function Of A Blurred Digital Image Using Gyro Data  
Patent #
US 20080100716A1
Filed 08/14/2007

Current Assignee
Seiko Epson Corporation

Sponsoring Entity
Seiko Epson Corporation

IMAGE PROCESSING APPARATUS AND IMAGE PICKUP APPARATUS  
Patent #
US 20080151064A1
Filed 12/14/2007

Current Assignee
Canon Kabushiki Kaisha

Sponsoring Entity
Canon Kabushiki Kaisha

Digital image stabilization method for correcting horizontal inclination distortion and vertical scaling distortion  
Patent #
US 20080225127A1
Filed 03/11/2008

Current Assignee
Samsung SDI Company Limited

Sponsoring Entity
Samsung SDI Company Limited

IMAGE STABILIZING APPARATUS, IMAGEPICKUP APPARATUS AND IMAGE STABILIZING METHOD  
Patent #
US 20080246848A1
Filed 04/03/2008

Current Assignee
Canon Kabushiki Kaisha

Sponsoring Entity
Canon Kabushiki Kaisha

Apparatus and method for removing motion blur of image  
Patent #
US 20080253676A1
Filed 10/31/2007

Current Assignee
Samsung SDI Company Limited

Sponsoring Entity
Samsung SDI Company Limited

Robust reconstruction of high resolution grayscale images from a sequence of low resolution frames  
Patent #
US 20070217713A1
Filed 11/16/2006

Current Assignee
Sina Farsiu, Peyman Milanfar, Michael Elad, Michael Robinson

Sponsoring Entity
Sina Farsiu, Peyman Milanfar, Michael Elad, Michael Robinson

Method and apparatus for initiating subsequent exposures based on determination of motion blurring artifacts  
Patent #
US 20060098237A1
Filed 11/10/2004

Current Assignee
FotoNation Limited

Sponsoring Entity
FotoNation Limited

Methods and apparatus for computing the input and output signals of a linear shiftvariant system  
Patent #
US 20060101106A1
Filed 09/26/2005

Current Assignee
Muralidhara Subbarao

Sponsoring Entity
Muralidhara Subbarao

Systems and methods for deblurring motion blurred images  
Patent #
US 20060119710A1
Filed 11/03/2004

Current Assignee
Trustees Of Columbia University In The City Of New York

Sponsoring Entity
Trustees Of Columbia University In The City Of New York

Circuit and method for reducing voltage spikes due to magnetizing current imbalances and power converter employing the same  
Patent #
US 20060152950A1
Filed 01/11/2005

Current Assignee
ABB Schweiz AG

Sponsoring Entity
ABB Schweiz AG

Method for deblurring an image  
Patent #
US 20060187308A1
Filed 02/23/2005

Current Assignee
HewlettPackard Development Company L.P.

Sponsoring Entity
HewlettPackard Development Company L.P.

Digital camera with integrated accelerometers  
Patent #
US 6,747,690 B2
Filed 07/03/2001

Current Assignee
Phase One AS

Sponsoring Entity
Phase One AS

Digital camera with integrated accelerometers  
Patent #
US 20040212699A1
Filed 05/18/2004

Current Assignee
Phase One AS

Sponsoring Entity
Phase One AS

Deconvolution method for the analysis of data resulting from analytical separation processes  
Patent #
US 5,748,491 A
Filed 12/20/1995

Current Assignee
Applied Biosystems LLC

Sponsoring Entity
PERKIN ELMER CORPORATION THE APPLIED BIOSYSTEMS DIVISION

Motion Blur Device, Method and Program  
Patent #
US 20110299793A1
Filed 02/15/2010

Current Assignee
National University Corporation Shizuoka University

Sponsoring Entity
National University Corporation Shizuoka University

Methods and Apparatus for Deblurring Images Using Lucky Frames  
Patent #
US 20120121202A1
Filed 11/30/2010

Current Assignee
Adobe Inc.

Sponsoring Entity
Adobe Inc.

SmartphoneBased Methods and Systems  
Patent #
US 20120284012A1
Filed 11/02/2011

Current Assignee
Digimarc Corporation

Sponsoring Entity
Digimarc Corporation

Removing motion blur from unaligned multiple blurred images  
Patent #
US 8,411,980 B1
Filed 02/24/2011

Current Assignee
Adobe Inc.

Sponsoring Entity
Adobe Systems Incorporated

Moving object detection using a mobile infrared camera  
Patent #
US 8,446,468 B1
Filed 06/19/2008

Current Assignee
University of Southern California

Sponsoring Entity
University of Southern California

METHOD AND DEVICE FOR RECOVERING A DIGITAL IMAGE FROM A SEQUENCE OF OBSERVED DIGITAL IMAGES  
Patent #
US 20130242129A1
Filed 09/28/2011

Current Assignee
MaxPlanckGesellschaft Zur Foerderung Der Wissenschaften e.V.

Sponsoring Entity
MaxPlanckGesellschaft Zur Foerderung Der Wissenschaften e.V.

21 Claims
 1. A method of estimating a nonuniform motion blur, the method comprising:
receiving an input image comprising a nonuniform motion blur; estimating homographies for the received input image, the estimated homographies comprising a homograph set; iteratively increasing a number of estimated homographies in the homography set; and obtaining a latent image based on a change of an error value of the homography set.  View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13)
 12. A method for robust estimation of a nonuniform motion blur, the method comprising:
calculating, by a processor, homographies of an input image, the calculated homographies comprising a homography set; iteratively increasing a number of calculated homographies in the homography set; and obtaining a final image of the input image using the calculated homographies, based on a change of an error value of the homography set.
 14. An apparatus for estimating a nonuniform motion blur, the apparatus comprising:
a processor to control one or more processorexecutable units; a receiving unit to receive an input image comprising a nonuniform motion blur; an estimating unit to estimate homographies for the received input image, the estimated homographies comprising a homography set, and iteratively increasing a number of estimated homographies in the homography set; and an obtaining unit to obtain a latent image based on a change of an error value of the homography set.  View Dependent Claims (15, 16, 17, 18, 19, 20, 21)
1 Specification
This application claims the priority benefit of Korean Patent Application No. 1020120022875, filed on Mar. 6, 2012, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.
1. Field
Example embodiments of the following disclosure relate to a method and apparatus for robust estimation of a nonuniform motion blur, and more particularly, to a method and apparatus for estimation of nonuniform motion blur from an image by estimating homographies iteratively.
2. Description of the Related Art
A blur is a phenomenon which commonly occurs during a process of obtaining an image while using an apparatus for obtaining an image, such as, a camera, and the like. The blur phenomenon is one of the main contributors to deterioration of image quality.
When an image is obtained using the apparatus for obtaining an image in an environment where an amount of light is insufficient, for example, a dark indoor location or an outdoor location in the evening, a sufficient amount of light is required to obtain a clear image. Accordingly, an image sensor may be exposed to light for a longer period of time than usual in order to obtain the sufficient amount of light. However, when an exposure time is too long, a blur may occur in the obtained image due to the image sensor being shaken during the period of time that the image sensor is exposed to light.
Removing a blur from an image may be difficult since an amount of information required may be greater than an amount of information provided. In particular, each pixel of an image generally includes a nonuniform motion blur in different directions and of different sizes due to a translational motion and a rotational motion of a camera.
Accordingly, a method and apparatus for robust estimation of nonuniform motion blur is needed.
The foregoing and/or other aspects are achieved by providing a method of estimating a nonuniform motion blur, the method including receiving an input image including a nonuniform motion blur, estimating nonuniform motion blur information about a blur, included in the nonuniform motion blur, in a predetermined area included in the input image, and obtaining a latent image by removing the nonuniform motion blur from the input image, based on the estimated nonuniform motion blur information.
The estimating of the nonuniform motion blur information may include estimating homographies for the input image, and estimating the nonuniform motion blur information by matching the homographies to the blur in the predetermined area.
The estimating of the nonuniform motion blur information may include estimating at least two homographies for the input image and a homography between the at least two homographies, iteratively.
The method may further include normalizing weights for the homographies, using the at least two homographies and the homography between the at least two homographies.
The method may further include performing the estimating of the nonuniform motion blur information and the obtaining of the latent image, iteratively.
The performing may include terminating an iterative performance of the estimating of the nonuniform motion blur information and the obtaining of the latent image, based on a change in an error value with respect to a homography set including the estimated homographies.
The method may further include restoring a final image from the input image, based on obtained final nonuniform motion blur information when the iterative performance is completed.
The input image may correspond to at least one of a single frame including the nonuniform motion blur, a multiframe including the nonuniform motion blur, and a multiframe including a single blurred image including the nonuniform motion blur and a single noise image without a blur.
The foregoing and/or other aspects are achieved by providing an apparatus for estimating a nonuniform motion blur, the apparatus including a receiving unit to receive an input image comprising a nonuniform motion blur, an estimating unit to estimate nonuniform motion blur information about a blur in a predetermined area included in the input image, and an obtaining unit to obtain a latent image by removing the nonuniform motion blur from the input image, based on the estimated nonuniform motion blur information.
The estimating unit may include a first estimator to estimate homographies for the input image, and a second estimator to estimate the nonuniform motion blur information by matching the homographies to the blur in the predetermined area.
The estimating unit may estimate at least two homographies for the input image and a homography between the at least two homographies, iteratively.
The apparatus may include a weight normalizing unit to normalize weights for the homographies, using the at least two homographies and the homography between the at least two homographies.
The apparatus may include an iterative performance unit to perform estimation of the nonuniform motion blur information and obtaining of the latent image, iteratively.
The iterative performance unit may complete an iterative performance of the estimation of the nonuniform motion blur information and the obtaining of the latent image, based on a change in an error value with respect to a homography set including the estimated homographies.
The apparatus may include a restoring unit to restore a final image from the input image, based on obtained final nonuniform motion blur information when the iterative performance is completed.
The input image may correspond to at least one of a single frame including the nonuniform motion blur, a multiframe including the nonuniform motion blur, and a multiframe including a single blurred image including the nonuniform motion blur and a single noise image without a blur.
The foregoing and/or other aspects of the present disclosure are achieved by providing a method for robust estimation of a nonuniform motion blur, the method including: estimating, by a processor, homographies of an input image, the estimated homographies comprising a homography set; iteratively increasing a number of estimated homographies in the homography set; and obtaining a final image of the input image using the estimated homographies, based on a change of an error value of the homography set.
Additional aspects of embodiments will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure.
These and/or other aspects will become apparent and more readily appreciated from the following description of embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. Embodiments are described below to explain the present disclosure by referring to the figures.
Referring to
When it is assumed that a motion blur or a trace of a camera being shaken while the image sensor is exposed to light appears as shown in 110, then 120 may indicate an approximate motion blur estimated using a blur model of Equation 3, and 130 may indicate an approximate motion blur estimated using a blur model of Equation 7.
Generally, a motion blur may be expressed by Equation 1.
B=K*L+N, [Equation 1]
where B denotes a blurred image, and K denotes a point spread function (PSF) or a motion blur kernel indicating blur information of an image. L denotes a latent image, that is, a clear image without a blur. N denotes an unknown noise occurring during a process of obtaining an image, and * denotes a convolution operator.
Equation 1 may be expressed by Equation 2 in a vectorial form.
where b, l, and n denote vector expressions of B, L, and N of Equation 1. T_{i }denotes a determinant representing a translational motion of a camera at a point in time t_{i}, and w_{i }denotes a relative length of time when the camera stops at the point in time t_{i}, that is, an exposure time of the camera at the point in time t_{i}. Here, Σ_{i}w_{i}=1.
Equation 2 may indicate that the blurred image B may be expressed using a sum of latent images L at each point on a route of the camera, that is, the determinant Ti. Here, Equation 1 and Equation 2 may express an identical model in different manners.
The latent images L may be computed using a motion blur model of Equation 1 or Equation 2. In this instance, since a blur model for estimating a latent image may assume that all pixels included in an image may be uniformly moved, it may be difficult to remove a nonuniform motion blur occurring due to a rotational motion, rather than a translational motion of a camera.
Accordingly, a nonuniform motion blur model describing a nonuniform motion blur effect of a camera may be derived by substituting T_{i }of Equation 2 with a homography P_{i}, as expressed by Equation 3.
where b, l, and n denote vector expressions of the blurred image B, the latent image L, and the unknown noise N, respectively. P_{i }denotes a matrix representing a projective transform motion of the camera at the point in time t_{i}, and w_{i }denotes a relative length of time when the camera stops at the point in time t_{i}, that is, an exposure time of the camera at the point in time t_{i}. Here, Σ_{i}w_{i}=1.
Equation 3 may indicate that the blurred image B may be expressed using a sum of latent images L at each coordinate and each viewpoint on a route of the camera.
In methods of dealing with a nonuniform motion blur, when the blurred image B and the latent image L being currently estimated are provided, a process of estimating a nonuniform motion blur that may exist in the blurred image B by comparing the blurred image B and the latent image L may be of significant importance. A method of estimating the nonuniform motion blur will be described hereinafter.
The method of estimating the nonuniform motion blur may include two operations, that is, estimation of a homography and estimation of a weight. When the blurred image B and the latent image L are provided, the nonuniform motion blur may be estimated by performing the two operations iteratively.
With respect to the estimation of the homography, each homography may be estimated using Equation 4, which is modified from Equation 3.
In order to compute a single homography P_{i }in Equation 4, an image registration algorithm that may reduce a difference between
of the left side and w_{i}P_{i}l of the right side may be applied. An entire homography set P may be obtained by computing every P_{i }while changing an index i of each homography P_{i}, in Equation 4.
When the entire homography set P is computed, a weight w of a homography may be computed using the computed homography set P.
With respect to computing the weight w, Equation 3 may be expressed as Equation 5.
b=Aw+n, [Equation 5]
where A=[P_{1}l P_{2}l . . . P_{n}l], and A corresponds to an mbyn (m×n) matrix. Here, m denotes a number of pixels included in an image, and n denotes a number of homographies.
Generally, m>>n, and the weight w in Equation 5 may need to have a value greater than or equal to 0. Accordingly, a nonnegative least square method may be used.
In order to use the nonnegative least square method, the weight w may be computed using Equation 6, expressed in a form of a normal equation.
w=(A^{T}A+βI)^{−1}A^{T}b, [Equation 6]
where β denotes a normalized parameter to be used for resolving a case in which an inverse matrix of a determinant in parenthesis is absent. I denotes an identity matrix.
The method of estimating the homography and the method of estimating the weight may be used in common for a blur model that may be assumed in an input image corresponding to a multiframe including a nonuniform motion blur or a multiframe including a blurred image and a noise image.
However, in Equation 3, since every motion of the camera may be defined using each homography, the greater the camera movement, the greater the number of homographies may be required to describe the motion of the camera. Accordingly, a considerable amount time may be used for a process of estimating and removing the nonuniform motion blur.
Generally, in order to estimate a nonuniform motion blur, M homographies may be assumed, and the M homographies may be estimated sequentially one by one. Although this example embodiment estimates homographies sequentially one by one, the present disclosure is not limited thereto. In a process of estimating a homography P_{i}, when another homography P_{j }has a wrong value, it may be likely to estimate a wrong value for the homography P_{i }as well, and the nonuniform motion blur may be estimated inaccurately. In addition, a weight w_{i }of each homography may be estimated after the M homographies are estimated. In this instance, when a great number of homographies have similar values, estimation of the weight w, may become unstable. Consequently, the process of estimating the nonuniform motion blur may become unstable.
According to example embodiments, a blur model, for example, a nonuniform motion blur model, differing from the model of Equation 3 may be used to remove the nonuniform motion blur using a relatively small number of homographies. Also, in order to increase stability in the process of estimating the nonuniform motion blur, the process of estimating the nonuniform motion blur may be performed while progressively increasing a number of homographies starting with a small number of homographies, for example, starting with at least two homographies.
In Equation 3, each homography may correspond to a single point of each PSF of Equation 1. Equation 3 may be modified to Equation 7 to derive a new blur model.
where G denotes an operator indicating a blur component in a small area. For example, the Gaussian blur operator may be used.
(P_{i}G) in Equation 7 may correspond to a blur component of a size of an area corresponding to a blur in a small area, as opposed to the signal point of each PSF of Equation 1. Accordingly, Equation 7 may be used to express an identical blur using a fewer number of homographies, when compared to Equation 3. Also, a latent image L may be downsampled by the Gaussian blur operation G, and thus, the blur may be estimated faster.
In order to increase the stability of the operation of estimating the nonuniform motion blur, the process of progressively increasing a number of homographies starting with a small number of homographies in the operation of estimating the nonuniform motion blur will be described. When a latent image L and a blurred image B are provided as inputs, only two homographies may be estimated at first, for example. Although this example embodiment estimates two homographies at first, the present disclosure is not limited thereto.
Each of the two homographies may be estimated using Equation 8 that is modified from Equation 7.
A homography P_{i }that may reduce a difference between
of the left side and w_{i}P_{i}Gl of the right side may be computed using an image registration algorithm.
An entire homography set P may be obtained by computing every P_{i }while changing an index i of each homography P_{i}, in Equation 8. Also, by iterating the process of computing every P_{i }while changing the index i, the entire homography set P may be improved progressively. In this instance, the foregoing process may be performed iteratively within a predetermined number of times until a reduction in an error value of the homography set P is less than a predetermined reference value.
An error in the homography set P may be defined as expressed by Equation 9.
where N_{pixels }denotes a number of pixels in an image.
After the two homographies are estimated using Equation 8, another homography connecting the two homographies may be added. In this instance, the added homography and a weight of the added homography may be computed as expressed by Equation 10 and Equation 11, respectively.
P′=0.5P_{1}+0.5P_{2} [Equation 10]
w′=0.5w_{1}+0.5w_{2} [Equation 11]
Here, the added homography P′ and the weight w′ of the added homography P′ may be arranged between P_{1 }and P_{2}. That is, a new homography set may be defined as P^{new}={P_{1}, P′, P_{2}}={P_{1}^{new}, P_{2}^{new}, P_{3}^{new}}, and a new weight set may be defined as w^{new}={w_{1}, w^{1}, w_{2}}={w_{1}^{new}, w_{2}^{new}, w_{3}^{new}}.
After the homography is added using Equation 10 and Equation 11, a sum of weights of all homographies may be greater than ‘1.’ Accordingly, in order to make the sum of the all weights be ‘1,’ the weights may be normalized using Equation 12.
The three homographies, that is, P_{1}^{new}, P_{2}^{new}, P_{3}^{new }may be improved using Equation 8, discussed earlier. After the three homographies are estimated, a homography connecting P_{1 }and P_{2 }and a homography connecting P_{2 }and P_{3 }may be added, and the foregoing process may be iterated. When the estimation of the three homographies is completed, still another homography may be added, and the foregoing process may be iterated, in an identical manner. The iterative process may be performed until a reduction in the error value, as defined in Equation 9, is less than a predetermined reference value.
When the estimation is started using a small number of homographies, stable estimation may be possible since estimation of a homography P_{i }may be affected by another homography relatively less when compared to estimation being started using a great number of homographies. Also, since the number of homographies may be increased sequentially one by one from the small number of homographies until an error is reduced, such that the reduction of the error value is less than a predetermined reference value, the number of homographies may be computed automatically, absent a need for predetermining a number of required homographies.
In operation 201, an apparatus for estimating a nonuniform motion blur, hereinafter referred to as an estimation apparatus, may receive an input image including a nonuniform motion blur.
In operation 203, the estimation apparatus may estimate nonuniform motion blur information about a blur, included in the nonuniform motion blur, in a predetermined area of the input image. In particular, the estimation apparatus may estimate homographies for the input image, and may estimate the nonuniform motion blur information by matching the homographies to the blur in the predetermined area. In this instance, the estimation apparatus may progressively increase a number of estimated homographies by estimating at least two homographies for the input image and a homography between the at least two homographies, iteratively.
Here, a method of progressively increasing the number of estimated homographies will be further described with reference to
Also, the estimation apparatus may normalize weights for the homographies using the at least two homographies and the homography between the at least two homographies.
In operation 205, the estimation apparatus may obtain a latent image L by removing the nonuniform motion blur from the input image, based on the estimated nonuniform motion blur information.
In operation 207, the estimation unit may perform the operation 203 of estimating the nonuniform motion blur information and the operation 205 of obtaining the latent image L, iteratively.
In operation 209, the estimation unit may determine whether a change in an error value with respect to a homography set including the estimated homographies is less than a predetermined reference value, during the iterative performance.
When the change in the error value is less than the predetermined reference value in 209, the estimation apparatus may determine that the input image may be improved, and may terminate or complete the iterative performance in operation 211.
Conversely, when the change in the error value is greater than or equal to the reference value in 209, the estimation apparatus may determine that the input image may be greatly improved by the iterative performance, and may continue performing the operations 203 through 207.
In operation 213, the estimation apparatus may restore a final image from the input image, using obtained final nonuniform motion blur information when the iterative performance is completed.
Here, the input image may correspond to at least one of a single frame including the nonuniform motion blur, a multiframe including the nonuniform motion blur, and a multiframe including a single blurred image including the nonuniform motion blur and a single noise image without a blur.
A method of removing a nonuniform motion blur using the method of estimating the nonuniform motion blur when the input image corresponds to the multiframe including the nonuniform motion blur will be described with reference to
In operation 203 of
In operation 303, the estimation apparatus may estimate homographies for an input image, and calculate an error value for each of the estimated homographies. In this instance, the estimation apparatus may estimate nonuniform motion blur information by matching the homographies to a blur in a predetermined area of the input image.
At first, the estimation apparatus may estimate at least two homographies for the input image, and a homography between the at least two homographies, iteratively.
In operation 305, the estimation apparatus may determine whether a reduction in the error value, from operation 303, is less than a predetermined reference value. Here, the predetermined reference value may refer to a predetermined value at which an error value is determined to be scarcely reduced. In this instance, when the reduction in the error value for the estimated homography is determined to be greater than or equal to the predetermined reference value, the estimation apparatus may obtain at least two homographies and an additional homography between the at least two homographies. In operation 307, the estimation apparatus may normalize weights for the homographies, that is, the homography set, using the at least two homographies and the additional homography. That is, the estimation apparatus may return to the operation 303, and may perform the process of estimating the homography.
Conversely, in operation 305, when the reduction in the error value for the homography is determined to be less than the predetermined reference value, the estimation apparatus may complete the operation. That is, the estimation apparatus may terminate the iterative operation when the error value for the estimated homography is reduced, such that the reduction in the error value is less than a predetermined reference value.
That is,
In operation 410, two homographies 401 and 403 may be estimated at first. In operation 420, a new homography 405 connecting the two homographies 401 and 403 may be added, and a nonuniform motion blur estimated.
After a nonuniform motion blur is estimated using the three homographies 401, 403, and 405, a new homography 407 connecting the homographies 403 and 405 and a new homography 409 connecting the homographies 401 and 405 may be added and the nonuniform motion blur may be estimated, in operation 430. Using the foregoing process, the nonuniform motion blur may be estimated while increasing sequentially one by one a number of the homographies in operation 440, until the reduction in the error value of the set of homographies is less than a predetermined reference value.
Referring to
In operation 530, a latent image L may be obtained by removing the nonuniform motion blur from the multiframe, based on the estimated nonuniform motion blur information. In operation 540, whether the obtained latent image L satisfies a predetermined quality may be determined. When the obtained latent image L fails to satisfy the predetermined quality, the nonuniform motion blur information may be reestimated, that is, updated, based on the obtained latent image L, and the latent image L may be updated based on the updated nonuniform motion blur information.
Conversely, when the obtained latent image L satisfies the predetermined quality in operation 540, a final image may be restored from the multiframe, based on the nonuniform motion blur information or updated final nonuniform motion blur information, in operation 550.
Referring to
Noise may be removed from the noise image in operation 620, and nonuniform motion blur information about the blurred image may be estimated, using the noise image from which the noise is removed, and the blurred image, in operation 630.
In operation 640, a final restored image may be obtained using the obtained final nonuniform motion blur information.
Referring to
The receiving unit 710 may receive an input image including a nonuniform motion blur. The input image may correspond to at least one of a single frame including the nonuniform motion blur, a multiframe including the nonuniform motion blur, and a multiframe including a single blurred image including the nonuniform motion blur and a single noise image without a blur.
The estimating unit 720 may estimate nonuniform motion blur information about a blur in a predetermined area included in the input image. The estimating unit 720 may include a first estimator 723, and a second estimator 726.
The first estimator 723 may estimate homographies for the input image, and the second estimator 726 may estimate nonuniform motion blur information by matching the homographies to the blur in the predetermined area.
Also, the estimating unit 720 may estimate at least two homographies for the input image, and a homography between the at least two homographies, iteratively.
The obtaining unit 730 may obtain a latent image L by removing the nonuniform motion blur from the input image, based on the estimated nonuniform motion blur information.
The restoring unit 740 may restore a final image from the input image, using the obtained final nonuniform motion blur information when the iterative performance has completed.
The iterative performance unit 750 may perform estimation of the nonuniform motion blur information and obtaining of the latent image, iteratively. Also, the iterative performance unit 750 may complete the iterative performance of the estimation of the nonuniform motion blur information and the obtaining of the latent image, based on a change in an error value with respect to a homography set including the estimated homographies.
That is, when the change in the error value with respect to the homography set is lower than a predetermined reference level, the iterative performance unit 750 may determine that the input image may be improved, and may complete the iterative performance.
The weight normalizing unit 760 may normalize weights for the homographies, using the at least two homographies and the homography between the at least two homographies.
According to example embodiments, by estimating nonuniform motion blur information about a blur in a predetermined area, an amount of the nonuniform motion blur information, that is, a number of homographies, may be reduced, and thus, the amount of time needed to remove a nonuniform motion blur may be reduced.
According to example embodiments, by estimating homographies for an input image while increasing a number of the homographies, accuracy and stability of nonuniform motion blur information may be improved.
The methods according to the abovedescribed embodiments may be recorded in nontransitory computerreadable media including program instructions to implement various operations embodied by a computer, including a processor. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. Examples of nontransitory computerreadable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM discs and DVDs; magnetooptical media such as optical discs; and hardware devices that are specially configured to store and perform program instructions, such as readonly memory (ROM), random access memory (RAM), flash memory, and the like. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. Examples of the magnetic recording apparatus include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape (MT). Examples of the optical disk include a DVD (Digital Versatile Disc), a DVDRAM, a CDROM (Compact DiscRead Only Memory), and a CDR (Recordable)/RW. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the abovedescribed embodiments, or vice versa.
Further, according to an aspect of the embodiments, any combinations of the described features, functions and/or operations can be provided.
Moreover, the apparatus for estimating a nonuniform blur may include at least one processor to execute at least one of the abovedescribed units and methods.
Although embodiments have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the disclosure, the scope of which is defined by the claims and their equivalents.