Motion blur correction

• US 20050231603A1
• Filed: 04/19/2004
• Published: 10/20/2005
• Est. Priority Date: 04/19/2004
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Poon, Eunice
• Granted Patent: US 7,561,186 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/75 Unsharp masking

Abstract

A method and apparatus of correcting blur in a motion blurred image includes estimating the direction of blur in the motion blurred image based on edge response of the motion blurred image over a set of discrete directions and over subgroups of the discrete directions. The extent of blur in the motion blurred image is also estimated. An initial guess image based on the motion blurred image is generated and the guess image is blurred as a function of the estimated blur direction and blur extent. The blurred guess image is compared with the motion blurred image to generate an error image. The error image is blurred and weighted and then combined with the initial guess image thereby to update the guess image and correct for blur.

Claims

1. A method of correcting blur in a motion blurred image using an estimate of motion blur direction and motion blur extent based on the motion blurred image, the method comprising:

generating an initial guess image based on said motion blurred image;

blurring the guess image as a function of said estimated blur direction and blur extent;

comparing the blurred guess image with the motion blurred image to generate an error image;

blurring and weighting the error image; and

combining the error image and the initial guess image thereby to update the guess image and correct for blur.

2. The method of claim 1, wherein said weighting is a function of the blur direction.

3. The method of claim 2, wherein said weighting is an estimate of the edge magnitude of said guess image in said blur direction.

4. The method of claim 3 wherein the edge magnitude is estimated using a high-pass filter.

5. The method of claim 2 wherein said initial guess image is said motion blurred image.

6. The method of claim 5, wherein said weighting is an estimate of the edge magnitude of said guess image in said blur direction.

7. The method of claim 2 wherein said blurring, comparing, blurring and weighting, and combining are performed iteratively.

8. The method of claim 7 wherein said blurring, comparing, blurring and weighting, and combining are performed iteratively a threshold number of times.

9. The method of claim 7 wherein said blurring, comparing, blurring and weighting, and combining are performed iteratively until the magnitude of the error image falls below a threshold level.

10. The method of claim 7 wherein said blurring, comparing, blurring and weighting, and combining are performed iteratively until the error image fails to change by more than a threshold amount between successive iterations.

11. The method of claim 7 wherein prior to performing said method said blur extent is compared with a threshold blur extent level, said method being performed only when the estimate of the motion blur extent is greater than said threshold blur extent level.

12. The method of claim 11, wherein said weighting is an estimate of the edge magnitude of said guess image in said blur direction.

13. The method of claim 7, wherein said weighting is an estimate of the edge magnitude of said guess image in said blur direction.

14. A method of correcting blur in a motion blurred image comprising:

estimating the direction of blur in said motion blurred image based on edge response of said motion blurred image over a set of discrete directions extending through said motion blurred image and over subgroups of said discrete directions;

estimating the extent of blur in said motion blurred image;

generating an initial guess image based on said motion blurred image;

blurring the guess image as a function of said estimated blur direction and blur extent;

comparing the blurred guess image with the motion blurred image to generate an error image;

blurring the error image; and

combining the error image and the initial guess image thereby to update the guess image and correct for blur.

15. The method of claim 14 wherein said set of discrete directions includes N discrete directions, said discrete directions being angularly spaced over the angular space of said motion blurred image between 0 and 180 degrees.

16. The method of claim 15 wherein said discrete directions are equi-angularly spaced and wherein N is an even number.

17. The method of claim 16 wherein during said blur direction estimating the edge response over a plurality of subgroup combinations of discrete directions is determined and compared with the edge response over the set of discrete directions, each subgroup combination partitioning said set of discrete directions into a pair of quadrants, with discrete directions in at least one of said quadrants being consecutive.

18. The method of claim 17 wherein said blur direction estimating further comprises:

for each subgroup combination, determining the deviation between the edge response of said motion blurred image over said set of discrete directions and the edge responses of said motion blurred image over the discrete directions in each of said quadrants;

for the subgroup combination yielding the highest deviation, declaring the direction that bisects the one quadrant as the blur direction when the edge response over the discrete directions in the one quadrant is less than the edge response over the discrete directions in the other quadrant; and

otherwise declaring the direction normal to the direction that bisects the one quadrant as the blur direction.

19. The method of claim 14 further comprising weighting said error image prior to said combining to inhibit blur correction from occurring in areas not requiring blur correction.

20. The method of claim 19, wherein said weighting is a function of the blur direction.

21. The method of claim 20, wherein said weighting is an estimate of the edge magnitude of said guess image in said blur direction.

22. The method of claim 21 wherein the edge magnitude is estimated using a high-pass filter.

23. The method of claim 21 wherein said initial guess image is said motion blurred image.

24. The method of claim 21 wherein said blurring, comparing, blurring, weighting, and combining are performed iteratively.

25. The method of claim 24 wherein said blurring, comparing, blurring, weighting, and combining are performed iteratively a threshold number of times.

26. The method of claim 24 wherein said blurring, comparing, blurring, weighting, and combining are performed iteratively until the magnitude of the error image falls below a threshold level.

27. The method of claim 24 wherein said blurring, comparing, blurring, weighting, and combining are performed iteratively until the error image fails to change by more than a threshold amount between successive iterations.

28. The method of claim 24 wherein prior to said generating said blur extent is compared with a threshold blur extent level, said generating, blurring, comparing, blurring, weighting and combining being performed only when the estimate of the motion blur extent is greater than said threshold blur extent level.

29. The method of claim 24 wherein said set of discrete directions includes N discrete directions, said discrete directions being angularly spaced over the angular space of said motion blurred image between 0 and 180 degrees.

30. The method of claim 29 wherein said discrete directions are equi-angularly spaced and wherein N is an even number.

31. The method of claim 30 wherein during said blur direction estimating the edge response over a plurality of subgroup combinations of discrete directions is determined and compared with the edge response over the set of discrete directions, each subgroup combination partitioning said set of discrete directions into a pair of quadrants, with discrete directions in at least one of said quadrants being consecutive.

32. The method of claim 31 wherein said blur direction estimating further comprises:

for each subgroup combination, determining the deviation between the edge response of said motion blurred image over said set of discrete directions and the edge responses of said motion blurred image over the discrete directions in each of said quadrants;

for the subgroup combination yielding the highest deviation declaring the direction that bisects the one quadrant as the blur direction when the edge response over the discrete directions in the one quadrant is less than the edge response over the discrete directions in the other quadrant; and

otherwise declaring the direction normal to the direction that bisects the one quadrant as the blur direction.

33. The method of claim 19, wherein said blur extent is estimated using a correlation based method.

34. The method of claim 21, wherein said blur extent is estimated using a correlation based method.

35. The method of claim 32, wherein said blur extent is estimated using a correlation based method.

36. A method of estimating blur direction in a motion blurred image comprising:

determining the deviation between the edge response of said motion blurred image over a set of discrete spaced directions extending through said motion blurred image and the edge response of said motion blurred image over possible subgroup combinations of the discrete directions in said set, each subgroup combination including a pair of quadrants with one quadrant comprising a plurality of consecutive discrete directions;

for the subgroup combination yielding the highest deviation declaring the direction that bisects the one quadrant as the blur direction when the edge response over the discrete directions in the one quadrant is less than the edge response over the discrete directions in the other quadrant; and

otherwise declaring the direction normal to the direction that bisects the one quadrant as the blur direction.

37. The method of claim 36 wherein said set of discrete sample directions includes N discrete directions, said discrete directions being angularly spaced over the angular space of said motion blurred image between 0 and 180 degrees.

38. The method of claim 37 wherein said discrete directions are equi-angularly spaced and wherein N is an even number.

39. The method of claim 38 wherein said quadrants for each subgroup combination are defined by the equations:

Q₁=[θ

_n,θ

_n+π

/2) and Q₂=[θ

_n+π

/2, θ

_n+π

).

where;

θ

_n is a discrete direction.

40. The method of claim 39, wherein the deviation in edge response DEV_n is calculated according to the equation:

(|1st quadrant avg−

overall avg|+|2^nd quadrant avg−

overall avg|)/2.

41. An apparatus for estimating blur direction in a motion blurred image comprising:

an edge detector determining the edge response through said motion blurred image along a plurality of different discrete directions forming a set;

a deviation determinor determining the deviation between the edge response of said motion blurred image over said set of discrete spaced directions and the edge response of said motion blurred image over possible subgroup combinations of said set, each subgroup combination including a pair of quadrants with one quadrant comprising a plurality of consecutive discrete directions; and

a comparator determining the subgroup combination yielding the highest deviation, said comparator for that determined subgroup combination declaring the direction that bisects the one quadrant as the blur direction when the edge response over the discrete directions in the one quadrant is less than the edge response over the discrete directions in the other quadrant and otherwise declaring the direction normal to the direction that bisects the one quadrant as the blur direction.

42. An apparatus according to claim 41 wherein said edge detector is a Sobel edge detector.

43. An apparatus according to claim 42 wherein said set of discrete sample directions includes N discrete directions, said discrete directions being equi-angularly spaced over the angular space of said motion blurred image between 0 and 180 degrees, N being an even number.

44. The method of claim 43 wherein said quadrants for each subgroup combination are defined by the equations:

Q₁=[θ

_n,θ

_n+π

/2) and Q₂=[θ

_n+π

/2,θ

_n+π

).

where;

θ

_n is a discrete direction.

45. The method of claim 44, wherein said deviation determiner calculates the deviation in edge response DEV_n according to the equation:

(|1st quadrant avg−

overall avg|+|2^nd quadrant avg−

overall avg|)/2.

46. An apparatus for correcting blur in a motion blurred image using an estimate of motion blur direction and motion blur extent based on the motion blurred image, said apparatus comprising:

a motion blur filter blurring an initial guess image based on said motion blurred image as a function of said estimated blur direction and blur extent;

a comparator comparing the blurred guess image with the motion blurred image and generating an error image, said motion blur filter further blurring the error image and weighting the error image according to an error weighting factor; and

an adder combining the error image and the initial guess image thereby to update the guess image and correct for blur.

47. An apparatus according to claim 46 further comprising a filter generating said error weighting factor as a function of said blur direction.

48. An apparatus according to claim 47 wherein said filter is a high-pass filter that calculates the edge magnitude of said guess image in the blur direction thereby to generate said error weighting factor.