Edge direction and curve based image de-blurring

• US 9,076,205 B2
• Filed: 11/19/2012
• Issued: 07/07/2015
• Est. Priority Date: 11/19/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Cho, Sunghyun, Wang, Jue, Zhong, Lin
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/680,952
• Publication Number:
  US 20140140626A1
• Time in Patent Office: 960 Days
• Field of Search: 348/208.4, 348/241, 348/251, 348/254, 382/199, 382/203, 382/274, 382/279
• US Class Current: 1/1
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/13 Edge detection

Abstract

An image de-blurring system obtains a blurred input image and generates, based on the blurred input image, a blur kernel. The blur kernel is an indication of how the image capture device was moved and/or how the subject captured in the image moved during image capture. Based on the blur kernel and the blurred input image, a de-blurred image is generated. The blur kernel is generated based on the direction of edges identified in the blurred input image and/or based on curves having a high curvature identified in the image (e.g., corners identified in the image).

Claims

1. A method implemented in a device, the method comprising:

obtaining an input image;

obtaining an edge direction for each of multiple pixels identified in the input image;

selecting, by the device and using the edge direction of each of the multiple pixels, one or more of the multiple pixels, the selecting including;

assigning each of the multiple pixels to one of multiple groups based on an angle of the edge direction of the pixel, and

selecting, as the one or more pixels, a number of pixels from each of the multiple groups, the numbers of pixels selected from the multiple groups being within a threshold difference of one another; and

generating, by the device and using the selected one or more pixels, a blur kernel based on which a de-blurred version of the input image is generated.

2. A method as recited in claim 1, the method further comprising obtaining gradient magnitudes for each of the multiple pixels.

3. A method as recited in claim 2, the selecting the number of pixels from each of the multiple groups comprising:

applying an adaptive gradient magnitude threshold to the multiple groups so that different groups can have different gradient magnitude thresholds; and

selecting, as the one or more of the multiple pixels, pixels from the multiple groups having gradient magnitudes that satisfy the gradient magnitude threshold for the group.

4. A method as recited in claim 3, further comprising determining for each group a gradient magnitude threshold so that the selecting pixels from the multiple groups comprises selecting approximately the same number of pixels from each of the multiple groups.

5. A method as recited in claim 3, wherein at least one pixel selected from one of the multiple groups has a lower gradient magnitude than a pixel not selected from another of the multiple groups.

6. A method as recited in claim 1, further comprising:

receiving an indication of one or more curves each having an angle within a threshold amount of a right angle in the input image; and

the generating comprising generating, using the selected one or more pixels and the indicated one or more curves, the blur kernel.

7. A method as recited in claim 6, the generating further comprising generating the blur kernel by reducing an energy function based on second-order derivatives of the one or more curves.

8. A method as recite in claim 7, the energy function being further based on first-order derivatives of the selected one or more pixels.

9. One or more computer-readable storage media having stored thereon multiple instructions that, when executed by one or more processors of a computing device, cause the one or more processors to:

obtain an input image;

select one or more curves each having an angle within a threshold amount of a right angle in the input image; and

generate, using the selected one or more curves, a blur kernel based on which a de-blurred version of the input image is generated, the blur kernel being generated by reducing an energy function that includes pixels that are part of the selected one or more curves but excludes pixels that are not part of the selected one or more curves.

10. One or more computer-readable storage media as recited in claim 9, wherein to generate the blur kernel is further to generate, based on the selected one or more curves and one or more pixels of edges in the input image, the blur kernel.

11. One or more computer-readable storage media as recited in claim 10, the multiple instructions further causing the one or more processors to select, using an edge direction of each of multiple pixels in the input image, the one or more pixels.

12. One or more computer-readable storage media as recited in claim 10, the multiple instructions further causing the one or more processors to obtain both gradient magnitudes and edge directions for each of the one or more pixels.

13. One or more computer-readable storage media as recited in claim 12, the multiple instructions further causing the one or more processors to:

assign each of the multiple pixels to one of multiple groups based on an angle of the edge direction of the pixel;

apply an adaptive gradient magnitude threshold to the multiple groups so that different groups can have different gradient magnitude thresholds; and

select, as the one or more pixels, pixels from the multiple groups having gradient magnitudes that satisfy the gradient magnitude threshold for the group.

14. One or more computer-readable storage media as recited in claim 9, wherein to generate the blur kernel is to generate the blur kernel by reducing the energy function based on second-order derivatives of the one or more curves.

15. One or more computer-readable storage media as recited in claim 14, the energy function being further based on first-order derivatives of one or more pixels of edges identified in the input image.

16. One or more computer-readable storage media as recited in claim 14, the energy function being further based on, for each of the one or more curves, a rotation angle of the curve.

17. A system comprising:

an image access module configured to obtain an input image;

an edge selection module configured to;

obtain an edge direction for each of multiple pixels identified in the input image,

select, using the edge direction of each of the multiple pixels, one or more of the multiple pixels by assigning each of the multiple pixels to one of multiple groups based on an angle of the edge direction of the pixel, and selecting, as the one or more pixels, a number of pixels from each of the multiple groups, the numbers of pixels selected from the multiple groups being within a threshold difference of one another;

a curve selection module configured to select one or more curves each having an angle within a threshold amount of a right angle in the input image; and

a blur kernel generation module configured to generate, using the selected one or more pixels and the selected one or more curves, a blur kernel indicating how an image capture device was moved while capturing the input image.

18. A system as recited in claim 17, the edge selection module being further configured to obtain both gradient magnitudes and the edge directions for each of the multiple pixels.

19. A system as recited in claim 18, the edge selection module being further configured to:

apply an adaptive gradient magnitude threshold to the multiple groups so that different groups can have different gradient magnitude thresholds; and

select, as the one or more of the multiple pixels, pixels from the multiple groups having gradient magnitudes that satisfy the gradient magnitude threshold for the group.

20. A system as recited in claim 17, the blur kernel generation module being further configure to generate the blur kernel by reducing an energy function based on second-order derivatives of the one or more curves and on first-order derivatives of the selected one or more pixels.