Repairing holes in images

• US 9,785,856 B2
• Filed: 02/29/2016
• Issued: 10/10/2017
• Est. Priority Date: 02/29/2016
• Status: Active Grant

• Current Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Original Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Inventors: Guarnieri, Stuart
• Primary Examiner(s): Do, Anh H
• Application Number:
  US15/056,687
• Publication Number:
  US 20170249525A1
• Time in Patent Office: 589 Days
• Field of Search: 382178, 382283, 382225, 257 59, 257 72, 349 43, 359381, 359818, 356432, 356433, 356441, 707706, 707722
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/148 Segmentation of character regions
• CPC Class Codes: G06V 30/18076 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 30/184 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 30/186 by deriving mathematical or geometrical properties from the whole image

Abstract

A method for image processing that includes: obtaining a mask of a connected component (CC) from an image; generating a stroke width transform (SWT) image based on the mask; calculating multiple stroke width parameters for the mask based on the SWT image; identifying a hole in the CC of the mask; calculating a stroke width estimate for the hole based on the stroke width values of pixels in the SWT image surrounding the hole; generating a comparison of the stroke width estimate for the hole with a limit based on the multiple stroke width parameters for the mask; and generating a revised mask by filling the hole in response to the comparison.

Claims

1. A method for image processing using a computer processor, comprising:

obtaining a mask of a connected component (CC) from an image;

generating, by the computer processor, a first stroke width transform (SWT) image based on the mask;

calculating a plurality of stroke width parameters for the mask based on the first SWT image;

identifying a first hole in the CC of the mask;

calculating a stroke width estimate for the first hole based on a plurality of stroke width values of a plurality of pixels in the first SWT image surrounding the first hole;

generating a first comparison of the stroke width estimate for the first hole with a limit based on the plurality of stroke width parameters for the mask; and

generating, by the computer processor, a revised mask by filling the first hole in response to the first comparison.

2. The method of claim 1, comprising:

identifying second hole in the CC of the mask; and

calculating a stroke width estimate for the second hole based on the first SWT image,

wherein generating the revised mask further comprises filling the second hole in response to the stroke width estimate for the second hole being less than the limit.

3. The method of claim 1, comprising:

identifying a second hole in the CC of the mask;

generating a second SWT image based on the revised mask;

calculating a plurality of stroke width parameters for the revised mask based on the second SWT image;

calculating a stroke width estimate for the second hole based on the second SWT image;

generating a second comparison of the stroke width estimate for the second hole with a limit based on the plurality of stroke width parameters for the revised mask; and

filling the second hole in the revised mask in response to the second comparison.

4. The method of claim 3, wherein the second SWT is generated in response to an area of the filled holes exceeding a filled holes threshold.

5. The method of claim 1, wherein:

the plurality of stroke width parameters comprises a mean stroke width and a standard deviation;

the limit is a summation of a scaled version of the mean stroke width and a scaled version of the standard deviation; and

the stroke width estimate for the first hole is a mean of the plurality of stroke width values of the plurality of pixels in the first SWT image surrounding the first hole.

6. The method of claim 5, wherein the scaled version and the mean stroke width are scaled by the same scaling factor of 0.73.

7. The method of claim 1, further comprising:

identifying a second hole in the CC of the mask;

calculating an area of the second hole;

determining a ratio of the area of the second hole to an area of the CC exceeds a threshold area ratio; and

keeping the second hole unfilled in response to the ratio exceeding the threshold area ratio.

8. The method of claim 1, further comprising:

generating an auxiliary mask of the first hole;

dilating the auxiliary mask by one pixel; and

obtaining the plurality of stroke width values by executing an intersection operation of the auxiliary mask with the first SWT image after dilating the auxiliary mask.

9. The method of claim 1, further comprising:

performing optical character recognition on the revised mask,

wherein the CC corresponds to a hand-drawn stroke on a writing board in the image.

10. A non-transitory computer readable medium (CRM) storing computer readable program code embodied therein that, when executed by a computer processor:

obtains a mask of a connected component (CC) from an image;

generates a first stroke width transform (SWT) image based on the mask;

calculates a plurality of stroke width parameters for the mask based on the first SWT image;

identifies a first hole in the CC of the mask;

calculates a stroke width estimate for the first hole based on a plurality of stroke width values of a plurality of pixels in the first SWT image surrounding the first hole;

generates a first comparison of the stroke width estimate for the first hole with a limit based on the plurality of stroke width parameters for the mask; and

generates a revised mask by filling the first hole in response to the first comparison.

11. The non-transitory CRM of claim 10, further storing computer readable program code embodied therein that, when executed by the computer processor:

identifies second hole in the CC of the mask; and

calculates a stroke width estimate for the second hole based on the first SWT image,

wherein generating the revised mask further comprises filling the second hole in response to the stroke width estimate for the second hole being less than the limit.

12. The non-transitory CRM of claim 10, further storing computer readable program code embodied therein that, when executed by the computer processor:

identifies a second hole in the CC of the mask;

generates a second SWT image based on the revised mask;

calculates a plurality of stroke width parameters for the revised mask based on the second SWT image;

calculates a stroke width estimate for the second hole based on the second SWT image;

generates a second comparison of the stroke width estimate for the second hole with a limit based on the plurality of stroke width parameters for the revised mask; and

fills the second hole in the revised mask in response to the second comparison.

13. The non-transitory CRM of claim 10, wherein:

the plurality of stroke width parameters comprises a mean stroke width and a standard deviation;

the limit is a summation of a scaled version of the mean stroke width and a scaled version of the standard deviation; and

the stroke width estimate for the first hole is a mean of the plurality of stroke width values of the plurality of pixels in the first SWT image surrounding the first hole.

14. The non-transitory CRM of claim 10, further storing computer readable program code embodied therein that, when executed by the computer processor:

identifies a second hole in the CC of the mask;

calculates an area of the second hole;

determines a ratio of the area of the second hole to an area of the CC exceeds a threshold area ratio; and

keeps the second hole unfilled in response to the ratio exceeding the threshold area ratio.

15. The non-transitory CRM of claim 10, further storing computer readable program code embodied therein that, when executed by the computer processor:

generates an auxiliary mask of the first hole;

dilates the auxiliary mask by one pixel; and

obtains the plurality of stroke width values by executing an intersection operation of the auxiliary mask with the first SWT image after dilating the auxiliary mask.

16. A system for image processing, comprising:

a memory storing a mask of a connected component (CC) from an image; and

a computer processor connected to the memory that;

identifies a first hole in the CC of the mask;

generates a first stroke width transform (SWT) image based on the mask;

calculates a plurality of stroke width parameters for the mask based on the first SWT image;

calculates a stroke width estimate for the first hole based on a plurality of stroke width values of a plurality of pixels in the first SWT image surrounding the first hole;

generates a first comparison of the stroke width estimate for the first hole with a limit based on the plurality of stroke width parameters for the mask; and

generates a revised mask by filling the first hole in response to the first comparison.

17. The system of claim 16, wherein the computer processor also:

generates an auxiliary mask of the first hole;

dilates the auxiliary mask by one pixel; and

obtains the plurality of stroke width values by executing an intersection operation of the auxiliary mask with the first SWT image after dilating the auxiliary mask.

18. The system of claim 16, wherein the computer processor also:

identifies a second hole in the CC of the mask;

generates a second SWT image based on the revised mask;

calculates a plurality of stroke width parameters for the revised mask based on the second SWT image;

calculates a stroke width estimate for the second hole based on the second SWT image; and

generates a second comparison of the stroke width estimate for the second hole with a limit based on the plurality of stroke width parameters for the revised mask; and

fills the second hole in the revised mask in response to the second comparison.

19. The system of claim 16, wherein the computer processor also:

identifies a second hole in the CC of the mask;

calculates an area of the second hole;

determines a ratio of the area of the second hole to an area of the CC exceeds a threshold area ratio; and

keeps the second hole unfilled in response to the ratio exceeding the threshold area ratio.

20. The system of claim 16, wherein the CC corresponds to a hand-drawn stroke on a writing board in the image, and wherein optical character recognition is performed on the revised mask.