Facilitating Text Identification and Editing in Images

• US 20170124417A1
• Filed: 01/17/2017
• Published: 05/04/2017
• Est. Priority Date: 11/14/2014
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Angara, Srinivasa Madhava Phaneendra, Bedi, Ajay
• Granted Patent: US 9,928,439 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 10/34 Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
• CPC Class Codes: G06V 30/19127 Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 30/245 Font recognition
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Facilitating text identification and editing in images in which in one or more embodiments, a user selection of a location in an area of text in an image is received. Given the location, a region of interest that includes text (including the location of the user selection) in the image is determined. Distortion resulting from a surface in the image on which the text is situated being at some angle other than parallel to the image capture plane is also corrected. One or more fonts and font sizes of the text in the region of interest are also detected. Various actions can be taken on the text in the region of interest (e.g., editing the text and/or identifying the text).

Claims

1. A method of removing text in an image by at least one computing device, the method comprising:

identifying, by the at least one computing device, a region of interest in the image including text;

transforming, by the at least one computing device, the region of interest to correct distortion based at least in part on a homography;

using, by the at least one computing device, an optical character recognizer to recognize a character based on a detected font used within the text;

receiving, by the at least one computing device, a user input to remove the character;

removing, by the at least one computing device, the character from the image based on a content aware fill process; and

outputting, by the at least one computing device, a result of the removing of the character from the image.

2. The method of claim 1, the transforming further comprising:

determining a transformation matrix to correct the distortion of the region of interest in the image;

transforming the pixels in the region of interest to a substantially rectangular region; and

recognizing, from the substantially rectangular region, characters of text.

3. The method of claim 2, wherein the homography comprises the transformation matrix, and the transformation matrix is a 3×

3 projective transformation matrix.

4. The method of claim 3, wherein values u_i refer to the coordinates of the vertices of the substantially rectangular region, values d_i refer to the coordinates of the vertices of the region of interest, a value H refers to the homography, and the homography is determined by solving an equation u_i=H*d_i.

5. The method of claim 1, wherein the optical character recognizer further comprises a font classifier trained on multiple fonts using linear discriminant analysis.

6. The method of claim 1, wherein the content aware fill process is configured to remove the character from the text by replacing locations where the character was present with a same appearance as a background on which the character was present.

7. The method of claim 1, wherein the removing the character from the image is based at least in part on the homography.

8. The method of claim 1, the distortion in the region of interest being a surface in the image on which the text is situated being at some non-zero angle relative to an image capture plane that captured the image.

9. The method of claim 1, further comprising:

receiving, as part of the user input, new text;

transforming the new text using a transformation matrix; and

replacing at least some of the text in the region of interest with the transformed new text.

10. A computing device comprising:

a processing system including one or more processors; and

one or more computer-readable storage media having stored thereon multiple instructions that, when executed by the processing system, cause the processing system to perform acts including;

identifying, by the at least one computing device, a region of interest in the image including text;

transforming, by the at least one computing device, the region of interest to correct distortion based at least in part on a homography;

using, by the at least one computing device, an optical character recognizer to recognize a character based on a detected font used within the text;

receiving, by the at least one computing device, a user input to remove the character;

removing, by the at least one computing device, the character from the image based on a content aware fill process; and

outputting, by the at least one computing device, a result of the removing of the character from the image.

11. The computing device of claim 10, the transforming further comprising:

determining a transformation matrix to correct the distortion of the region of interest in the image;

transforming the pixels in the region of interest to a substantially rectangular region; and

recognizing, from the substantially rectangular region, characters of text.

12. The computing device of claim 11, wherein the homography comprises the transformation matrix, and the transformation matrix is a 3×

3 projective transformation matrix.

13. The computing device of claim 12, wherein values u_i refer to the coordinates of the vertices of the substantially rectangular region, values d_i refer to the coordinates of the vertices of the region of interest, a value H refers to the homography, and the homography is determined by solving an equation u_i=H*d_i.

14. The computing device of claim 10, wherein the optical character recognizer further comprises a font classifier trained on multiple fonts using linear discriminant analysis.

15. The computing device of claim 10, wherein the content aware fill process is configured to remove the character from the text by replacing locations where the character was present with a same appearance as a background on which the character was present.

16. The computing device of claim 10, wherein the removing the character from the image is based at least in part on the homography.

17. The computing device of claim 10, the distortion in the region of interest being a surface in the image on which the text is situated being at some non-zero angle relative to an image capture plane that captured the image.

18. The computing device of claim 10, the acts further including:

receiving, as part of the user input, new text;

transforming the new text using a transformation matrix; and

replacing at least some of the text in the region of interest with the transformed new text.

19. In a digital medium environment to remove text in an image, a system comprising:

means for transforming a region of interest in the image to correct distortion based at least in part on a homography, the region of interest in the image including text;

means for receiving a user input to remove a character recognized within the text;

means for removing the character from the image based on a content aware fill process; and

means for outputting a result of the removing of the character from the image.

20. The system of claim 19, further comprising:

means for receiving, as part of the user input, new text;

means for transforming the new text using a transformation matrix; and

means for replacing at least some of the text in the region of interest with the transformed new text.