Font Recognition using Text Localization

• US 20180239995A1
• Filed: 04/25/2018
• Published: 08/23/2018
• Est. Priority Date: 10/06/2015
• Status: Active Grant

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Wang, Zhaowen, Liu, Luoqi, Jin, Hailin
• Granted Patent: US 10,467,508 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/245 Font recognition

Abstract

Font recognition and similarity determination techniques and systems are described. In a first example, localization techniques are described to train a model using machine learning (e.g., a convolutional neural network) using training images. The model is then used to localize text in a subsequently received image, and may do so automatically and without user intervention, e.g., without specifying any of the edges of a bounding box. In a second example, a deep neural network is directly learned as an embedding function of a model that is usable to determine font similarity. In a third example, techniques are described that leverage attributes described in metadata associated with fonts as part of font recognition and similarity determinations.

Claims

1. In a digital medium environment to improve image font recognition through use of text localization, a method implemented by one or more computing devices comprising:

obtaining a model, by the one or more computing devices, that is trained using machine learning as applied to a plurality of training images having text rendered using a corresponding font;

predicting a bounding box, automatically and without user intervention by the one or more computing devices, for text in an image received using the obtained model by forming a plurality of cropped portions of the image and processing each of the plurality of cropped portions of the image by the model independently, one to another; and

generating an indication of the predicted bounding box by the one or more computing devices, the indication usable to specify a region of the image that includes the text having a font to be recognized.

2. The method as described in claim 1, further comprising recognizing the font of the text in the received image by the one or more computing devices using the generated indication of the predicted bounding box.

3. The method as described in claim 1, wherein the predicting includes processing each of the plurality of cropped portions of the image by a trained convolutional network of the model independently, one to another.

4. The method as described in claim 1, wherein the predicting further comprising generating the predicted bounding box based on a result of the processing of each of the plurality of cropped portions of the image.

5. The method as described in claim 4, wherein the generating is performed by taken an average, a median, or through use of a line fitting algorithm.

6. The method as described in claim 1, wherein the predicting includes resizing the image by the one or more computing devices to correspond to an image size of the model.

7. The method as described in claim 1, further comprising training the model by the one or more computing devices using the machine learning for a plurality of iterations.

8. The method as described in claim 7, wherein the training is performed for at least one of the plurality of iterations using the plurality of training images having text rendered using the corresponding font and performed for one or more subsequent ones of the plurality of iterations in which one or more perturbations are introduced to the training images.

9. The method as described in claim 8, wherein the perturbations includes at least one of noise, rotation, scale, shading, rotation, kerning, or cropping.

10. The method as described in claim 7, wherein the machine learning is performed by the one or more computing devices using a convolutional neural network, the convolutional neural network is used as an architecture of the machine learning by the one or more computing devices and stochastic gradient decent is used as a training algorithm of the machine learning by the one or more computing devices.

11. The method as described in claim 1, wherein the font to be recognized in the image is arbitrary such that the model is trainable without using the font.

12. In a digital medium environment to improve image font recognition through use of text localization, a method implemented by one or more computing devices comprising:

obtaining a model, by the one or more computing devices, that is trained using machine learning over a plurality of iterations using a plurality of training image, at least one iteration of the plurality of iterations using the plurality of training images having text rendered using the corresponding font and performed for one or more subsequent ones of the plurality of iterations in which one or more perturbations are introduced to the training images;

predicting a bounding box, automatically and without user intervention by the one or more computing devices, for text in an image received using the obtained model; and

generating an indication of the predicted bounding box by the one or more computing devices, the indication usable to specify a region of the image that includes the text having a font to be recognized.

13. The method as described in claim 12, wherein the perturbations include noise.

14. The method as described in claim 12, wherein the perturbations include rotation.

15. The method as described in claim 12, wherein the perturbations include scaling.

16. The method as described in claim 12, wherein the perturbations include shading.

17. The method as described in claim 12, wherein the perturbations include rotation.

18. The method as described in claim 12, wherein the perturbations include kerning.

19. The method as described in claim 12, wherein the perturbations include cropping.

20. In a digital medium environment to train a model to improve image font recognition through use of text localization, a system comprising one or more computing devices including a processing system and memory having instructions stored thereon that are executable by the processing system to perform operations comprising:

obtaining a plurality of training images having text rendered using a corresponding font images including;

an anchor image having text rendered using a corresponding font type;

the positive image having text that is different than the text of the anchor image or text having one or more applied perturbations; and

the negative image having text that is not in the font type; and

training the model to predict a bounding box for text in an image, the model trained using machine learning as applied to the plurality of training images having text rendered using the corresponding font.