NEURAL NETWORK TRAINING UTILIZING SPECIALIZED LOSS FUNCTIONS

• US 20200134382A1
• Filed: 11/02/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Abandoned Application

• Current Assignee: ABBYY Production LLC (ABBYY Software)
• Original Assignee: ABBYY Production LLC (ABBYY Software)
• Inventors: Zhuravlev, Aleksey
• Application Number:
  US16/178,979
• Publication Number:
  US 20200134382A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/698 Matching; Classification
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

Systems and methods for neural network training utilizing specialized loss functions. An example method comprises: receiving, by a computer system, a training dataset comprising a plurality of images, wherein each image of the training dataset is associated with an identifier of a class of a set of classes; computing, by a neural network, a plurality of feature vectors, wherein each feature vector of the plurality of feature vectors represents an image of the training dataset in a space of image features; computing, for the training dataset, a value of a loss function reflecting a plurality of probabilities, wherein each probability of the plurality of probabilities characterizes a hypothesis associating an image of the training dataset with a class associated with the image by the training dataset, wherein the loss function further reflects a plurality of distances, wherein each distance of the plurality of distances is computed in the space of image features between a feature vector representing an image of the training dataset and a center of a class associated with the image by the training dataset; and adjusting one or more parameters of the neural network based on the value of the loss function.

Claims

1. A method, comprising:

receiving, by a computer system, a training dataset comprising a plurality of images, wherein each image of the training dataset is associated with an identifier of a class of a set of classes;

computing, by a neural network, a plurality of feature vectors, wherein each feature vector of the plurality of feature vectors represents an image of the training dataset in a space of image features;

computing, for the training dataset, a value of a loss function reflecting a plurality of probabilities, wherein each probability of the plurality of probabilities characterizes a hypothesis associating an image of the training dataset with a class associated with the image by the training dataset, wherein the loss function further reflects a plurality of distances, wherein each distance of the plurality of distances is computed in the space of image features between a feature vector representing an image of the training dataset and a center of a class associated with the image by the training dataset; and

adjusting one or more parameters of the neural network based on the value of the loss function.

2. The method of claim 1, wherein the center of the class is represented by an average of features of images belonging to the class.

3. The method of claim 1, wherein computing the value of the loss function further comprises:

adjusting a center of a class of the set of classes.

4. The method of claim 1, wherein the loss function is represented by a linear combination of a cross entropy loss function and a center loss function.

5. The method of claim 1, further comprising:

performing, using the neural network, optical character recognition (OCR) of a grapheme image.

6. The method of claim 1, wherein each class of the set of classes corresponds to a character of an alphabet.

7. The method of claim 1, wherein the loss function further reflects one or more distances between a feature vector of a negative training sample and centers of one or more classes of the set of classes.

8. The method of claim 1, wherein the loss function is represented by a linear combination of a cross entropy loss function, a center loss function, and a close-to-center penalty loss function.

9. The method of claim 8, wherein the training dataset comprises a first plurality of valid grapheme images and a second plurality of invalid grapheme images.

10. The method of claim 8, wherein the training dataset comprises plurality of synthetic invalid grapheme images.

11. A system, comprising:

a memory;

a processor, coupled to the memory, the processor configured to;

receive a training dataset comprising a plurality of images, wherein each image of the training dataset is associated with an identifier of a class of a set of classes;

compute, by a neural network, a plurality of feature vectors, wherein each feature vector of the plurality of feature vectors represents an image of the training dataset in a space of image features;

compute, for the training dataset, a value of a loss function reflecting a plurality of probabilities, wherein each probability of the plurality of probabilities characterizes a hypothesis associating an image of the training dataset with a class associated with the image by the training dataset, wherein the loss function further reflects a plurality of distances, wherein each distance of the plurality of distances is computed in the space of image features between a feature vector representing an image of the training dataset and a center of a class associated with the image by the training dataset; and

adjust one or more parameters of the neural network based on the value of the loss function.

12. The system of claim 11, wherein computing the value of the loss function further comprises:

adjusting a center of a class of the set of classes.

13. The system of claim 11, wherein the loss function is represented by a linear combination of a cross entropy loss function and a center loss function.

14. The system of claim 11, wherein the processor is further configured to:

perform, using the neural network, optical character recognition (OCR) of a grapheme image.

15. The system of claim 11, wherein the loss function further reflects one or more distances between a feature vector of a negative training sample and centers of one or more classes of the set of classes.

16. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to:

receive a training dataset comprising a plurality of images, wherein each image of the training dataset is associated with an identifier of a class of a set of classes;

compute, by a neural network, a plurality of feature vectors, wherein each feature vector of the plurality of feature vectors represents an image of the training dataset in a space of image features;

compute, for the training dataset, a value of a loss function reflecting a plurality of probabilities, wherein each probability of the plurality of probabilities characterizes a hypothesis associating an image of the training dataset with a class associated with the image by the training dataset, wherein the loss function further reflects a plurality of distances, wherein each distance of the plurality of distances is computed in the space of image features between a feature vector representing an image of the training dataset and a center of a class associated with the image by the training dataset; and

adjust one or more parameters of the neural network based on the value of the loss function.

17. The computer-readable non-transitory storage medium of claim 16, wherein computing the value of the loss function further comprises:

adjusting a center of a class of the set of classes.

18. The computer-readable non-transitory storage medium of claim 16, wherein the loss function is represented by a linear combination of a cross entropy loss function and a center loss function.

19. The computer-readable non-transitory storage medium of claim 16, further comprising:

performing, using the neural network, optical character recognition (OCR) of a grapheme image.

20. The computer-readable non-transitory storage medium of claim 16, wherein the loss function is represented by a linear combination of a cross entropy loss function, a center loss function, and a close-to-center penalty loss function.