Differential classification using multiple neural networks

• US 10,565,478 B2
• Filed: 06/16/2017
• Issued: 02/18/2020
• Est. Priority Date: 05/30/2017
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY Production LLC (ABBYY Software)
• Inventors: Alekseevich, Zhuravlev Aleskey, Rybkin, Vladimir, Vladimirovich, Anisimovich Konstantin, Aydarovich, Davletshin Azat
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/625,894
• Publication Number:
  US 20180349742A1
• Time in Patent Office: 977 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A classification engine stores a plurality of neural networks in memory, where each neural network is trained to recognize a set of confused graphemes from one or more sets of confused graphemes identified in recognition data for a plurality of document images. The classification engine receives an input grapheme image associated with a document image comprising a plurality of graphemes, determines a set of recognition options for the input grapheme image, wherein the set of recognition options comprises a set of target characters that are similar to the input grapheme image, selects a first neural network from the plurality of neural networks, wherein the first neural network is trained to recognize a first set of confused graphemes, and wherein the first set of graphemes comprises at least a portion of the set of recognition options for the input grapheme image, and determines a grapheme class for the input grapheme image using the selected first neural network.

Claims

1. A method comprising:

analyzing a plurality of document images to construct a weighted graph that associates a plurality of pairs of confusing graphemes with a corresponding number of occurrences for each pair;

generating, using the weighted graph, a plurality of sets of confused graphemes based on recognition data for the plurality of document images, wherein each set of confused graphemes from the plurality of sets of confused graphemes comprises a plurality of different graphemes that are graphically similar to each other;

storing a plurality of neural networks in memory, wherein each neural network of the plurality of neural networks is trained to recognize a set of confused graphemes from the plurality of sets of confused graphemes;

receiving an input grapheme image associated with a document image comprising a plurality of grapheme images;

determining a set of recognition options for the input grapheme image, wherein the set of recognition options comprises a set of target characters that are similar to the input grapheme image;

selecting, by a processing device, a first neural network from the plurality of neural networks, wherein the first neural network is trained to recognize a first set of confused graphemes, and wherein the first set of confused graphemes comprises at least a portion of the set of recognition options for the input grapheme image; and

determining a grapheme class for the input grapheme image using the selected first neural network.

2. The method of claim 1, wherein analyzing the plurality of document images further comprises:

collecting statistical data associated with the recognition data for the plurality of document images;

identifying the plurality of pairs of confusing graphemes using the statistical data, wherein a first pair of confusing graphemes of the plurality of pairs comprises a first grapheme and a second grapheme that are graphically similar; and

selecting a first set of confused graphemes using the weighted graph.

3. The method of claim 2, wherein selecting the first set of confused graphemes using the weighted graph comprises:

determining a set size for each of the plurality of sets of confused graphemes;

determining a first weighted edge with a greatest weight value;

identifying a first pair of connected nodes that are connected by the first weighted edge;

selecting, for the first set of confused graphemes, the graphemes associated with the first pair of connected nodes; and

responsive to determining that a number of selected graphemes has not met the set size;

identifying an additional node that is connected at least one of the nodes associated with the first pair of connected nodes, and

selecting, for the set of confused graphemes, the grapheme associated with the additional node.

4. The method of claim 1, further comprising:

training each neural network of the plurality of neural networks to recognize a corresponding set of confused graphemes.

5. The method of claim 1, wherein determining the set of recognition options for the input grapheme image comprises:

analyzing the input grapheme image using a simple classifier;

determining one or more target characters with characteristics that are similar to the input grapheme image; and

assigning the one or more target characters to the set of recognition options.

6. The method of claim 1, wherein selecting the first neural network comprises:

comparing the set of recognition options to the plurality of sets of confused graphemes;

determining an intersection between the set of recognition options and each of the plurality of sets of confused graphemes;

selecting a first set of the plurality of sets of confused graphemes, wherein the intersection between the set of recognition options and the first set of confused graphemes is greater than the intersection between the set of recognition options and each of a remaining one or more sets of confused graphemes; and

determining that the first neural network is associated with the first set of the plurality of sets of confused graphemes.

7. The method of claim 6, wherein determining the intersection further comprises:

ranking a set of recognition options, wherein top recognition options have a highest confidence levels; and

determining the intersection between the first set of confused graphemes and the top recognition options from the ranked set of recognition options.

8. A computing apparatus comprising:

a memory to store instructions; and

a processing device, operatively coupled to the memory, to execute the instructions, wherein the processing device is to;

analyze a plurality of document images to construct a weighted graph that associates a plurality of pairs of confusing graphemes with a corresponding number of occurrences for each pair;

generate, using the weighted graph, a plurality of sets of confused graphemes based on recognition data for the plurality of document images, wherein each set of confused graphemes from the plurality of sets of confused graphemes comprises a plurality of different graphemes that are graphically similar to each other;

store a plurality of neural networks in memory, wherein each neural network of the plurality of neural networks is trained to recognize a set of confused graphemes from the plurality of sets of confused graphemes;

receive an input grapheme image associated with a document image comprising a plurality of grapheme images;

determine a set of recognition options for the input grapheme image, wherein the set of recognition options comprises a set of target characters that are similar to the input grapheme image;

select a first neural network from the plurality of neural networks, wherein the first neural network is trained to recognize a first set of confused graphemes, and wherein the first set of confused graphemes comprises at least a portion of the set of recognition options for the input grapheme image; and

determine a grapheme class for the input grapheme image using the selected first neural network.

9. The computing apparatus of claim 8, wherein to analyze the plurality of document images, the processing device is further to:

collect statistical data associated with the recognition data for the plurality of document images;

identify the plurality of pairs of confusing graphemes using the statistical data, wherein a first pair of confusing graphemes of the plurality of pairs comprises a first grapheme and a second grapheme that are graphically similar; and

select a first set of confused graphemes using the weighted graph.

10. The computing apparatus of claim 9, wherein to select the first set of confused graphemes using the weighted graph, the processing device is further to:

determine a set size for each of the plurality of sets of confused graphemes;

determine a first weighted edge with a greatest weight value;

identify a first pair of connected nodes that are connected by the first weighted edge;

select, for the first set of confused graphemes, the graphemes associated with the first pair of connected nodes; and

responsive to determining that a number of selected graphemes has not met the set size;

identify an additional node that is connected at least one of the nodes associated with the first pair of connected nodes, and

select, for the set of confused graphemes, the grapheme associated with the additional node.

11. The computing apparatus of claim 8, wherein the processing device is further to:

train each neural network of the plurality of neural networks to recognize a corresponding set of confused graphemes.

12. The computing apparatus of claim 8, wherein to determine the set of recognition options for the input grapheme image, the processing device is further to:

analyze the input grapheme image using a simple classifier;

determine one or more target characters with characteristics that are similar to the input grapheme image; and

assign the one or more target characters to the set of recognition options.

13. The computing apparatus of claim 8, wherein to select the first neural network, the processing device is further to:

compare the set of recognition options to the plurality of sets of confused graphemes;

determine an intersection between the set of recognition options and each of the plurality of sets of confused graphemes;

select a first set of the plurality of sets of confused graphemes, wherein the intersection between the set of recognition options and the first set of confused graphemes is greater than the intersection between the set of recognition options and each of a remaining one or more sets of confused graphemes; and

determine that the first neural network is associated with the first set of the plurality of sets of confused graphemes.

14. The computing apparatus of claim 13, wherein to determine the intersection, the processing device is further to:

rank a set of recognition options, wherein top recognition options have a highest confidence levels; and

determine the intersection between the first set of confused graphemes and the top recognition options from the ranked set of recognition options.

15. A non-transitory computer readable storage medium, having instructions stored therein, which when executed by a processing device of a computer system, cause the processing device to perform operations comprising:

analyzing a plurality of document images to construct a weighted graph that associates a plurality of pairs of confusing graphemes with a corresponding number of occurrences for each pair;

generating, using the weighted graph, a plurality of sets of confused graphemes based on recognition data for the plurality of document images, wherein each set of confused graphemes from the plurality of sets of confused graphemes comprises a plurality of different graphemes that are graphically similar to each other;

storing a plurality of neural networks in memory, wherein each neural network of the plurality of neural networks is trained to recognize a set of confused graphemes from the plurality of sets of confused graphemes;

receiving an input grapheme image associated with a document comprising a plurality of graphemes;

determining a set of recognition options for the input grapheme image, wherein the set of recognition options comprises a set of target characters that are similar to the input grapheme image;

selecting, by the processing device, a first neural network from the plurality of neural networks, wherein the first neural network is trained to recognize a first set of confused graphemes, and wherein the first set of confused graphemes comprises at least a portion of the set of recognition options for the input grapheme image; and

determining a grapheme class for the input grapheme image using the selected first neural network.

16. The non-transitory computer readable storage medium of claim 15, wherein analyzing the plurality of document images further comprises:

collecting statistical data associated with the recognition data for the plurality of document images;

identifying the plurality of pairs of confusing graphemes using the statistical data, wherein a first pair of confusing graphemes of the plurality of pairs comprises a first grapheme and a second grapheme that are graphically similar; and

selecting a first set of confused graphemes using the weighted graph.

17. The non-transitory computer readable storage medium of claim 16, wherein selecting the first set of confused graphemes using the weighted graph comprises:

determining a set size for each of the plurality of sets of confused graphemes;

determining a first weighted edge with a greatest weight value;

identifying a first pair of connected nodes that are connected by the first weighted edge;

selecting, for the set of confused graphemes, the graphemes associated with the first pair of connected nodes; and

responsive to determining that a number of selected graphemes has not met the set size;

identifying an additional node that is connected at least one of the nodes associated with the first pair of connected nodes, and

selecting, for the set of confused graphemes, the grapheme associated with the additional node.

18. The non-transitory computer readable storage medium of claim 15, wherein determining the set of recognition options for the input grapheme image comprises:

analyzing the input grapheme image using a simple classifier;

determining one or more target characters with characteristics that are similar to the input grapheme image; and

assigning the one or more target characters to the set of recognition options.

19. The non-transitory computer readable storage medium of claim 15, wherein selecting the first neural network comprises:

comparing the set of recognition options to the plurality of sets of confused graphemes;

determining an intersection between the set of recognition options and each of the plurality of sets of confused graphemes;

selecting a first set of the plurality of sets of confused graphemes, wherein the intersection between the set of recognition options and the first set of confused graphemes is greater than the intersection between the set of recognition options and each of a remaining one or more sets of confused graphemes; and

determining that the first neural network is associated with the first set of the plurality of sets of confused graphemes.

20. The non-transitory computer readable storage medium of claim 19, wherein determining the intersection further comprises:

ranking a set of recognition options, wherein top recognition options have a highest confidence levels; and

determining the intersection between the first set of confused graphemes and the top recognition options from the ranked set of recognition options.