Concurrent two-stage multi-network optical character recognition system

• US 5,835,633 A
• Filed: 11/20/1995
• Issued: 11/10/1998
• Est. Priority Date: 11/20/1995
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mao, Jianchang, Fujisaki, Tetsunosuke, Mohiuddin, Kottappuram Mohamedali
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Bella, Matthew C.
• Application Number:
  US08/561,509
• Time in Patent Office: 1,086 Days
• Field of Search: 382/227, 382/187, 382/224, 382/159, 382/155, 382/156
• US Class Current: 382/187
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches

Abstract

A multi-stage multi-network character recognition system decomposes the estimation of a posteriori probabilities into coarse-to-fine stages. Classification is then based on the estimated a posteriori probabilities. This classification process is especially suitable for the tasks that involve a large number of categories. The multi-network system is implemented in two stages: a soft pre-classifier and a bank of multiple specialized networks. The pre-classifier performs coarse evaluation of the input character, developing different probabilities that the input character falls into different predefined character groups. The bank of specialized networks, each corresponding to a single group of characters, performs fine evaluation of the input character, where each develops different probabilities that the input character represents each character in that specialized network'"'"'s respective predefined character group. A network selector is employed to increase the system'"'"'s efficiency by selectively invoking certain specialized networks selected, using a combination of prior external information and outputs of the pre-classifier. Relative to known single network or one-stage multiple network recognition systems, the invention provides improved recognition, accuracy, confidence measure, speed, and flexibility.

Claims

1. A character recognition system, comprising:

a soft pre-classifier responsive to a representation of an input character to provide a first output signal comprising a coarse vector estimating probabilities of the input character belonging to each one of multiple predefined groups of recognized target characters;

for each particular group of target characters, a specialized neural network to receive the input character and provide a second output signal comprising a fine vector estimating probabilities of the input character representing each one of multiple target characters in the particular group of target characters;

a multiplier, coupled to the soft pre-classifier and each of the specialized neural networks, to weight each specialized neural network'"'"'s second output signal in proportion to the first output signal to provide multiplier output lists, each multiplier list including a list of target characters each associated with a confidence value indicating a probability that the associated target character correctly represents the input character; and

a candidate selector, coupled to the multiplier, to compile and rank contents of the multiplier output lists and provide an aggregate list of target characters and associated confidence values.

2. The character recognition system of claim 1, the aggregate list of target characters being ordered according to their confidence values.

3. The character recognition system of claim 1, the aggregate list being limited to target characters having a number of the highest confidence values.

4. The character recognition system of claim 1, the aggregate list being limited to target characters having confidence values exceeding a predetermined threshold.

5. The character recognition system of claim 1, the soft pre-classifier comprising a feedforward neural network to perform probability calculations to identify one group of target characters that the input character most likely belongs to.

6. The character recognition system of claim 1, each specialized neural network comprising a feedforward neural network to perform probability calculations to identify a target character in a particular group of characters that the input character most likely represents.

7. The character recognition system of claim 1, each of the multiple groups of target characters including non-overlapping sets of target characters.

8. The character recognition system of claim 1, the multiple groups of target characters including partially overlapping sets of characters.

9. The character recognition system of claim 1, the multiple groups of target characters including:

a group of lower-case characters, a group of upper-case characters, a group of digits, and a group of special symbols.

10. The character recognition system of claim 1, further including a transducer coupled to the soft pre-classifier to receive the input character and provide the representation of the input character, said representation comprising a bit map.

11. The character recognition system of claim 10, the transducer comprising a digital optical scanner.

12. The character recognition system of claim 10, the transducer comprising a pen and an associated digitizing pad.

13. The character recognition system of claim 10, further including a feature extractor interposed between the transducer and the soft pre-classifier.

14. The character recognition system of claim 1, further comprising:

a transducer to receive a tangible medium bearing the input character and create a machine-readable matrix representative of the input character; and

a feature extractor, coupled between the transducer and the soft pre-classifier, responsive to the matrix to generate a feature vector containing representations of selected characteristics of the input character.

15. The character recognition system of claim 14, the transducer comprising a digitizing optical scanner.

16. The character recognition system of claim 14, the transducer comprising a pen and an associated digitizing pad.

17. The character recognition system of claim 14, the feature extractor comprising a mechanism responsive to the matrix to generate a feature vector containing representations of characteristics including contour direction features of the input character.

18. The character recognition system of claim 14, the feature extractor comprising a mechanism responsive to the matrix to generate a feature vector containing representations of characteristics including bending point features of the input character.

19. The character recognition system of claim 1, the soft pre-classifier including a first input to receive an external information signal associating the input character with one or more specialized neural networks, a second input to receive the representation of the input character, and a network selector responsive to the first output signal and the external information signal to select all specialized neural networks identified by the external information signal and all specialized neural networks whose component of the first output signal exceeds a predetermined threshold.

20. The character recognition system of claim 19, the network selector further including circuitry to deactivate all specialized neural networks not selected.

21. The character recognition system of claim 1, further including a network selector including a third input to receive the first output signal from the soft pre-classifier, a fourth input to receive an external information signal associating the input character with one or more specialized neural networks, and circuitry responsive to the first output signal and the external information signal to select all specialized neural networks identified by the external information signal and all specialized neural networks whose component of the first output signal exceeds a predetermined threshold.

22. The character recognition system of claim 21, the network selector inhibiting outputs of the selected specialized neural networks.

23. The character recognition system of claim 21, the network selector disabling outputs of the selected specialized neural networks.

24. The character recognition system of claim 1, further including a network selector including a third input to receive the coarse vector from the soft pre-classifier, the coarse vector including multiple components each estimating a probability that the input character belongs to a different one of the predefined groups of recognized target characters, a fourth input to receive external information associating the input character with one or more specialized neural networks, and circuitry programmed to select one or more specialized neural networks by performing steps comprising:

selecting all specialized neural networks identified by the external information;

normalizing the components of the coarse vector; and

selecting all specialized neural networks whose normalized component of the coarse vector exceeds a predetermined threshold.

25. The character recognition system of claim 24, the network selector further being programmed to inhibit output of the selected specialized neural networks.

26. The character recognition system of claim 1, including a programmed digital data processing device to implement at least one of the pre-classifier, specialized neural networks, multiplier, and candidate selector.

27. The character recognition system of claim 1, further including a network selector including a third input to receive the coarse vector from the soft pre-classifier, the coarse vector including multiple components each estimating a probability that the input character belongs to a different one of the predefined groups of recognized target characters, a fourth input to receive external information associating the input character with one or more specialized neural networks, and circuitry programmed to select one or more specialized neural networks by performing steps comprising:

selecting all specialized neural networks identified by the external information;

normalizing the components of the coarse vector; and

selecting all specialized neural networks whose normalized component of the coarse vector exceeds a predetermined threshold.

28. A method for character recognition, comprising the steps of:

responsive to a representation of an input character, providing a first output signal comprising a coarse vector estimating probabilities of the input character belonging to each one of multiple groups of characters;

responsive to the input character, providing multiple second output signals each corresponding to a particular group of characters and comprising a fine vector estimating probabilities of the input character representing each one of multiple characters in the particular group of characters;

weighting each second output signal in proportion to the first output signal to provide multiplier output lists each comprising an estimate of individual probabilities that the input character corresponds to each of the characters of a different specialized neural network; and

assembling and ranking contents of the multiplier output lists to provide individual ranked estimates according to a selected criteria.

29. The method of claim 28, the selected criteria comprising all individual ranked estimates.

30. The method of claim 28, the selected criteria comprising all individual ranked estimates exceeding a predetermined threshold.

31. The method of claim 28, the step of providing the first output signal comprising the steps of employing a feedforward neural network to perform probability calculations to identify a group of characters to which the input character most likely belongs.

32. The method of claim 28, the step of providing multiple second output signals comprising the steps of employing multiple feedforward neural networks each corresponding to a different group of characters, to perform probability calculations to identify a character in the corresponding group of characters that the input character most likely corresponds to.

33. The method of claim 28, each of the multiple groups including a non-overlapping set of characters.

34. The method of claim 28, at least two of the multiple groups including characters of overlapping sets.

35. The method of claim 33, the multiple groups including a first group of upper-case characters, a second group of lower-case characters, a third group of digits, and a fourth group of special symbols.

36. The method of claim 28, the step of providing the first output signal comprising the steps of receiving external information associating the input character with one or more specialized neural networks, receiving the representation of the input character, and responsive to the first output signal and the external information selecting all specialized neural networks identified by the external information and all specialized neural networks whose component of the first output signal exceeds a first predetermined threshold.

37. The method of claim 36, further comprising a step of deactivating the specialized neural networks not selected.

38. The method of claim 28, the weighting step comprising the steps of:

responsive to the first output signal, selecting second output signals having coarse estimates greater than a predetermined threshold; and

weighting each selected second output signal in proportion to the first output signal to provide multiplier output lists each comprising an estimate of individual probabilities that the input character corresponds to characters of a different selected specialized neural network.

39. The method of claim 28, the step of providing multiple second output signals comprising the steps of, responsive to the first output signal and the input character, providing multiple second output signals each corresponding to a particular group of characters having coarse estimates greater than a predetermined threshold, each second output signal comprising a fine vector estimating probabilities of the input character representing each one of multiple characters in the particular group of characters.

40. The method of claim 28, the step of providing a first output signal comprising the steps of:

employing a transducer to receive a tangible medium bearing the input character and create machine-readable matrix representative of the input character;

utilizing a feature extractor responsive to the matrix to generate a feature vector containing representations of selected characteristics of the input character; and

responsive to the feature vector, providing a first output signal comprising a coarse vector estimating probabilities of the input character belonging to each one of multiple groups of characters.

41. The method of claim 40, the employing step being performed by operating a digitizing optical scanner to optically scan the tangible medium and create a machine-readable matrix representative of the input character.

42. The method of claim 40, the selected characteristics including contour directions of the input character.

43. The method of claim 40, the selected characteristics including bending point features of the input character.