Hierarchical constrained automatic learning network for character recognition

• US 5,058,179 A
• Filed: 01/31/1990
• Issued: 10/15/1991
• Est. Priority Date: 01/31/1990
• Status: Expired due to Fees

• Current Assignee: Dana Transmissions, Inc.
• Original Assignee: AT&T, Inc.
• Inventors: LeCun, Yann, Howard, Richard E., Denker, John S., Jackel, Lawrence D.
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US07/472,991
• Time in Patent Office: 622 Days
• Field of Search: 382/14, 382/10, 382/49, 364/513, 364/274.9, 364/276.6
• US Class Current: 382/158
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

Highly accurate, reliable optical character recognition is afforded by a hierarchically layered network having several layers of parallel constrained feature detection for localized feature extraction followed by several fully connected layers for dimensionality reduction. Character classification is also performed in the ultimate fully connected layer. Each layer of parallel constrained feature detection comprises a plurality of constrained feature maps and a corresponding plurality of kernels wherein a predetermined kernel is directly related to a single constrained feature map. Undersampling is performed from layer to layer.

Claims

1. A massively parallel computation network for recognition of a character included in an image map, said network including a first constrained feature detection layer for extracting features from said image map and for undersampling said image, a second constrained feature detection layer for extracting features from said first constrained feature detection layer and for undersampling said first feature detection layer, first dimensionality reduction layer substantially fully connected to and responsive to said second constrained feature detection layer, and second dimensionality reduction layer substantially fully connected to and responsive to said first dimensionality reduction layer for classifying the character recognized by the network and generating an indication representative of the character recognized by the network.

2. The computation network defined in claim 1 wherein said image map includes a substantially constant predetermined background surrounding an original character image.

3. The computation network defined in claim 1 wherein said first constrained feature detection layer includes M groups of m units arranged as independent feature maps and said second constrained feature detection layer includes N groups of n units arranged as independent feature maps, and M, N, m, and n are positive integers where M≧

N and m≧

n.

4. The computation network defined in claim 3 wherein N and M are equal.

5. The computation network as defined in claim 3 wherein said first dimensionality reduction layer comprises L groups of one unit each, said second dimensionality reduction layer comprises K groups of one unit each, where K and L are positive integers and K is greater than N and less than L.

6. The computation network defined in claim 5 wherein N and M are equal.

7. The computation network defined in claim 3 wherein substantially each unit has associated therewith a corresponding computational element for generating a value for the associated unit, each said computational element having a weighting kernel associated therewith and being responsive to a plurality of substantially neighboring units from at least a predetermined other layer for mapping a dot product of said associated weighting kernel with said predetermined plurality of substantially neighboring units into an output value in accordance with a selected nonlinear criterion, each said computation element responsive to a different plurality of substantially neighboring units than each other computation element associated with the same map, said first constrained feature detection layer responsive to image units, said second constrained feature detection layer responsive to units responsive to units from at least one feature map in said first constrained feature detection layer, each unit in said first dimensionality reduction layer responsive to substantially every unit in said second constrained feature detection layer representative of the character recognized by the network, and each unit in said second dimensionality reduction layer responsive to substantially every unit in said first dimensionality reduction layer.

8. The computation network defined in claim 7 wherein the selected nonlinear criterion includes a sigmoidal function.

9. The computation network defined in claim 7 wherein the selected nonlinear criterion includes a piecewise nonlinear function.

10. The computation network defined in claim 7 wherein N and M are equal.

11. The computation network as defined in claim 7 wherein said first dimensionality reduction layer comprises L groups of one unit each, said second dimensionality reduction layer comprises K groups of one unit each, where K and L are positive integers and K is greater than N and less than L.

12. The computation network defined in claim 11 wherein N and M are equal.

13. The computation network defined in claim 12 wherein the selected nonlinear criterion includes a sigmoidal function.

14. The computation network defined in claim 12 wherein the selected nonlinear criterion includes a piecewise nonlinear function.