Hierarchical constrained automatic learning neural network for character recognition

• US 5,067,164 A
• Filed: 11/30/1989
• Issued: 11/19/1991
• Est. Priority Date: 11/30/1989
• Status: Expired due to Term

• Current Assignee: NCR Corporation
• Original Assignee: AT&T, Inc.
• Inventors: Howard, Richard E., LeCun, Yann, Denker, John S., Jackel, Lawrence D.
• Primary Examiner(s): Moore, David K.
• Assistant Examiner(s): Cammarata, Michael
• Application Number:
  US07/444,455
• Time in Patent Office: 719 Days
• Field of Search: 382/14, 382/15, 364/513, 364/274.9, 364/276.6, 364/972.4
• US Class Current: 382/158
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

Highly accurate, reliable optical character recognition is afforded by a layered network having several layers of constrained feature detection wherein each layer of constrained feature detection includes a plurality of constrained feature maps and a corresponding plurality of feature reduction maps. Each feature reduction map is connected to only one constrained feature map in the same layer for undersampling that constrained feature map. Units in each constrained feature map of the first constrained feature detection layer respond as a function of a corresponding kernel and of different portions of the pixel image of the character captured in a receptive field associated with the unit. Units in each feature map of the second constrained feature detection layer respond as a function of a corresponding kernel and of different portions of an individual feature reduction map or a combination of several feature reduction maps in the first constrained feature detection layer as captured in a receptive field of the unit. The feature reduction maps of the second constrained feature detection layer are fully connected to each unit in the final character classification layer. Kernels are automatically learned by constrained back propagation during network initialization or training.

Claims

1. A massively parallel computation network for character recognition of an image map having a plurality of image units, said network comprising

first and second feature detection layer means wherein each of said feature detection layer means includes a plurality of constrained feature maps and a corresponding plurality of feature reduction maps, each constrained feature map and each feature reduction map comprising a plurality of units and a corresponding plurality of computation elements for generating values for said units in said map, each of said feature reduction maps having fewer units than each of said constrained feature maps, said computation element having a weighting kernel associated therewith and being responsive to a plurality of substantially neighboring units from at least a predetermined other map for mapping a dot product of said associated weighting kernel with said predetermined plurality of substantially neighboring units into an output value in accordance with a selected nonlinear criterion, said computation element responsive to a different plurality of substantially neighboring units than each other computation element associated with the same map, said second feature detection layer means having fewer units than said first feature detection layer means,

said constrained feature maps of said first feature detection layer means responsive to image units, each feature reduction map of said first feature detection layer means responsive to units from its corresponding constrained feature map, said constrained feature maps of said second feature detection layer means responsive to units from at least one feature reduction map in said first feature detection layer means, each feature reduction map of said second feature detection layer means responsive to units from its corresponding constrained feature map,

said network further including a character classification layer fully connected to all feature reduction maps of said second feature detection layer means for generating an indication representative of the character recognized by the network.

2. The computation network defined in claim 1 wherein the selected nonlinear criterion includes a sigmoidal function.

3. The computation network defined in claim 2 wherein said image map includes a substantially constant predetermined background surrounding an original character image.

4. A massively parallel computation network for character recognition of a character included in an image map, said network including

a first constrained feature detection layer means for extracting features from said image map,

a first feature reduction layer means connected to said first constrained feature detection layer means for undersampling features from said first constrained feature detection layer means,

a second constrained feature detection layer means for extracting features from said first feature reduction layer means,

a second feature reduction layer means for undersampling features from said second constrained feature detection layer means, and character classification layer means fully connected to said second feature reduction layer means for generating an indication representative of the character recognized by the network.

5. The computation network defined in claim 4 wherein said image map includes a substantially constant predetermined background surrounding an original character image.