Neural network with back propagation controlled through an output confidence measure
First Claim
1. Apparatus for recognizing the existence of a plurality of patterns in unknown input data comprising:
- network means, responsive to unknown input data, for producing a plurality of output values that collectively identifies one of said patterns existing in said unknown input data, said network means having a plurality of numeric weights associated therewith;
means, responsive to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one pattern, for determining changes in the value of each of said numeric weights and, in response thereto, for adjusting a value of each of said weights while said unknown input data is applied to said network means so as reduce error arising between said output values and said output target values for the unknown input data;
means, responsive to said plurality of output values, for determining a confidence measure associated therewith; and
means, responsive to said confidence measure, for generating said control signal so as to enable said determining and adjusting means during pattern recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during pattern recognition whenenver the confidence measure associated with the output values produced for said one pattern is either greater or less than said numeric range whereby said network means is trained to recognize said pattern in the event the pattern deviates from a pre-defined pattern.
1 Assignment
0 Petitions
Accused Products
Abstract
Apparatus, and an accompanying method, for a neural network, particularly one suited for use in optical character recognition (OCR) systems, which through controlling back propagation and adjustment of neural weight and bias values through an output confidence measure, smoothly, rapidly and accurately adapts its response to actual changing input data (characters). Specifically, the results of appropriate actual unknown input characters, which have been recognized with an output confidence measure that lies within a pre-defined range, are used to adaptively re-train the network during pattern recognition. By limiting the maximum value of the output confidence measure at which this re-training will occur, the network re-trains itself only when the input characters have changed by a sufficient margin from initial training data such that this re-training is likely to produce a subsequent noticeable increase in the recognition accuracy provided by the network. Output confidence is measured as a ratio between the highest and next highest values produced by output neurons in the network. By broadening the entire base of training data to include actual dynamically changing input characters, the inventive neural network provides more robust performance than which heretofore occurs in neural networks known in the art.
214 Citations
36 Claims
-
1. Apparatus for recognizing the existence of a plurality of patterns in unknown input data comprising:
-
network means, responsive to unknown input data, for producing a plurality of output values that collectively identifies one of said patterns existing in said unknown input data, said network means having a plurality of numeric weights associated therewith; means, responsive to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one pattern, for determining changes in the value of each of said numeric weights and, in response thereto, for adjusting a value of each of said weights while said unknown input data is applied to said network means so as reduce error arising between said output values and said output target values for the unknown input data; means, responsive to said plurality of output values, for determining a confidence measure associated therewith; and means, responsive to said confidence measure, for generating said control signal so as to enable said determining and adjusting means during pattern recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during pattern recognition whenenver the confidence measure associated with the output values produced for said one pattern is either greater or less than said numeric range whereby said network means is trained to recognize said pattern in the event the pattern deviates from a pre-defined pattern. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. In an optical character reader, apparatus for recognizing the existence of a plurality of bit-mapped alphanumeric characters in unknown bit-mapped input data comprising:
-
network means, responsive to unknown bit-mapped input data, for producing a plurality of output values that collectively identifies one of said bit-mapped patterns existing in said unknown bit-mapped input data, said network means having a plurality of numeric weights associated therewith; means, responsive to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one bit-mapped pattern, for determining changes in the value of each of said numeric weights and, in response thereto, for adjusting a value of each of said weights while said unknown bit-mapped input data is applied to said network means so as reduce error arising between said output values and said output target values for the unknown bit-mapped input data; means, responsive to said plurality of output values, for determining a confidence measure associated therewith; and means, responsive to said confidence measure, for generating said control signal so as to enable said determining and adjusting means during pattern recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during pattern recognition whenever the confidence measure associated with the output values produced for said one pattern is either greater or less than said numeric range whereby said network means is trained to recognize said bit-mapped pattern in the event the pattern deviates from a pre-defined bit-mapped pattern for an associated alphanumeric character. - View Dependent Claims (14, 15, 16, 17, 18)
-
-
19. A method for recognizing the existence of a plurality of patterns in unknown input data comprising the steps of:
-
producing, in response to unknown input data and through a network, a plurality of output values that collectively identifies one of said patterns existing in said unknown input data, said network having a plurality of numeric weights associated therewith; determining, in response to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one pattern, changes in the value of each of said numeric weights and, in response thereto, adjusting a value of each of said weights while said unknown input data is applied to said network so as reduce error arising between said output values and said output target values for the unknown input data; determining, in response to said plurality of output values, a confidence measure associated therewith; and generating, in response to said confidence measure, said control signal so as to enable said determining and adjusting steps during pattern recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during pattern recognition whenever the confidence measure associated with the output values produced for said one pattern is either greater or less than said numeric range whereby said network is trained to recognize said pattern in the event the pattern deviates from a pre-defined pattern. - View Dependent Claims (20, 21, 22, 23, 24, 25)
-
-
26. In an optical character reader, a method for recognizing the existence of a plurality of bit-mapped alphanumeric characters in unknown bit-mapped input data comprising:
-
producing, in response to unknown bit-mapped input data and through a network, a plurality of output values that collectively identifies one of said bit-mapped patterns existing in said unknown bit-mapped input data, said network having a plurality of numeric weights associated therewith; determining, in response to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one bit-mapped pattern, changes in the value of each of said numeric weights and, in response thereto, adjusting a value of each of said weights while said unknown bit-mapped input data is applied to said network so as reduce error arising between said output values and said output target values for the unknown bit-mapped input data; determining, in response to said plurality of output values, a confidence measure associated therewith; and generating, in response to said confidence measure, said control signal so ass to enable said determining and adjusting steps during character recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during character recognition whenever the confidence measure associated with the output values produced for said one pattern in either greater or less than said numeric range whereby said network is trained to recognize said bit-mapped pattern in the event the pattern deviates from a pre-defined bit-mapped pattern for an associated alphanumeric character. - View Dependent Claims (27, 28, 29, 30, 31)
-
-
32. In an optical character reader, a method for recognizing the existence of a plurality of bit-mapped alphanumeric characters in unknown bit-mapped input data comprising:
-
in a pattern recognition procedure; producing, in response to unknown bit-mapped input data and through a network, a plurality of output values that collectively identifies one of said bit-mapped patterns existing in said unknown bit-mapped input data, said network having a plurality of numeric weights associated therewith; determining, in response to a control signal and to said plurality of output values and pre-defined output target values associated therewith for said one bit-mapped pattern, changes in the value of each of said numeric weights and, in response thereto, adjusting a value of each of said weights while said unknown bit-mapped input data is applied to said network so as reduce error arising between said output values and said output target values for the unknown bit-mapped input data; determining, in response to said plurality of output values, a confidence measure associated therewith; and generating, in response to said confidence measure, said control signal so as to enable said determining and adjusting steps during pattern recognition to determine said changes and adjust the values of said weights when said confidence measure has a numeric value lying within a pre-defined numeric range so that network training does not occur during pattern recognition whenever the confidence measure associated with the output values produced for said one pattern is either greater or less than said numeric range whereby said network is trained to recognize said bit-mapped pattern in the event the pattern deviates from a pre-defined bit-mapped pattern for an associated alphanumeric character; and in a training procedure; applying the pre-defined bit-mapped pattern for an alphanumeric character as said unknown input data and corresponding pre-defined neural output values associated therewith as said target output values, and enabling said determining and adjusting steps during a training period to determine said changes and adjust the values of said weights to minimize error arising between said output values and said output target values whereby said network is trained to initially recognize said known pattern in said unknown bit-mapped input data; and executing said training procedure followed by said pattern recognition procedure. - View Dependent Claims (33, 34, 35, 36)
-
Specification