Pattern recognition method using a network and system therefor

• US 5,832,108 A
• Filed: 09/09/1996
• Issued: 11/03/1998
• Est. Priority Date: 08/26/1993
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fukita, Masayuki, Toyokawa, Kazuharu, Katoh, Shin
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Do, Anh Hong
• Application Number:
  US08/707,888
• Time in Patent Office: 785 Days
• Field of Search: 382/155, 382/190, 382/209, 382/217, 382/277, 382/279, 382/156, 382/173, 382/181, 382/159, 364/602
• US Class Current: 382/159
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

An interval 0, 1! of the output of neural network is equally divided into M (M being an integer of two or more), and the numbers or frequencies of data for the correct/incorrect patterns contained in the i-th interval (i-1)/M, i/M! are μ1i and μ0i, respectively (where, i=1 . . . M). In this case, if this network provides an output contained in the i-th interval to unknown pattern data, this pattern is stored as a likelihood conversion table so that the pattern outputs likelihood P1i, which is a category, in an equation P1i=(μ1i+1)/(μ1i+μ0i+2). Then, when a value contained in the i-th interval (i-1)/M, i/M! is output from a neural network, the likelihood convertor receives it as an input and outputs P1i which is so to speak normalized likelihood.

Claims

1. A pattern recognition system using a neural network comprising:

(a) a neural network having been made to learn to output different output values in response to the input of different input patterns; and

(b) a likelihood convertor for receiving the output value of said neural network and outputting a likelihood value corresponding thereto, the likelihood value being an empirically determined probability that an input pattern which produces the neural network output value received by the likelihood convertor corresponds to a correct pattern match,

wherein the output of said neural network is between 0 and 1, wherein when the output value of the neural network is in the i-th interval (a_i-1, a_i), the likelihood convertor outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein, when the frequencies of the correct and incorrect pattern data contained in said i-th interval (a_i-1, a_i) are μ

1i and μ

0i, respectively, said likelihood value P1i is output as a value calculated by P1i=(μ

1i+1)/(μ

1i+μ

0i+2).

2. A pattern recognition system using a neural network comprising:

(a) a neural network having been made to learn to output different output values in response to the input of different input patterns; and

(b) a likelihood convertor for receiving the output value of said neural network and outputting a likelihood value corresponding thereto, the likelihood value being an empirically determined probability that an input pattern which produces the neural network output value received by the likelihood convertor corresponds to a correct pattern match,

wherein the output of said neural network is between 0 and 1, wherein when the output value of the neural network is in the i-th interval (a_i-1, a_i), the likelihood convertor outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein said (a_i-1, a_i) is an interval with equal intervals of ((i-1)/M, i/M) (where, i=1 . . . M).

3. A pattern recognition system using a neural network comprising:

(a) a plurality of neural networks having been made to learn to output different output values in response to input of different input patterns and each neural network corresponding to a different class of patterns to be recognized;

b) a plurality of likelihood convertors, each likelihood convertor being coupled to a different one of said neural networks for receiving the output value thereof and outputting a likelihood value corresponding thereto, the likelihood value outputted by each likelihood convertor being an empirically determined probability that an input pattern which produces the neural network output value received by said each likelihood convertor corresponds to a correct pattern match; and

(c) means for inputting a pattern to be recognized to the plurality of neural networks,

wherein the class of said neural network coupled to the likelihood convertor which outputs the maximum likelihood value is judged to be the recognized class,

wherein the output of said neural networks is between 0 and 1, and wherein when the output value of a neural network is in the i-th interval (a_i-1, a_i), the likelihood convertor coupled thereto outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein, when the frequencies of the correct and incorrect pattern data contained in said i-th interval (a_i-1, a_i) are μ

1i and μ

0i, respectively, said likelihood value P1i is output as a value calculated by P1i=(μ

1i+1)/(μ

1i +μ

0i+2).

4. A pattern recognition system using a neural network comprising:

(a) a plurality of neural networks having been made to learn to output different output values in response to input of different input patterns and each neural network corresponding to a different class of patterns to be recognized;

b) a plurality of likelihood convertors, each likelihood convertor being coupled to a different one of said neural networks for receiving the output value thereof and outputting a likelihood value corresponding thereto, the likelihood value outputted by each likelihood convertor being an empirically determined probability that an input pattern which produces the neural network output value received by said each likelihood convertor corresponds to a correct pattern match; and

(c) means for inputting a pattern to be recognized to the plurality of neural networks,

wherein the class of said neural network coupled to the likelihood convertor which outputs the maximum likelihood value is judged to be the recognized class,

wherein the output of said neural networks is between 0 and 1, and wherein when the output value of a neural network is in the i-th interval (a_i-1, a_i), the likelihood convertor coupled thereto outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein said (a_i-1, a_i) is an interval with equal intervals of ((i-1)/M, i/M) (where, i=1 . . . M).

5. A pattern recognition system using a neural network comprising:

(a) a pattern matching means for comparing an input pattern to be recognized with a template prepared in advance and giving a probability of a correct pattern match for each different class to be recognized;

(b) a plurality of neural networks having been made to learn to output different output values in response to input of different input patterns, each neural network corresponding to a different class of patterns to be recognized;

(c) a plurality of likelihood convertors, each separately coupled to one of said neural networks for receiving the output value thereof and outputting a likelihood value corresponding thereto, the likelihood value outputted by each likelihood converter being an empirically determined probability that an input pattern which produces the neural network output value received by said each likelihood convertor corresponds to a correct pattern match; and

(d) a means for combining the probability of a correct pattern match for each class given by said pattern matching means and said likelihood value output for each class by said likelihood convertors by Bayes theorem to give a conditional probability,

wherein the class of said neural network corresponding to a maximum given conditional probability is judged to be the recognized class,

wherein the output of said neural networks is between 0 and 1, and wherein when the output value of a neural network is in the i-th interval (a_i-1, a_i) the coupled likelihood convertor outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein, when the frequencies of the correct and incorrect pattern data contained in said i-th interval (a_i-1, a_i) are μ

1i and μ

0i, respectively, said likelihood value P1i is output as a value calculated by P1i=(μ

1i+1)/(μ

1i+μ

0i+2).

6. A pattern recognition apparatus using a neural network comprising:

(a) a pattern matching means for comparing an input pattern to be recognized with a template prepared in advance and giving a probability of a correct pattern match for each different class to be recognized;

(b) a plurality of neural networks having been made to learn to output different output values in response to input of different input patterns, each neural network corresponding to a different class of patterns to be recognized;

(c) a plurality of likelihood convertors, each separately coupled to one of said neural networks for receiving the output value thereof and outputting a likelihood value corresponding thereto, the likelihood value outputted by each likelihood converter being an empirically determined probability that an input pattern which produces the neural network output value received by said each likelihood convertor corresponds to a correct pattern match; and

(d) a means for combining the probability of a correct pattern match for each class given by said pattern matching means and said likelihood value output for each class by said likelihood convertors by Bayes theorem to give a conditional probability,

wherein the class of said neural network corresponding to a maximum given conditional probability is judged to be the recognized class,

wherein the output of said neural networks is between 0 and 1, and wherein when the output value of a neural network is in the i-th interval (a_i-1, a_i) , the coupled likelihood convertor outputs likelihood value P1i (where, i=1 . . . M) when the output interval (0,

     1) is divided into M (M being an integer of two or more) by the values of 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1, and

wherein said (a_i-1, a_i) is an interval with equal intervals of ((i-1)/M, i/M) (where, i-1 . . . M).

7. A pattern recognition method using a neural network comprising the steps of:

(a) preparing a plurality of correct patterns and incorrect patterns to be recognized by a neural network;

(b) making the neural network learn to output "1" or a value close to "1" for at least some of the correct patterns and "0" or a value close to "0" for at least some of the incorrect patterns;

(c) dividing the output range (0,

     1) of the neural network into M (M being an integer of two or more) intervals 0=a₀ <

a₁ . . . <

a_i-1 <

a_i . . . <

a_M-1 <

a_M =1;

(d) inputting to the neural network a plurality of the correct patterns and incorrect patterns in nearly the same number;

(e) counting the number or frequency of correct and incorrect patterns that produce an output value of the neural network in each i-th interval (a_i-1, a_i);

(f) storing the counted number or frequency of correct and incorrect patterns as μ

1i and μ

0i (where, i=1 . . . M), respectively; and

(g) calculating and storing a likelihood value P1i from the values of μ

1i and μ

0i for each i-th interval, thereby forming a likelihood conversion table, each P1i being a probability that an input pattern which produces an output value from the neural network in the i-th interval corresponds to a correct pattern match,

wherein the output of said neural network is enabled to be converted to a correct probability P1i in response to the output of said neural network being in the interval (a_i-1, a_i).

8. The pattern recognition system using a neural network as set forth in claim 7, wherein said likelihood value P1i is calculated as P1i=(μ

1i+1)/(μ

1i+μ

0i+2).

9. The pattern recognition method having a neural network as set forth in claim 7, wherein said (a_i-1, a_i) is an interval with equal intervals of (i-1)/M, i/M) (where, i=1 . . . M).

10. A pattern recognition system using a neural network comprising:

(a) a neural network having been made to learn to output different output values in response to the input of different input patterns; and

(b) a likelihood convertor for receiving the output value of said neural network and outputting a likelihood value corresponding thereto the likelihood value being an empirically determined probability that an input pattern which produces the neural network output value received by the likelihood convertor corresponds to a correct pattern match,

wherein the likelihood value is the number of times plus one that the neural network produced said output value from inputted learning pattern data when there was a correct pattern match divided by the total number of times plus two that the neural network produced that output value from the inputted learning pattern data.