Self-organizing neural network for pattern classification

• US 5,479,575 A
• Filed: 10/31/1994
• Issued: 12/26/1995
• Est. Priority Date: 02/12/1991
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Yoda, Fumio
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/332,471
• Time in Patent Office: 421 Days
• Field of Search: 395/23, 395/24, 395/22
• US Class Current: 706/20
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

A neural network includes a plurality of input nodes for receiving the respective elements of the input vector. A copy of all of the elements of the input vector is sent to the next level of nodes in the neural network denoted as intermediate nodes. The intermediate nodes each encode a separate template pattern. They compare the actual input pattern with the template and generate a signal indicative of the difference between the input pattern and the template pattern. Each of the templates encoded in the intermediate nodes has a class associated with it. The difference calculated by the intermediate nodes is passed to an output node for each of the intermediate nodes at a given class. The output node then selects the minimum difference amongst the values sent from the intermediate nodes. This lowest difference for the class represented by the output node is then forwarded to a selector. The selector receives such values from each of the output nodes of all of the classes and then selects that to output value which is a minimum difference. The selector in turn, generates a signal indicative of the class of the intermediate node that sent the smallest difference value.

Claims

1. A self-organizing neural network for classifying an input pattern signal having a plurality of elements, which comprises:

a) input buffers, each having an input for receiving an element of the input pattern signal, and an output which provides a copy of the element of the input pattern signal;

b) intermediate nodes, each having an input connected to an output of each input buffer via first signal lines, an output through which an intermediate output value is provided, and means for storing weighting factors;

c) output nodes, each having inputs connected to an output of each intermediate node of a class via second signal lines, each output node determining a minimum value among output values from the intermediate nodes of the class, and each having an output through which a class signal is provided as an indication of an intermediate node which provided said minimum value, and further as an indication of the minimum value; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which provided the minimum value output, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern signal, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern signal, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is the same as the correct class, to generate a learning signal so that weighting factors of said intermediate node are modified based on said learning signal if a difference between the output value of the output node of the correct class and the output value for the output node which has a second smallest output value of all of the output nodes is below a predetermined threshold.

2. The self-organizing neural network of claim 1, wherein each of said intermediate nodes comprises:

(a) element comparators including the means for storing weighting factors and having a number of weighting factors corresponding to the number of elements in the input pattern signal, said element comparators each squaring a difference between each element of the input pattern signal and a corresponding weighting factor to provide a comparator output value;

(b) an adder for summing comparator output values from said element comparators to provide a summation result;

(c) a computer for computing a square root of said summation result and providing the square root as the intermediate output value; and

said element comparators having means responsive to said learning signal from said self-organizing selector for modifying said weighting factors on the basis of the learning signal.

3. The self-organizing neural network of claim 1, wherein said self-organizing selector operates in alternatively

a recognition mode wherein the self-organizing selector determines a minimum value among said output values and outputs as the response signal a class name of said output node which gives said minimum output value among said output nodes;

and

a learning mode wherein said weighting factors of said intermediate nodes are modified based on a classification result accounting for said class signals from said output nodes and a teaching signal from an external component.

4. The self-organizing neural network of claim 3, wherein said self-organizing selector comprises a comparison means wherein if said response signal indicates a class equal to the class indicated by the teaching signal when said selector is in said learning mode, said first comparison means detects an identification number of an intermediate node Ui and an output value of Oi of said intermediate node which give a second smallest value among class signals output from said output nodes, and determines whether said output Oi and an output Oj of an intermediate node Uj of the class indicated by the teaching signal satisfy the following condition:

space="preserve" listing-type="equation">Oi-Oj≦

th2

wherein th2 is a predetermined second threshold constant, and, if said condition is satisfied, said first comparison means modifies the weighting factors Wi1, Wi2, . . . , WiN of said intermediate node Ui and the weighting factors Wj1, Wj2, . . . , WjN of said intermediate node Uj according to the following expression;

space="preserve" listing-type="equation">Weight of Ui;

Wik=Wik-α

(Sk-Wik) for k=1, . . . N

space="preserve" listing-type="equation">Weight of Uj;

Wjk=Wjk+α

(Sk-Wjk) for k=1, . . . N

wherein α

is a small positive constant.

5. The self-organizing neural network of claim 3 wherein said self-organizing selector comprises a comparison means, wherein if said response signal indicates a class different from the class indicated by said teaching signal when said selector is in said learning mode and if said second comparison means determines that a class indicated by said teaching signal has not been created in said neural network, new intermediate node and an output node are created by the comparison means having a class identical to that indicated by the teaching signal.

6. The self-organizing pattern classification system of claim 5, wherein elements of said input pattern signal are set as weighting factors in said new intermediate node.

7. The self-organizing neural network of claim 3 wherein said self-organizing selector comprises a comparison means, wherein if said response signal indicates a class different from the class indicated by the teaching signal and an output node and intermediate nodes of the class of the teaching signal already exist, said comparison means determines whether an output Oj of an intermediate node Uj having a class identical to that indicated by the teaching signal satisfies the following condition:

space="preserve" listing-type="equation">Oj≧

th1

wherein th1 is a predetermined threshold value, and creates a new intermediate node of the class that is identical to that indicated by the teaching signal if said above condition is satisfied.

8. The self-organizing neural network of claim 7, wherein elements of said input pattern signal are set as weighting factors in said new intermediate nodes.

9. The self-organizing neural network of claim 1, wherein if the class indicated by the response signal is different from the class indicated by the teaching signal and the self-organizing selector is in said learning mode, said self-organizing selector detects an intermediate node Ui and an intermediate output Oi based on a class signal of an output node which gives a minimum value among class signals sent from output nodes and determines whether said intermediate output Oi and an intermediate output Oj of an intermediate node Uj of a class given by a class signal from an output node of a correct class satisfy the following condition:

space="preserve" listing-type="equation">Oj-Oi≦

th2

wherein th2 is a predetermined threshold constant, and modifies the weighting factors Wi1, Wi2, . . . WiN of said intermediate node Ui and the weighting factors Wji, Wj2, . . . , WjN of said intermediate factor Uj according to the following expression;

space="preserve" listing-type="equation">Weight of Ui;

Wik=Wik-α

(Sk-Wik) for k=1, . . . N

space="preserve" listing-type="equation">Weight of Uj;

Wjk=Wjk+α

(Sk-Wjk) for k=1, . . . N

wherein α

is a positive constant.

10. The self-organizing neural network of claim 1, wherein if the class indicated by the response signal is different from the class indicated by the teaching signal, said self-organizing selector detects an intermediate node Ui and an intermediate output Oi from a class signal of an output node which gives a minimum value among intermediate outputs representing respective classes, and determines whether said output value Oi and an output value Oj of an intermediate node Uj of a class given by a class signal from an output node of a class identical to the class indicated by the teaching signal satisfy the following conditions:

space="preserve" listing-type="equation">Oj<

th2 and

space="preserve" listing-type="equation">Oj-Oi>

th2

and modifies weighting factors Wj1, Wj2, . . . , WjN of said intermediate node Uj according to the following expression;

space="preserve" listing-type="equation">Weight of Uj;

Wjk=Wjk+α

(Sk-Wjk) for k=1, . . . N

wherein α

is a positive constant.

11. A self-organizing neural network system for classifying an input pattern, comprising:

a) input buffers, each having an input for receiving an element of an input pattern signal, and an output which provides a copy of said input pattern signal;

b) intermediate nodes, each having an input connected to an output of each input buffer via first signal lines, and an output through which an output value is provided;

c) output nodes, each having inputs connected to an output of each intermediate node of a class via second signal lines, each output node determining a minimum value among output values from the intermediate nodes of the class, and each having an output through which a class signal is provided as an indication of an intermediate node which provided said minimum value, and further as an indication of the minimum value; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which provided the minimum value output, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern signal, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is not the same as the correct class, to generate a learning signal so that an intermediate node and an output node are created for the correct class based on the input pattern if an output node for the correct class does not exist.

12. The self-organizing neural network system of claim 11 wherein when the correct class is not selected for the input pattern, the selector determines whether an intermediate node the correct class has weighting factors sufficiently close to the input pattern to permit correct classification and when said weighting factors are not sufficiently close to the input pattern to permit correct classification, the selector creates a new intermediate node connected to the output node of the correct class having weighting factors identical to the input pattern.

13. The self-organizing neural network of claim 12, wherein when the correct class is not selected for the input pattern and the correct class has a template sufficient to permit correct classification, the selector determines a reason for incorrect classification and, responsive to a failure to determine the reason for incorrect classification, adjusts the weighting factors of only the intermediate node of the correct class which are most similar to the input pattern.

14. The self-organizing neural network of claim 11, wherein when an intermediate node of the correct class exists, the selector determines a reason for incorrect classification and responsive to a failure to determine the reason for incorrect classification and adjusts the weighting factors of only the intermediate node the correct class which are most similar to the input pattern.

15. A self-organizing neural network system for classifying an input pattern, comprising:

a) input buffers, each having an input for receiving an element of an input pattern signal, and an output which provides a copy of said input pattern signal;

b) intermediate nodes, each having an input connected to an output of each input buffer via first signal lines, and an output through which an output value is provided;

c) output nodes, each having inputs connected to an output of each intermediate node of a class via second signal lines, each output node determining a minimum value among output values from the intermediate nodes of the class, and each having an output through which a class signal is provided as an indication of the intermediate node which provided said minimum-value, and further as an indication of the minimum value; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which has the minimum value output, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern signal, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is not the same as the correct class and a difference between each template of the correct class and the input pattern signal is greater than a predetermined threshold, to generate a learning signal so that an intermediate node connected to the output node for the correct class is created having weighting factors based on the input pattern signal.

16. The self-organizing neural network of claim 15, wherein when the correct class does not have a template sufficiently close to the input pattern to permit correct classification, the selector determines a reason for incorrect classification and, responsive to a failure to determine the reason for correct classification, adjusts the weighting factors of the intermediate node of the correct class which is most similar to the input pattern.

17. A self-organizing neural network system for classifying an input pattern, comprising:

a) input buffers, each having an input for receiving an element of an input pattern signal, and an output which provides a copy of the element of the input pattern signal;

b) intermediate nodes, each having an input connected to an output of said input buffers via first signal lines, and an output through which an output value is provided;

c) output nodes, each having inputs connected to an output of each intermediate node of a class via second signal lines, each output node determining a minimum value among output values from the intermediate nodes of the class, and each having an output through which a class signal is provided as an indication of an intermediate node which provided said minimum value, and further as an indication of the minimum value; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which provided the minimum value output, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern signal, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is the same as the correct class, to generate a learning signal so that weighting factors are adjusted so as to ensure stable classification when the correct class has not been stably classified, and so that weighting factors are adjusted according to a determined reason for any incorrect classification to improve a likelihood of correct classification when the determined reason exists, and further so that the weighting factors of only the intermediate node of the correct class which is most similar to the input pattern are adjusted when a reason for incorrect classification cannot be determined.

18. A self-organizing neural network, for classifying an input pattern represented by an input pattern signal, which comprises:

a) input buffers, each having an input for receiving a scalar element of said input pattern signal, and an output which provides a copy of said scalar element;

b) intermediate nodes, each having an input connected to the outputs said input buffers via first signal lines, means for storing weighting factors indicative of a template pattern, and an output which provides an intermediate output value indicative of a measure of similarity of the input pattern to the template pattern;

c) output nodes, each defining a class and having inputs connected to the output of each intermediate node of the class via second signal lines, each output node determining, from the intermediate output values from the intermediate nodes of the class, which intermediate node has a template pattern most similar to the input pattern, and each output node having an output through which a class signal is provided as an indication of the intermediate node of the class which has a template pattern most similar to the input pattern and the indication of the intermediate output value of said intermediate node; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which has a template pattern most similar to the input pattern, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is the same as the correct class, to generate a learning signal so that weighting factors of said intermediate node are modified based on said learning signal if a difference between the intermediate output value indicated by the class signal of the output node of the correct class and the intermediate output value indicated by the class signal for the output node connected to an intermediate node with a template pattern which is second most similar to the input pattern is below a predetermined threshold.

19. A self-organizing neural network, for classifying an input pattern represented by an input pattern signal, which comprises:

a) input buffers, each having an input for receiving a scalar element of said input pattern signal, and an output which provides a copy of said scalar element;

b) intermediate nodes, each having an input connected to the outputs said input buffers via first signal lines, means for storing weighting factors indicative of a template pattern, and an output which provides an intermediate output value indicative of a measure of similarity of the input pattern to the template pattern;

c) output nodes, each defining a class and having inputs connected to the output of each intermediate node of the class via second signal lines, each output node determining, from the intermediate output values from the intermediate nodes of the class, which intermediate node has a template pattern most similar to the input pattern, and each output node having an output through which a class signal is provided as an indication of the intermediate node of the class which has a template pattern most similar to the input pattern and the indication of the intermediate output value of said intermediate node; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which has a template pattern most similar to the input pattern, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is not the same as the correct class, to generate a learning signal so that an intermediate node and an output node are created for the correct class based on the input pattern if an output node for the correct class does not exist.

20. A self-organizing neural network for classifying an input pattern represented by an input pattern signal, which comprises:

a) input buffers, each having an input for receiving a scalar element of said input pattern signal, and an output which provides a copy of said scalar element;

b) intermediate nodes, each having an input connected to the outputs said input buffers via first signal lines, means for storing weighting factors indicative of a template pattern, and an output which provides an intermediate output value indicative of a measure of similarity of the input pattern to the template pattern;

c) output nodes, each defining a class and having inputs connected to the output of each intermediate node of the class via second signal lines, each output node determining, from the intermediate output values from the intermediate nodes of the class, which intermediate node has a template pattern most similar to the input pattern, and each output node having an output through which a class signal is provided as an indication of the intermediate node of the class which has a template pattern most similar to the input pattern and the indication of the intermediate output value of said intermediate node; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which has a template pattern most similar to the input pattern, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and, when the selected class is not the same as the correct class and a difference between each template pattern of the correct class and the input pattern is greater than predetermined threshold, to generate a learning signal so that an intermediate node connected to the output node for the correct class is created having weighting factors based on the input pattern.

21. A self-organizing neural network for classifying an input pattern represented by an input pattern signal, which comprises:

a) input buffers, each having an input for receiving a scalar element of said input pattern signal, and an output which provides a copy of said scalar element;

b) intermediate nodes, each having an input connected to the outputs said input buffers vie first signal lines, means for storing weighting factors indicative of a template pattern, and an output which provides an intermediate output value indicative of a measure of similarity of the input pattern to the template pattern;

c) output nodes, each defining a class and having inputs connected to the output of each intermediate node of the class via second signal lines, each output node determining, from the intermediate output values from the intermediate nodes of the class, which intermediate node has a template pattern most similar to the input pattern, and each output node having an output through which a class signal is provided as an indication of the intermediate node of the class which has a template pattern most similar to the input pattern and the indication of the intermediate output value of said intermediate node; and

d) a self-organizing selector having first inputs connected to the outputs of the output nodes, a first output through which a response signal is provided to an external component, the response signal indicating the class of the intermediate node which has a template pattern most similar to the input pattern, a second output through which a learning signal is provided to said intermediate nodes via third signal lines, and a second input adapted to receive a teaching signal from an external component, the teaching signal indicating a correct class for said input pattern, said self-organizing selector being responsive to said inputs so as to select a class for the input pattern, to compare the selected class to the correct class indicated by the teaching signal and to generate a learning signal so that weighting factors of said intermediate nodes are adjusted so as to ensure stable classification when the correct class has not been stably classified, and so that weighting factors of said intermediate nodes are adjusted according to a determined reason for any incorrect classification to improve a likelihood of correct classification, and so that the weighting factors of an intermediate node of only the correct class are adjusted when a reason for incorrect classification cannot be determined.