Supervised training of a neural network

• US 5,566,273 A
• Filed: 12/30/1993
• Issued: 10/15/1996
• Est. Priority Date: 12/30/1993
• Status: Expired due to Term

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Wang, Hsu-Pin, Lin, Shui-Shun, Huang, Hsin-Hao, Knapp, Gerald M.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/176,458
• Time in Patent Office: 1,020 Days
• Field of Search: 395/22, 395/23, 395/24, 395/27
• US Class Current: 706/25
• CPC Class Codes: G06F 18/23211 with adaptive number of clusters
• CPC Class Codes: G06N 3/0409 Adaptive resonance theory [ART] networks
• CPC Class Codes: Y10S 128/925 Neural network

Abstract

The present invention provides a system and method for supervised training of a neural network. A neural network architecture and training method is disclosed that is a modification of an ARTMAP architecture. The modified ARTMAP network is an efficient and robust paradigm which has the unique property of incremental supervised learning. Furthermore, the modified ARTMAP network has the capability of removing undesired knowledge that has previously been learned by the network.

Claims

1. A computer-implemented method of training and testing a modified ARTMAP neural network, wherein the modified ARTMAP neural network has an ART module that accepts an input pattern, and a Map field, connected to said ART module, that accepts a target output pattern, the computer-implemented method comprising:

(1) providing both the input pattern and the target output pattern to the modified ARTMAP neural network;

(2) updating a set of top-down weights and bottom-up weights based on a first vigilance test;

(3) propagating a predicted output from the ART module to the Map field;

(4) updating said set of top-down weights and said bottom-up weights based on a second vigilance test, wherein said second vigilance test is based on a comparison between the target output pattern and said predicted output; and

(5) removing undesired knowledge from the modified ARTMAP neural network, wherein said step of removing further comprises the steps of;

(a) presenting a sensory pattern to be unlearned to the modified ARTMAP neural network;

(b) updating bottom-up and top-down weights and Map field weights based on said first vigilance test and said second vigilance test.

2. A computer-implemented method of training and testing a modified ARTMAP neural network, wherein the modified ARTMAP neural network has an ART module that accepts an input pattern and has a first layer and a second layer, and a Map field, connected to said ART module, that accepts a target output pattern, the computer-implemented method comprising:

(1) presenting an input pattern to be learned to the modified ARTMAP neural network;

(2) determining a set of activations for the first layer;

(3) determining a set of matching scores for the second layer;

(4) activating a node with a largest matching score from the second layer, wherein said activated node is a predicted output;

(5) performing a first vigilance test based on said activation of said node, and if said first vigilance test fails then deactivating said activated node and repeat steps (2) through (5), otherwise update bottom-up and top-down weights between the first layer and the second layer, and update weights between the second layer and the Map field;

(6) propagating said matching scores from said second layer to the Map field;

(7) performing a second vigilance test to determine a level of match between said predicted output from said second layer and the target output pattern, if said second vigilance test is passed, updating said bottom-up weights, said top-down weights, and said weights between the second layer and the Map field; and

(8) removing undesired knowledge from the modified ARTMAP neural network, wherein said step of removing further comprises the steps of;

(a) presenting a sensory pattern to be unlearned to the modified ARTMAP neural network;

(b) determining a set of activations for the first layer;

(c) determining a set of matching scores for the second layer based on said set of activations;

(d) activating a node from the second layer with the largest matching score; and

(e) performing at least one vigilance test, and if said at least one vigilance test fails then deactivate said activated node and repeat steps (2) through (5), otherwise update bottom-up and top-down weights between the first layer and the second layer, and Map field weights between the second layer and the Map field.

3. The method of claim 2, wherein said input pattern is normalized before being presented to said modified ARTMAP neural network.

4. The method of claim 2, wherein step (7) further comprises the steps of sending a reset signal to the ART module if said second vigilance test fails, raising said first vigilance test to prevent further errors from occurring, and activating another node from the second layer.

5. The method of claim 4, further comprising the steps of repeating steps (5) through (7) for said activated another node.

6. The method of claim 5, further comprising the steps of iteratively repeating steps (1) through (7) for each input pattern.

7. The method of claim 6, further comprising the step of saving said bottom-up weights and said top-down weights for use during a diagnosis stage.

8. The method of claim 7, wherein said diagnosis stage includes the steps of:

(i) acquiring input data from a user;

(ii) hypothesizing a diagnosis based on said acquired input data; and

(iii) checking to determine whether said diagnosis passes the first vigilance test, if said diagnosis passes said first vigilance test than reporting the results of said diagnosis to said user, wherein said user can train said neural network with the present input pattern, if said diagnosis fails said first vigilance test then said user can either train the modified ARTMAP neural network with a known correct fault classification, or request another diagnosis.

9. The method of claim 2, further comprising the step of normalizing said input pattern before presenting said input pattern to the modified ARTMAP neural network.

10. The method of claim 2, further comprising the step of initializing said bottom-up weights and said top-down weights before initiating the training of the modified ARTMAP neural network.

11. A computer-implemented method of removing undesired knowledge from a modified ARTMAP neural network, wherein said modified ARTMAP neural network has an ART module and a Map field connected to the ART module, wherein the ART module comprises a first and a second layer, the first layer accepting an input pattern, and the Map field accepting a target output pattern, the computer-implemented method comprising:

(1) presenting a sensory pattern to be unlearned to the modified ARTMAP neural network;

(2) determining a set of activations for the first layer;

(3) determining a set of matching scores for the second layer based on said set of activations;

(4) activating a node from the second layer with a largest matching score; and

(5) performing at least one vigilance test, and if said at least one vigilance test fails then deactivate said activated node and repeat steps (2) through (5), otherwise update bottom-up and top-down weights between the first layer and the second layer, and update Map field weights between the second layer and the Map field.

12. The method of claim 11, wherein said step (5) further comprises the step of updating said bottom-up and top-down weights and said Map field weights according to the following equations:

##EQU14##

space="preserve" listing-type="equation">T.sub.j*k =0; and

w.sub.j*k =0,

wherein d is a constant between 0 and 1 and N is the dimension of the input pattern.

13. A system for removing undesired knowledge from a modified ARTMAP neural network, wherein the modified ARTMAP neural network has an ART module and a Map field connected to said ART module, wherein the ART module comprises a first and a second layer, the first layer accepting an input pattern, and the Map field accepting a target output pattern, comprising:

(1) means for presenting a sensory pattern to be unlearned to the modified ARTMAP neural network;

(2) means for determining a set of activations for the first layer;

(3) means for determining a set of matching scores for the second layer based on said set of activations;

(4) means for activating a node from the second layer with a largest matching score; and

(5) means for performing at least one vigilance test, and if said at least one vigilance test fails then deactivate said activated node and repeat steps (2) through (5), otherwise update bottom-up and top-down weights between the first layer and the second layer, and update Map field weights between the second layer and the Map field.

14. The system of claim 13, wherein said bottom-up and top-down weights and said Map field weights are updated according to the following equations:

##EQU15##

space="preserve" listing-type="equation">T.sub.j*k =0; and

w.sub.j*k =0,

wherein d is a constant between 0 and 1 and N is the dimension of the input pattern.