Self organizing adaptive replicate (SOAR)

• US 5,598,510 A
• Filed: 10/18/1993
• Issued: 01/28/1997
• Est. Priority Date: 10/18/1993
• Status: Expired due to Fees

• Current Assignee: Patrick F. Castelaz
• Original Assignee: LOMA Linda University Medical Center (Mission Healthcare Services LLC)
• Inventors: Castelaz, Patrick F.
• Primary Examiner(s): Downs, Robert W.
• Assistant Examiner(s): HAFIZ, TARIQ R
• Application Number:
  US08/137,569
• Time in Patent Office: 1,198 Days
• Field of Search: 395/20-27, 382/12-15, 382/155-159, 348/222, 364/724.01
• US Class Current: 706/25
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

A self-organizing adaptive replicated (SOAR) for creating a replicate of human expert behavior. The SOAR can be embedded invisibly within multiple types of systems to observe, adapt and grow to emulate a user'"'"'s interactive behavior and performance level. The system yields near equivalent responses to near equivalent stimuli in real time. The SOAR is based on a three layer perceptron type architecture which guarantees arbitrary M to N mapping of continuous valued spaces. The architecture uses a competitive, additive, and layer independent learning rule which insures excellent rapid learning. A self-organizing, adaptive algorithm permits the SOAR to adapt to the true classification space. The SOAR has applications in areas such a speech recognition, target detection, pattern recognition of multi-feature data, electro-mechanical subsystem control and resource allocation and optimization.

Claims

1. A neural network comprising:

a plurality of input nodes each receiving one or more input node inputs and producing an input node output;

a plurality of hidden nodes each receiving one or more hidden node inputs and producing a hidden node output, each of said hidden node outputs being a non-linear function of their respective hidden node inputs;

one or more output nodes, each receiving one or more output node inputs and producing an output node output;

first set of weighted connections coupling said input node outputs to said hidden nodes, the weights of said connections for each hidden node comprising an input weight vector;

second set of weighted connections coupling said hidden node outputs to said output nodes, the weights of said connections for each output node comprising a hidden node weight vector, wherein substantially every input node is connected to every hidden node, and substantially every hidden node is connected to every output node;

means for computing a set of first distance vectors that are a function of the difference between each input weight vector and an input feature vector comprising the inputs to said input nodes, said first distance vectors being fed to the hidden nodes as inputs;

means for computing a set of second distance vectors that are a function of the difference between each hidden node weight vector, and a vector comprising the outputs of said hidden nodes;

means for determining the smallest of the second distance vectors;

means for generating an output from the output node associated with the smallest second distance vector, whereby for each unique input feature vector one output node generates a response and;

means for training said network in response to a training feature vector including, means for determining if the smallest of said first distance vectors is less than a predetermined threshold and,

means for adjusting the weights for the hidden node weight vector associated with said smallest first distance vector so as to make said distance smaller if the smallest of said first distance vectors is less than the predetermined threshold.

2. The neural network of claim 1, wherein said means for computing a first distance vector and said means for computing said second distance vector both take the sum of the differences between each weight vector and each input vector, and divide the sum by the number of elements in the input vector.

3. The neural network of claim 1, wherein said hidden node output is a function of the first distance vector.

4. The neural network of claim 3, wherein said non-linear function generated by said hidden nodes takes the form w2(m)=1/1+(α

(dist-0.5)); and

where w2(m) is the output of hidden node m, α

is an arbitrary damping scaler and dist is the distance between the input feature vector and the hidden node input weight vector.

5. The neural network of claim 1, wherein said means for training further comprises means for adjusting the weights in the hidden node weight vector according to the expression:

space="preserve" listing-type="equation">v[near,j]={((1-x.sub.k)*v[near,j])+x.sub.k *wk[j])}

where;

x_k is a contribution factor, wk[j] is the input feature vector, and v[near,j] is the weight vector for the intermediate class "near".

6. The neural network of claim 5, wherein the contribution factor is calculated by the expression:

space="preserve" listing-type="equation">x.sub.k =xsc1*(1/xn[near]).

7. The neural network of claim 1 further comprising means for adding and for expanding said first set of weighted connections to include said new hidden node when said smallest first distance vector is not less than said threshold, said hidden node created having an input weight vector which is a function of said input feature vector.

8. The neural network of claim 1, wherein said weight vector created for said new hidden node is equal to said input feature vector.

9. The neural network of claim 1, wherein said means for training further comprises:

means for determining if the smallest of said second distance vectors is less than a second predetermined threshold, and if so, for adjusting the weight of the output node weight vector associated with said smallest second distance vector so as to make said distance smaller.

10. The neural network of claim 9, wherein said means for training further comprises means for adjusting the weights for the output node weight vector according to the expression:

space="preserve" listing-type="equation">h[near,m]={((1-x.sub.k)*h[near,m])+(x.sub.k *w2[m])}

where;

x_k is a contribution factor, w2[m] is the output of hidden node m, and h[near,m] is the output node weight vector.

11. The neural network of claim 9 further comprising means for adding a new output node and expanding said second set of weighted connections to include the new output node when the smallest second distance vector is not less than the second threshold, said new output node having a hidden node weight vector that is a function of said vector comprising the outputs of the hidden nodes.

12. The neural network of claim 11, wherein the hidden node weight vector the new output node has is the vector comprising the outputs of the hidden nodes.

13. The neural network of claim 1, wherein each output node vector generates an output which comprises a predetermined output response vector.

14. The neural network of claim 13, wherein said predetermined input response vector is acquired from the behavior of a separate system in response to said input vector.

15. The neural network of claim 14 further comprising interface means for receiving and storing the input and output relationships of said second system without interfering with the operation of said second system and further comprising means for using said stored input/output relationships to determine said input response vectors.

16. An adaptive replicate system comprising:

a plurality of input nodes each receiving one or more input node inputs and producing an input node output;

a plurality of hidden nodes each receiving one or more hidden node inputs and producing a hidden node output, each of said hidden node outputs being a non-linear function of their respective hidden node inputs;

one or more output nodes each receiving one or more output node inputs and producing an output node output;

first set of weighted connections coupling said input node outputs to said hidden nodes, the weights of said connections for each hidden node comprising an input weight vector;

second set of weighted connections coupling said hidden node outputs to said output nodes, the weights of said connections for each output node comprising a hidden node weight vector, wherein substantially every input node is connected to every hidden node, and substantially every hidden node is connected to every output node;

means for computing a set of first distance vectors that are a function of the difference between each input weight vector and an input feature vector comprising the inputs to said input nodes, said first distance vectors being fed to the hidden nodes as input;

means for computing a set of second distance vectors that are a function of the difference between each hidden node weight vector, and a vector comprising the outputs of said hidden nodes, wherein said means for computing first distance vector and said means for computing said second distance vector both take the sum of the differences between each weight vector and each input vector, and divide the sum by the number of elements in the input vector;

means for determining the smallest of the second distance vectors;

means for generating an output from the output node associated with the smallest second distance vector, whereby for each unique input feature vectors one output node generates a response; and

means for training said network in response to a training feature vector including;

means for determining if the smallest of said first distance vectors is less than a predetermined threshold and, if so, for adjusting the weights for the hidden node weight vector associated with said smallest first distance vector so as to make said distance smaller.

17. A method for processing an input vector in a processor having a plurality of input nodes, a plurality of hidden nodes, one or more output nodes, a first set of weighted connections coupling said input nodes to said hidden nodes, the weights of said connections for each hidden node comprising an input weight vector;

and

a second set of weighted connections coupling said hidden nodes to said output nodes, the weights of said connections for each output node comprising a hidden node weight vector, wherein substantially every input node is connected to every hidden node and substantially every hidden node is connected to every output node, the method comprising the steps of;

receiving one or more inputs and producing one or more outputs which are a function of said inputs in each of said nodes, said hidden node outputs being a non-linear function of their inputs;

computing a set of first distance vectors that are a function of the difference between each input weight vector and an input feature vector comprising the inputs to said input nodes, said first distance vectors being fed to the hidden nodes as input;

computing a set of second distance vectors that are a function of the difference between each hidden node weight vector and a vector comprising the outputs of said hidden nodes;

determining the smallest of the second distance vectors; and

generating an output from the output node associated with the smallest second distance vector, whereby for each unique input feature vector one output node generates a response.

18. The method of claim 17 further comprising the steps of:

training said network in response to a training feature vector by determining if the smallest of said first distance vectors is less than a predetermined threshold and, if so, for adjusting the weights for the hidden node weight vector associated with said smallest first distance vector so as to make said distance smaller.