Artificial neural network method and architecture

• US 5,408,588 A
• Filed: 05/18/1993
• Issued: 04/18/1995
• Est. Priority Date: 06/06/1991
• Status: Expired due to Fees

• Current Assignee: Mehmet E. Ulug
• Original Assignee: Mehmet E. Ulug
• Inventors: Ulug, Mehmet E.
• Primary Examiner(s): Downs, Robert W.
• Application Number:
  US08/063,370
• Time in Patent Office: 700 Days
• Field of Search: 395/24, 395/23, 395/27
• US Class Current: 706/25
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

An architecture and data processing method for a neural network that can approximate any mapping function between the input and output vectors without the use of hidden layers. The data processing is done at the sibling nodes (second row). It is based on the orthogonal expansion of the functions that map the input vector to the output vector. Because the nodes of the second row are simply data processing stations, they remain passive during training. As a result the system is basically a single-layer linear network with a filter at its entrance. Because of this it is free from the problems of local minima. The invention also includes a method that reduces the sum of the square of errors over all the output nodes to zero (0.000000) in fewer than ten cycles. This is done by initialization of the synaptic links with the coefficients of the orthogonal expansion. This feature makes it possible to design a computer chip which can perform the training process in real time. Similarly, the ability to train in real time allows the system to retrain itself and improve its performance while executing its normal testing functions.

Claims

1. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function, where the arguments of said terms are from said input data signal and not from the cumulative probability distribution function of said input data signal;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network.

2. The artificial neural network architecture of claim 1, further comprising an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals.

3. The artificial neural network architecture of claim 2, further comprising an output multiplexer connected between said output nodes and said means for adding.

4. The artificial neural network architecture of claim 3, wherein said artificial neural network architecture is embodied on a computer chip.

5. The artificial neural network architecture of claim 3, wherein said artificial neural network architecture is embodied on a hardware unit.

6. The artificial neural network architecture of claim 3, wherein the learning process is carried out by simulation of hardware implementation of the artificial neural network on a computer.

7. The artificial neural network architecture of claim 3, wherein said sets of arithmetic and storage units generate either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function and output signals of said multipliers set to correspond to products of output signals from each of said N arithmetic and storage units and the corresponding N coefficients of said expansion function.

8. The artificial neural network architecture of claim 7, wherein said arithmetic and storage units express said input data signals in an expanded form and thereby create a single-layer neural network having (n) (N) virtual input nodes and m said output nodes that do not use nonlinear functions and, thereby, free said neural network from the problems of local minima.

9. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, each output signal of each of the N said arithmetic and storage units applied to a first unit of each set of N said weight multipliers is independent of said input data signals and equals unity when an orthogonal polynomial expansion function is used.

10. The artificial neural network architecture of claim 9, wherein said expansion function stored in (n) (N), said arithmetic and storage units is in accordance with the following equation when an orthogonal polynomial expansion function is used:

##EQU20## where P_km (x)=orthogonal polynomial of degree m=1,2, . . . ,k.

11. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, said output signal of said weight changer is independent of said input data signals and equals (1/2⁰.5) when an orthogonal expansion function is used.

12. The artificial neural network architecture of claim 11, wherein the term of the expansion function stored in each of said (n) (N) arithmetic and storage unit is in accordance with a generalized Fourier series as formulated in the following equation when an orthogonal expansion function is used:

##EQU21## where f(x)=generalized Fourier series;

Q_k (x)=orthogonal function;

C_k =k^th coefficient.

13. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, target values associated with all of said output nodes are identical and consequent learning time of said network is reduced by assigning an initial value to a first unit of each set of N weight multipliers in accordance with the following equation and setting values of all remaining (N-1) weight multipliers to zero when using orthogonal functions;

##EQU22## where C₁ =value of first weight multiplier;

a=actual target value of output nodes;

e=percentage of target value achieved;

n=number of input nodes.

14. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing of the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signal, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, said predetermined output node target values are identical and consequent learning time of said network is reduced by assigning an initial value to a first unit of each set of N said weight multipliers in accordance with the following equation and setting values of all remaining (N-1) weight multipliers to zero when using an orthogonal polynomial expansion function;

##EQU23## where C₀ =value of first weight multiplier;

a=actual target value of output nodes;

e=percentage of target value achieved;

n=number of input nodes.

15. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of using either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signal by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, each of said output nodes are assigned the same target value and said learning is achieved by training each said output node only with a set of said input data signals associated with that particular output node and, thereby, decoupling said output nodes from each other so that when a new class of input data signals are to be identified, said network will be trained only for that class.

16. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, an output vector is used to identify each class of data signals corresponding to an input pattern, and said learning time is reduced by assigning through said weight initializer initial values to the output signal of a first unit of each weight multiplier in accordance with the following equation and setting the value of the output signal of all remaining weight multipliers to zero when using either an orthogonal expansion function or an orthogonal polynomial expansion function where c₁ and c₂ correspond to said output nodes with target values of a₁ +a₂, respectively;

##EQU24## where

space="preserve" listing-type="equation">A=a.sub.1.sup.2 +(m-1)a.sub.2.sup.2 (

     36)

space="preserve" listing-type="equation">B=2a.sub.1.sup.2 a.sub.2 +2(m-1)a.sub.2.sup.3 (

     37)

space="preserve" listing-type="equation">C=a.sub.1.sup.2 a.sub.2.sup.2 +(m-1)a.sub.2.sup.4 -ma.sub.1.sup.2 a.sub.2.sup.2 (1-e).sup.2 (

     38)

k=2⁰.5 if orthogonal functions used;

k=1 if orthogonal polynomials are used;

n=number of input terminals;

e=percentage of activation to be achieved before training;

a₁ =target value associated with coefficient c₁ ;

a₂ =target value associated with coefficient c₂.

17. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, m components of said output signals form an output vector corresponding to a set of input data signals comprising a class, said m components being assigned to m said output nodes and said network learning process is achieved by training said m output nodes with said class of input data signals.

18. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signal from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signals received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signal;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function, thereby creating a single-layer neural network having (n) (N) virtual input nodes and m said output nodes that do not use nonlinear functions and, thereby, free said neural network from the problems of local minima; and

, all output nodes are assigned an identical target value, and further wherein during training of said artificial neural network said arithmetic and storage units process all said input data signals associated with a first output node, store said processed data in memory, and start said training process for said first output node, and thereafter said arithmetic and storage units process and store input data associated with a second output node, and processed signals of all remaining (m-2) output nodes are generated and stored.

19. A neural network test apparatus comprising:

a plurality of input terminal means for receiving n input data signals corresponding to an input pattern to be identified by said network;

each said input terminal means being connected to a set of N arithmetic and storage units, said arithmetic and storage units capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signal;

said arithmetic and storage units are in turn connected to N weight multipliers said weight multipliers using synaptic link value signals generated in a previous learning process of an artificial neural network architecture, adapted to generate products of output signals from said arithmetic and storage units and said synaptic link value signals;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by generating sums of said products of output signals and without using said sums as an argument of a non-linear function;

a comparitor and error generator adapted to receive said activation values from m said output nodes and compare said activation values to predetermined target values initially ascribed to said m output nodes during training of said neural network to generate a set of m error signals;

said comparitor and error generator for comparing said error signals and selecting the output node which generates the smallest error signal, and identifying said input pattern with a class associated with said output node which generates the smallest error signal.

20. The artificial neural network architecture of claim 19, wherein said artificial neural network architecture is embodied on a computer chip.

21. The neural network test apparatus of claim 19, wherein said artificial neural network architecture is embodied on a hardware unit.

22. The neural network test apparatus of claim 19, wherein testing is carried out by simulation of hardware implementation of the neural network on a computer.

23. An artificial neural network architecture comprising:

means for receiving a plurality of input data signals representing an input pattern;

a neural network test unit connected to said means for receiving, said neural network test unit comprising;

a plurality of input terminals for receiving n input data signals corresponding to an input pattern to be identified by said network;

each said input terminals being connected to a set of N arithmetic and storage units, said arithmetic and storage units capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

said arithmetic and storage units are in turn connected to N weight multipliers said weight multipliers using synaptic link value signals generated in a previous learning process of said artificial neural network architecture, adapted to generate products of output signals from said arithmetic and storage units and said synaptic link value signals;

means for adding connected on one side to (n.N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by generating sums of said products of output signals and without using said sums as an argument of the non-linear function;

a comparitor and error generator adapted to receive said activation values from m said output nodes and subtract said activation values from predetermined target values initially ascribed to said m output nodes during training of said neural network to generate a set of m error signals;

said comparitor and error generator for comparing said error signals and selecting the output node which generates the smallest error signal, and identifying said input pattern with a class associated with said output node which generates the smallest error signal;

means for providing an output of said neural network test unit to a user based on synaptic link information obtained in a previous learning process of said artificial neural network;

means for monitoring said input data signals and the output of said neural network test unit connected between said means for providing receiving a plurality of input data signals and said neural network test unit;

a neural network learning unit connected to said means for monitoring;

memory means for storing information loaded with said synaptic link information connected to said neural network learning unit and to said neural network test unit.

24. The artificial neural network architecture of claim 23, wherein weights of said weight multipliers are variable to provide a known output activation at each of said m output nodes based upon a known input data signal at each said input terminal, said weights of said weight multipliers being adjustable by a means for changing said weights in proportion to the difference between said activation value at each said output node and a target value assigned to each said output node, which difference is considered to be minimized at said output node which corresponds to a response which said neural network learned to provide, said weights of said weight changers which gave rise to said response being stored in said memory means for storing information.

25. The artificial neural network architecture of claim 23, further comprising:

a means for transferring information corresponding to said synaptic link value signals to said weight changer to allow mapping of said input data signals from said input terminal means to said output nodes and to also, selectively, allow mapping of output node activation values from said output nodes, in reverse, to said input terminal means to determine which said input data signals create an erroneous output node activation value.

26. The neural network apparatus of claim 23, wherein said neural network test unit means identifies an input pattern by recognizing any larger than allowed error measure signal as an unknown input pattern and informs said means for monitoring by storing data in a results file to be obtained and read by said means for monitoring, and further wherein said monitor instructs said training program means of said artificial neural network to teach an (m+1)th output node said unknown input pattern without teaching all other patterns that said artificial neural network architecture has already learned, and generates a new set of said synaptic link value signals to be stored in said means for storing information so that said neural network test unit means using said synaptic link value signals in said weight changer can, thereafter, identify said unknown input pattern by generating a minimum error in said (m+1)th output node.

27. An artificial neural network architecture comprising:

a plurality of input terminals adapted to receive n input data signals corresponding to an input pattern to be learned by said network;

each said input terminal being connected to a set of N arithmetic and storage units, said arithmetic and storage units being capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

a set of N weight multipliers connected on one side to said arithmetic and storage units and on the other to a weight changer and a weight initializer, each said weight multiplier for generating a weight multiplier output signal consisting of products of output signals from each set of said N arithmetic and storage units, said weight changer and said weight initializer;

means for adding connected on one side to (n) (N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by summing N said weight multiplier output signal received from N said weight multipliers and without using said sum as an argument of a nonlinear function;

a comparitor and error generator adapted to receive m said activation values from m said output nodes and compare said activation values from predetermined target values initially ascribed to said output nodes to generate m error signals;

said weight changer adapted to receive m said error signals from said comparitor and error generator to determine how much a particular signal from said weight changer is to be changed to minimize said error signals, said weight changer thereafter sending an output signal to said weight multiplier during a learning process of the network;

an input multiplexer separate from said artificial neural network architecture connected between said N arithmetic and storage units and said input terminals;

an output multiplexer connected between said output nodes and said means for adding;

wherein said sets of arithmetic and storage units generate the first N terms of either an orthogonal expansion function or an orthogonal polynomial expansion function as a function of said input data signals by placing at each of said arithmetic and storage units a term of said expansion function, and output signals of said weight changers being set to correspond to the first N coefficients of said expansion function; and

, the number of input terminals is the same as the number of output nodes, and each output node is connected to a corresponding input terminal, further wherein the input data signal at any one of said input terminals is assigned to a correspondingly numbered input terminal, and consequent learning time of said artificial neural network architecture is reduced by equally distributing between n sets of said weight multipliers between 50% and 97% of said input data signal connected to said output node multiplied by 2⁰.5, as an initial value of said weights of the first unit of each set of N said weight multiplier associated with said output node, and setting the remaining (N-1) weights of each said N said weight multiplier to zero when using orthogonal functions as follows;

space="preserve" listing-type="equation">c.sub.i1 =(ex.sub.i 2.sup.0.5)/n (39)

e=% of the input data signal used as target value;

x_i =input data signal at the i'"'"'th input node;

c_i1 =coefficient when using orthogonal functions.

28. The artificial neural network architecture of claim 27, wherein the number of input terminals is the same as the number of output nodes, and each output node is connected to a corresponding input terminal, further wherein the input data signal at any one of said input terminals is assigned to a correspondingly numbered input terminal, and consequent learning time of said artificial neural network architecture is reduced by equally distributing between n sets of said weight multipliers between 90% and 97% of said input data signal connected to said output node multiplied by 2⁰.5, as an initial value of said weights of the first unit of each set of N said weight multiplier means associated with said output node, and setting the remaining (N-1) weights of each said N said weight multiplier means to zero when using orthogonal polynomials:

space="preserve" listing-type="equation">c.sub.i0 =ex.sub.i /n (40)

e=% of the input data signal used as target value;

x_i =input data signal at the i'"'"'th input node;

c_i0 =coefficient when using orthogonal polynomials.

29. The artificial neural network architecture of claim 28, wherein said network architecture is embodied on a computer chip.

30. The artificial neural network architecture of claim 28, wherein said network architecture is embodied on a hardware unit.

31. The artificial neural network architecture of claim 28, wherein the described functions are performed by simulation of hardware implementation of parallel distributed processing on a computer.

32. The artificial neural network architecture of claim 27, wherein said network architecture is embodied on a computer chip.

33. The artificial neural network architecture of claim 27, wherein said network architecture is embodied on a hardware unit.

34. The artificial neural network architecture of claim 27, wherein the described functions are performed by simulation of hardware implementation of parallel distributed processing on a computer.

35. A damaged symbol identification system comprised of an auto-associative neural network as defined by claim 27, and a pattern associative neural network, said pattern associative network comprising:

means for receiving a plurality of input data signals representing an input pattern;

a neural network test unit connected to said means for receiving, said neural network test unit comprising;

a plurality of input terminals for receiving n input data signals corresponding to an input pattern to be identified by said network;

each said input terminals being connected to a set of N arithmetic and storage units, said arithmetic and storage units capable of producing the first N terms of either an orthogonal polynomial expansion function or an orthogonal expansion function as a function of said input data signals;

said arithmetic and storage units are in turn connected to N weight multipliers said weight multipliers using synaptic link value signals generated in a previous learning process of said artificial neural network architecture, adapted to generate products of output signals from said arithmetic and storage units and said synaptic link value signals;

means for adding connected on one side to (n.N) said weight multipliers and on the other to m output nodes so as to generate activation values at each of said m output nodes by generating sums of said products of output signals and without using said sums as an argument of the non-linear function;

a comparitor and error generator adapted to receive said activation values from m said output nodes and subtract said activation values from predetermined target values initially ascribed to said m output nodes during training of said neural network to generate a set of m error signals;

said comparitor and error generator for comparing said error signals and selecting the output node which generates the smallest error signal, and identifying said input pattern with a class associated with said output node which generates the smallest error signal;

means for providing an output of said neural network test unit means to a user based on synaptic link information obtained in a previous learning process of said artificial neural network;

means for monitoring said input data signals and the output of said neural network test unit means connected between said means for providing receiving a plurality of input data signals and said neural network test unit means;

a neural network learning unit connected to said means for monitoring;

memory means for storing information loaded with said synaptic link information connected to said neural network learning unit and to said neural network test unit means;

wherein the auto-associative network learns in real-time the undamaged symbols so that after said learning process said system can restore missing features of a damaged symbol, and allow the pattern associative neural network to identify the damaged symbol using its restored features when at least one half of the information about each of its said missing features survived the damage.

36. The damaged symbol identification system of claim 35, wherein the appearance of a previously unknown symbol causes said auto-associative neural network to learn in real-time, without retraining previously trained output nodes, so that the damaged versions of the new symbol can be restored, and subsequently identified, by the pattern associative neural network.

37. An artificial neural network architecture having the ability to learn patterns in real time, comprising:

means for providing input data signals to said neural network corresponding to a pattern, said means for providing connected to a neural network test unit;

said means for providing input data signals and said artificial neural network test unit connected to a means for monitoring m output signals from m corresponding neural network test unit output nodes;

said means for monitoring connected to a neural network training unit;

said means for monitoring connected to a means for inputting a signal representing an actual outcome;

said neural network training unit connected to a means for storing information and to said neural network test unit;

wherein if an unknown set of signals is received from said means for providing by said monitor, said unknown set of signals is stored in said means for storing information and a first (m+1)^th output node is added to said neural network training unit and trained in said neural network training unit with said unknown set of signals without retraining any previously trained output nodes, and a second (m+1)^th output node is added to said neural network test unit, and a synaptic link value associated with said first (m+1)^th output node is incorporated into said neural network test unit in association with only said second (m+1)^th output node.

38. An artificial neural network as in claim 37, further comprising a means for providing an output in human perceptible form connected to said neural network test unit.