Neural network and method of using same

• US 5,720,002 A
• Filed: 04/17/1995
• Issued: 02/17/1998
• Est. Priority Date: 06/14/1993
• Status: Expired due to Fees

• Current Assignee: Freescale Semiconductor, Inc. (NXP Semiconductors NV)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Wang, Shay-Ping Thomas
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/422,478
• Time in Patent Office: 1,037 Days
• Field of Search: 395/21, 395/24, 395/23
• US Class Current: 706/25
• CPC Class Codes: G06F 2207/5523 Calculates a power, e.g. the square, of a number or a function, e.g. polynomials
• CPC Class Codes: G06F 7/552 Powers or roots , e.g. Pythagorean sums
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

A neural network, which can be implemented either in hardware or software, is constructed of neurons or neuron circuits each having only one significant processing element in the form of a multiplier. The number of training examples is compared to the number of neurons in the neural network to effectuate training. The neural network utilizes a training algorithm which does not require repetitive training and which yields a global minimum to each given set of input vectors.

Claims

1. A method for training a neural network comprising a plurality of neurons, said method for calculating weight values, said method comprising the following steps:

(a) providing a plurality of training examples;

(b) comparing the number of said training examples with the number of neurons in said neural network;

(c) providing at least two techniques for calculating said weight values, wherein

(i) if the number of neurons is equal to the number of training examples, using a matrix-inversion technique to solve for the value of each weight;

(ii) if the number of neurons is not equal to the number of training examples, using a least-squares estimation technique to solve for the value of each weight.

2. The method of training a neural network recited in claim 1 wherein said method produces a global minimum.

3. The method of training a neural network recited in claim 1 wherein an operation of said neural network is based upon a polynomial expansion.

4. The method of training a neural network recited in claim 3 wherein said polynomial expansion has the form:

space="preserve" listing-type="equation">y=w.sub.i-1 x.sub.1.sup.g 1i x.sub.2.sup.g 2i . . . x.sub.n.sup.g ni i=1

wherein y represents the output of the neural network;

wherein w_i-1 represents the weight value for the ith neuron;

wherein x₁, x₂, . . . , x_n represent inputs to said neural network;

wherein g_1i, . . . , g_ni represent gating functions for the ith neuron which are applied to said inputs; and

wherein m and n are positive integers.

5. The method of training a neural network recited in claim 4 wherein each x is represented by the function x=f_i (zj), wherein zj is any arbitrary variable, and wherein the indices i and j are any positive integers.

6. The method of training a neural network recited in claim 4 wherein an operation of said neural network is based upon a truncated version of said polynomial expansion.

7. The method of training a neural network recited in claim 3 wherein said polynomial expansion has the form:

##EQU4## wherein y represents the output of the neural network;

wherein w_i-1 represents the weight value for the ith neuron;

wherein x_i, x_j, x_k represent inputs to said neural network;

wherein i, i, k, and n are integers; and

wherein f₁(i) through f6.sub.(i,j,k) are index-generating functions.

8. The method of training a neural network recited in claim 3 wherein said polynomial expansion has the form:

##EQU5## wherein y represents the output of the neural network;

wherein w₀, w₁, . . . , w_n-1 represent weight values, N being a positive integer representing the Nth neuron, and the term w_i in general representing the weight value associated with the i+1 neuron;

wherein x₁, x₂, . . . , x_n represent inputs to said neural network, n being a positive integer; and

wherein g_1N, . . . , g_nN represent gating functions for the Nth neuron which are applied to said inputs.

9. The method of training a neural network recited in claim 1 wherein an operation of said neural network is based upon an orthogonal function.

10. The method of training a neural network recited in claim 9 wherein said orthogonal function is a sine, cosine, exponential, logarithmic, Fourier transformation, Legendre polynomial, or a non-linear function such as a Volterra function or a radial basis function.

11. The method of training a neural network recited in claim 1 wherein an operation of said neural network is based upon a combination of polynomial expansion and orthogonal functions.

12. The method of training a neural network recited in claim 1 wherein every example is used only once by said method.

13. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes a least-squares error technique.

14. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes an extended least-squares technique.

15. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes a Kalman filter.

16. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes a maximum-likelihood algorithm.

17. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes a pseudo-inverse algorithm.

18. The method of training a neural network recited in claim 1 wherein said least-squares estimation technique utilizes a Bayesian estimation.

19. The method of training a neural network recited in claim 1 wherein said neural network is contained on at least one integrated circuit.

20. The method of training a neural network recited in claim 1 wherein said neural network is contained in a computer program.