Neural network and method of using same

• US 5,781,701 A
• Filed: 04/17/1995
• Issued: 07/14/1998
• Est. Priority Date: 06/14/1993
• Status: Expired due to Fees

• Current Assignee: Freescale Semiconductor, Inc. (NXP Semiconductors NV)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Wang, Shay-Ping Thomas
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/422,469
• Time in Patent Office: 1,184 Days
• Field of Search: 395/24, 395/21, 395/23
• US Class Current: 706/27
• CPC Class Codes: G06F 2207/5523 Calculates a power, e.g. the square, of a number or a function, e.g. polynomials
• CPC Class Codes: G06F 7/552 Powers or roots , e.g. Pythagorean sums
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

A method of operating a neural network and a neural network, which is implemented either in hardware or software, is constructed of neurons or neuron circuits each having only one significant processing element in the form of a multiplier. The neural network utilizes a training algorithm which does not require repetitive training and which yields a global minimum to each given set of input vectors.

Claims

1. A method of operating a neural network having a plurality of hidden layer neurons, said method comprising the following steps:

(a) distributing network inputs to said hidden layer neurons without performing a non-linear transformation on said network inputs;

(b) said hidden layer neurons applying a plurality of gating functions to said neural network inputs to produce a plurality of gated inputs;

(c) said hidden layer neurons multiplying said gated inputs together to generate a plurality of products;

(d) said hidden layer neurons multiplying said plurality of products by a weight value to generate a plurality of non-binary neuron outputs without using a non-linear sigmoid function; and

(e) summing ones of said non-binary neuron outputs of a first group of said plurality of hidden layer neurons to generate a first network output; and

(f) summing ones of said non-binary neuron outputs of a second group of said plurality of hidden layer neurons to generate a second network output.

2. The method of claim 1 wherein in steps (e) and (f) at least some of said neurons are in both said first and second groups of said plurality of hidden layer neurons.

3. The method of claim 1 wherein the operation of said neural network is based upon a polynomial expansion.

4. The method of claim 3 wherein said polynomial expansion has the form:

##EQU4## wherein y represents an output selected from said first and second network output signals;

wherein w_i-1 represents the weight value for the ith neuron;

wherein x₁, x₂, . . . , x_n represent a plurality of network inputs;

wherein g_1i, . . . , g_ni represent ones of said gating functions for the ith neuron which are applied to said plurality of network inputs; and

wherein i, m and n are integers.

5. The method of claim 4 wherein each x is represented by the function x=f_i (z_j), wherein z_j is any arbitrary variable, and wherein the indices i and j are any positive integers.

6. The method of claim 4 wherein the operation of said neural network is based upon a truncated version of said polynomial expansion.

7. The method of claim 3 wherein said polynomial expansion has the form:

##EQU5## wherein y represents an output selected from said first and second network output;

wherein w_i-1 represents the weight value for the ith neuron;

wherein x_i, x_j, x_k represent a plurality of network input signals;

wherein n is a positive integer; and

wherein f₁(i) through f6.sub.(i,j,k) are index-generating functions.

8. The method of claim 3 wherein said polynomial expansion has the form:

##EQU6## wherein y represents an output selected from said first and second network outputs;

wherein w₀, w₁, . . . , w_N-1 represent weight values, N being a positive integer representing the Nth neuron, and the term w_i in general representing the weight value associated with the i+1 neuron;

wherein x₁, x₂, . . . , x_n represent a plurality of network inputs, n being a positive integer; and

wherein g_1N, . . . , g_nN represent ones of said gating functions for the Nth neuron which are applied to said plurality of network inputs.

9. The method of claim 1 wherein the operation of said neural network is based upon an orthogonal function.

10. The method of claim 9 wherein said orthogonal function is a sine, cosine, exponential, logarithmic, Fourier transformation, Legendre polynomial, or a non-linear function such as a Volterra function or a radial basis function.

11. The method of claim 1 wherein the operation of said neural network is based upon a combination of polynomial expansion and orthogonal functions.

12. The method of claim 1 wherein said neural network is implemented using integrated circuit.

13. The method of claim 1 wherein said neural network is implemented using a computer program.

14. An apparatus, comprising:

a plurality of circuits for applying a plurality of gating functions to an input signal to produce a plurality of gated inputs, wherein at least one of the gated inputs represents the input signal raised to a power other than zero or one, the circuits multiplying the gated inputs together to generate a plurality of products and multiplying the products by a plurality of weight values to generate a plurality of non-binary output signals, wherein the circuits generate the non-binary output signals without using a non-linear sigmoid function; and

a first summing circuit for summing ones of the non-binary neuron outputs from a first group of the circuits to generate a first output signal; and

a second summing circuit for summing ones of the non-binary neuron outputs from a second group of the circuits to generate a second output signal.