Neural network with semi-localized non-linear mapping of the input space

• US 5,113,483 A
• Filed: 06/15/1990
• Issued: 05/12/1992
• Est. Priority Date: 06/15/1990
• Status: Expired due to Term

• Current Assignee: Stovokor Technology LLC (Intellectual Ventures LLC)
• Original Assignee: Microelectronics and Computer Technology Corporation
• Inventors: Keeler, James D., Hartman, Eric J.
• Primary Examiner(s): MacDonald, Allen R.
• Application Number:
  US07/538,833
• Time in Patent Office: 697 Days
• Field of Search: 364/513, 395/23, 395/24
• US Class Current: 706/25
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

A neural network includes an input layer comprising a plurality of input units (24) interconnected to a hidden layer with a plurality of hidden units (26) disposed therein through an interconnection matrix (28). Each of the hidden units (26) is a single output that is connected to output units (32) in an output layer through an interconnection matrix (30). Each of the interconnections between one of the hidden units (26) to one of the output units (32) has a weight associated therewith. Each of the hidden units (26) has an activation in the i'"'"'th dimension and extending across all the other dimensions in a non-localized manner in accordance with the following equation: ##EQU1## that the network learns by the Back Propagation method to vary the output weights and the parameters of the activation function μ_hi and σ_hi.

Claims

1. A neural network for effecting non-linear mapping of an input space, comprising:

an input layer for receiving a multi-dimensional input vector x_i, for i ranging from 1 to n, n being an integer;

a hidden layer, having m hidden units, m being an integer, each of said hidden units receiving a plurality of inputs and providing a single output and having an activation function;

##EQU12## where;

h is the number of the associated one of said hidden units,

μ

_hi is the center of a_h,

f_hi is a localized function in the x_i dimension and non-localized in at least one other of said x_i dimensions,

C_hi is a constant and {P_hi } is a set of parameters for f_hi ;

a connection matrix for interconnecting the output of select ones of the input vectors x_i to the inputs of said hidden units in accordance with an interconnection scheme;

an output layer comprising M output units, M being an integer, each of said output units for providing a single output Y_i for i ranging from 1 to M, each of said output units having a predetermined transfer function g for i ranging from 1 to M; and

an output interconnection matrix for interconnecting the output of select ones of said hidden units to the inputs of select ones of said M output units in accordance with an interconnection scheme, each of said interconnections between the output of said hidden units and the associated one of said M output units having an output weight associated therewith.

2. The neural network of claim 1 wherein said activation function a_h is Gaussian according to the following equation:

##EQU13## where;

σ

_hi =the standard deviation.

3. The neural network of claim 1 wherein the constant C_hi comprises input weights associated with each interconnection between one of said input vectors x_i and the interconnected one of the inputs of said hidden units.

4. The neural network of claim 1 wherein said predetermined transfer function g_i in said m output units is a sigmoidal function.

5. The neural network of claim 1 wherein said output weights are variable.

6. The neural network of claim 5 and further comprising learning means for varying the value of μ

_hi in said activation function and said output weights in response to the input of known values for said input vectors and a predetermined target output such that said target output is realized after a finite number of said known input values are input to said input layer.

7. The neural network of claim 6 wherein said learning algorithm is a Back Propagation learning algorithm.

8. The neural network of claim 6 wherein said means for minimizing the error minimizes the error in accordance with a competitive learning algorithm.

9. The neural network of claim 1 wherein the output of each of said output units is defined by the following relationship:

##EQU14## where;

i ranges from 1 to M

h is the number of said hidden unit from 1 to m, and

w_hi is the value of said output weight on the output of the h'"'"'th one of said hidden units.

10. A method for mapping an input space in the neural network, comprising the steps of:

receiving a multi-dimensional input vector x_i, for i ranging from 1 to n, n being an integer;

providing m hidden units, m being an integer, each of the hidden units providing a single output value and having an activation function;

##EQU15## where;

h is the number of the associated one of the hidden units,

μ

_hi is the localized center of a_h in the x₁ dimension, and

f_hi is a localized function in the x_i dimension and non-localized in at least one other of the x_i dimensions;

interconnecting the output of select ones of the input vectors x_i to the inputs of the hidden units in accordance with an interconnection scheme;

providing a plurality of output units in an output layer, each of the output units for providing a single output y_i for i ranging from 1 to M, M being an integer, each of the output units having associated therewith a predetermined control function g; and

interconnecting the output of select ones of the hidden units to the inputs of predetermined ones of the M output units in accordance with the predetermined interconnection scheme, each of the interconnections between the output of the hidden units and the associated one of the M output units having an output weight associated therewith.

11. The method of claim 10 wherein the activation function of each of the hidden units is defined as a Gaussian function according to the following equation:

##EQU16## where σ

_hi =the standard deviation of a Gaussian function.

12. The method of claim 10 wherein the function f_hi is non-localized for all of the dimensions other than the i dimension.

13. The method of claim 10 and further comprising weighting the interconnection between each of the input vectors x_i and the interconnected one of the inputs of the hidden units.

14. The method of claim 10 wherein the function g in the M output units comprises a sigmoidal function.

15. The method of claim 10 and further comprising varying the value of the output weights.

16. The method of claim 15 and further comprising:

inputting a plurality of known input values to the input vectors x_i, each of the known input values having associated therewith a known target output value;

comparing the known target value with the actual output value from the output units for a given one of the known input values input to the network;

determining the error between the actual output value and the associated known target value; and

varying the value of μ

_hi in the activation function and the output weights to minimize the error.

17. The method of claim 16 wherein the step of minimizing the error comprises minimizing the error in accordance with a Back Propagation learning algorithm.