Neural network with dynamically adaptable neurons
First Claim
1. In a neural network employing a plurality of neurons each associated with a respective conductor in a network of conductors which are selectively interconnected by a plurality of problem-defining synpases each having an adjustable weighting factor whereby each neuron receives a weighted sum of inputs from plural conductors of a previous layer of said network and produces an output to a conductor in a following layer of said network, wherein the neural network is parameterized in an iterative learning process in which problem-defining signals are input to the neurons whereby to produce error signals between actual outputs from the network and expected outputs from the network and said error signals are used to incrementally change each weighting factor, an improvement for reducing the number of iterations required from the neural network to learn how to solve a problem of interest comprising:
- in each neuron between an input thereof for receiving said weighted sum of inputs and an output therefor for outputting an output signal value, including a neural conductive element having a variable gain defining said output signal value as a function of (a) said variable gain and (b) said weighted sum of inputs and gain adjustment logic means for dynamically adjusting the variable gain of the neuron independently of the other neurons in the network during a learning process of the neural network.
0 Assignments
0 Petitions
Accused Products
Abstract
This invention is an adaptive neuron for use in neural network processors. The adaptive neuron participates in the supervised learning phase of operation on a coequal basis with the synapse matrix elements by adaptively changing its gain in a similar manner to the change of weights in the synapse io elements. In this manner, training time is decreased by as much as three orders of magnitude.
29 Citations
19 Claims
-
1. In a neural network employing a plurality of neurons each associated with a respective conductor in a network of conductors which are selectively interconnected by a plurality of problem-defining synpases each having an adjustable weighting factor whereby each neuron receives a weighted sum of inputs from plural conductors of a previous layer of said network and produces an output to a conductor in a following layer of said network, wherein the neural network is parameterized in an iterative learning process in which problem-defining signals are input to the neurons whereby to produce error signals between actual outputs from the network and expected outputs from the network and said error signals are used to incrementally change each weighting factor, an improvement for reducing the number of iterations required from the neural network to learn how to solve a problem of interest comprising:
in each neuron between an input thereof for receiving said weighted sum of inputs and an output therefor for outputting an output signal value, including a neural conductive element having a variable gain defining said output signal value as a function of (a) said variable gain and (b) said weighted sum of inputs and gain adjustment logic means for dynamically adjusting the variable gain of the neuron independently of the other neurons in the network during a learning process of the neural network. - View Dependent Claims (2, 3, 4)
-
5. A neural network comprising:
-
a plurality of signal paths; a plurality of synapses each comprising means for coupling respective pairs of said signal paths together, each of said synapses having a signal weighting factor which is adjusted during an iterative learning process of said neural network; and a plurality of neurons each comprising means for providing a respective gain in respective ones of said signal paths and for individually adjusting said respective gain independently of the other neurons in the network during an iterative learning process of said neural network. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method for training a neural network having a plurality of signal paths, a plurality of synapses each coupling a respective pair of said signal paths together, each of said synapses having a signal weighting factor and a plurality of neurons each providing a respective gain in a respective one of said signal paths, said method comprising performing an iterative learning process while:
-
adjusting the weighting factors of said synapses; and individually adjusting the respective gains of each of said neurons independently of the other neurons in the network. - View Dependent Claims (15, 16, 17, 18, 19)
-
Specification