Analog hardware for learning neural networks
First Claim
Patent Images
1. A neural network processor capable of fully-parallel analog implementation of both single-attractor and multiple-attractor configurations with maximum speed of operation comprising:
- a) a plurality of input neurons each having an input for receiving a voltage stimulus and an output for outputting a voltage output;
b) a plurality of output neurons each having an input for receiving a voltage input and an output for outputting a voltage output;
c) a synaptic array comprising a plurality of synapses connecting said output of respective ones of said input neurons to said input of respective ones of said output neurons, each of said synapses having a weighting factor associated therewith determining a conductance value therethrough and means for changing said weighting factor including a weighting factor input;
d) a weighting factor update conductor disposed throughout said synaptic array;
e) a plurality of individually addressable switches connected between said weighting factor input of respective ones of said synapses and said weighting factor update conductor;
f) learning controller means connected to said individually addressable switches and said weighting factor update conductor for enabling selected ones of said switches to connect said weighting factor input of a corresponding one of said synapses to said weighting factor update conductor and for simultaneously applying a weighting factor update voltage value to said weighting factor update conductor; and
,wherein said learning controller means includes logic for updating each said synapse in turn by enabling a corresponding one of said individually addressable switches, making an initial change in the conductance value of said corresponding synapse, evaluating corresponding changes at outputs of said output neurons, and implementing therefrom a learning rule to determine a final change in said conductance of said corresponding synapse.
2 Assignments
0 Petitions
Accused Products
Abstract
This is a recurrent or feedforward analog neural network processor having a multi-level neuron array and a synaptic matrix for storing weighted analog values of synaptic connection strengths which is characterized by temporarily changing one connection strength at a time to determine its effect on system output relative to the desired target. That connection strength is then adjusted based on the effect, whereby the processor is taught the correct response to training examples connection by connection.
61 Citations
10 Claims
-
1. A neural network processor capable of fully-parallel analog implementation of both single-attractor and multiple-attractor configurations with maximum speed of operation comprising:
-
a) a plurality of input neurons each having an input for receiving a voltage stimulus and an output for outputting a voltage output; b) a plurality of output neurons each having an input for receiving a voltage input and an output for outputting a voltage output; c) a synaptic array comprising a plurality of synapses connecting said output of respective ones of said input neurons to said input of respective ones of said output neurons, each of said synapses having a weighting factor associated therewith determining a conductance value therethrough and means for changing said weighting factor including a weighting factor input; d) a weighting factor update conductor disposed throughout said synaptic array; e) a plurality of individually addressable switches connected between said weighting factor input of respective ones of said synapses and said weighting factor update conductor; f) learning controller means connected to said individually addressable switches and said weighting factor update conductor for enabling selected ones of said switches to connect said weighting factor input of a corresponding one of said synapses to said weighting factor update conductor and for simultaneously applying a weighting factor update voltage value to said weighting factor update conductor; and
,wherein said learning controller means includes logic for updating each said synapse in turn by enabling a corresponding one of said individually addressable switches, making an initial change in the conductance value of said corresponding synapse, evaluating corresponding changes at outputs of said output neurons, and implementing therefrom a learning rule to determine a final change in said conductance of said corresponding synapse. - View Dependent Claims (2, 3, 4)
-
-
5. A neural network processor capable of fully-parallel analog implementation of both single-attractor and multiple-attractor configurations with maximum speed of operation comprising:
-
a) a data retrieval network portion comprising, a1) a plurality of input neurons each having an input for receiving a voltage stimulus and an output for outputting a voltage output, a2) a plurality of output neurons each having an input for receiving a voltage input and an output for outputting a voltage output, and a3) a synaptic array comprising a plurality of synapses connecting said output of respective ones of said input neurons to said input of respective ones of said output neurons, each of said synapses having a weighting factor associated therewith determining a conductance value therethrough and means for changing said weighting factor including a weighting factor input; and
,b) a learning network portion comprising, b1) a weighting factor update conductor disposed throughout said synaptic array, b2) a plurality of individually addressable switches connected between said weighting factor input of respective ones of said synapses and said weighting factor update conductor, and b3) learning controller means connected to said individually addressable switches and said weighting factor update conductor for enabling selected ones of said individually addressable switches to connect said weighting factor input of a corresponding one of said synapses to said weighting factor update conductor and for simultaneously applying a weighting factor update voltage value to said weighting factor update conductor, wherein said learning controller means includes logic for updating each said synapse in turn by enabling a corresponding one of said individually addressable switches, making an initial change in the conductance value of said corresponding synapse, directly evaluating corresponding changes at outputs of said output neurons, and implementing therefrom a learning rule to determine a final change in said conductance of said corresponding synapse. - View Dependent Claims (6, 7, 8)
-
-
9. A neural network processor capable of fully-parallel analog implementation of both single-attractor and multiple-attractor configurations with maximum speed of operation comprising:
-
a) a data retrieval network portion comprising, a1) a plurality of input neurons each having an input for receiving a voltage stimulus and an output for outputting a voltage output, a2) a plurality of output neurons each having an input for receiving a voltage input and an output for outputting a voltage output, and a3) a synaptic array comprising a plurality of synapses connecting said output of respective ones of said input neurons to said input of respective ones of said output neurons, each of said synapses having a weighting factor associated therewith determining a conductance valve therethrough and means for changing said weighting factor including a weighting factor input; b) a learning network portion comprising, b1) a weighting factor update conductor disposed throughout said synaptic array, b2) a plurality of individually addressable switches connected between said weighting factor input of respective ones of said synapses and said weighting factor update conductor, and b3) learning controller means connected to said individually addressable switches and said weighting factor update conductor for enabling selected ones of said individually addressable switches to connect said weighting factor input of a corresponding one of said synapses to said weighting factor update conductor and for simultaneously applying a weighting factor update voltage value to said weighting factor update conductor, said learning controller means comprising, a plurality of learning rule processor means connected to said output of respective ones of said output neurons for determining an error reduction or increase due to a change in the output of each of said output neurons in response to voltage stimuli applied to said inputs of said input neurons and said initial change in the conductance value of said corresponding synapse, according to a learning rule implemented thereby, summing amplifier means for receiving said component of error from said plurality of learning rule processor means at an input thereof and for providing a summation thereof at an output, scaling means for receiving said summation from said summing amplifier means at an input thereof and for providing said summation scaled by a pre-established learning rate at an output thereof, update generator means for receiving a scaled summation from said scaling means at an input thereof and for generating a voltage signal required to change the conductance of a said synapse and applying said voltage signal to said weighting factor update conductor, and, digital events sequencer means having a CLOCK input thereto for causing said digital event sequencer means to repeatedly sequence through a plurality of learning steps. - View Dependent Claims (10)
-
Specification