Neuron for use in self-learning neural network

• US 5,412,256 A
• Filed: 01/06/1994
• Issued: 05/02/1995
• Est. Priority Date: 01/06/1994
• Status: Expired due to Fees

• Current Assignee: TTI Inventions A LLC (Intellectual Ventures LLC)
• Original Assignee: Bell Communications Research, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Alspector, Joshua, Jayakumar, Anthony
• Primary Examiner(s): Westin, Edward P.
• Assistant Examiner(s): ROSEEN, RICHARD R
• Application Number:
  US08/178,428
• Time in Patent Office: 481 Days
• Field of Search: 301/201, 301/464, 395/21, 395/23
• US Class Current: 706/38
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/065 Analogue means

Abstract

A neuron for use in a self-learning neural network comprises a current input node at which a plurality of synaptic input currents are summed using Kirchoff'"'"'s current law. The summed input currents are normalized using a coarse gain current normalizer. The normalized summed inputs current is then converted to a voltage using a current to voltage converter. This voltage is then amplified by a gain controlled cascode output amplifier. Gain control inputs are provided in the output amplifier so that the neuron can be settled by the Mean Field Approximation. A noise input stage is also connected to the output amplifier so that the neuron can be settled using simulated annealing. The resulting neuron is a variable gain, bi-directional current transimpedance neuron with a controllable noise input.

Claims

1. A neuron for use in a self-learning neural network comprising,

an input node for summing a plurality of bi-directional synaptic input currents to obtain a summed bi-directional input current,

a current normalizer for normalizing said summed input current to obtain a normalized summed input current,

a current to voltage converter for converting the normalized summed input current into a bi-directional voltage representative of the normalized summed bi-directional input current, and

an output amplifier having a gain for generating bi-directional output voltage in response to said voltage representative of said normalized summed input current.

2. The neuron of claim 1 wherein said input node sums said synaptic input currents using Kirchoff'"'"'s current summation law.

3. The neuron of claim 1 wherein said current normalizer is a coarse gain current normalizer whose normalization factor is selected in response to a plurality of binary selection signals.

4. The neuron of claim 3 wherein said current normalizer comprises a plurality of current paths each of which is selected by one of said binary selection signals.

5. The neuron of claim 1 wherein said current to voltage converter comprises a cascode amplifier having an output connected back to an inverting input.

6. The neuron of claim 1 wherein said output amplifier includes a circuit for varying a gain of the output amplifier so the neuron can be settled using the Mean Field Approximation.

7. The neuron of claim 1 wherein said output amplifier includes a circuit for injecting noise into the output amplifier so the neuron can be settled using simulated annealing.

8. The neuron of claim 7 wherein said noise is modulated by a noise envelope which is slowly decaying.

9. The neuron of claim 1 wherein an output of said output amplifier is connected to an output resistor.

10. The neuron of claim 1 wherein said neuron is connected to a neuron bias circuit, said neuron bias circuit comprising a noise modulator stage for generating a signal representative of a noise envelope and a neuron bias stage for generating bias voltages for said converter and said output amplifier.

11. The neuron of claim 1 wherein said neuron comprises at least part of a VLSI chip.

12. The neuron of claim 1 wherein said output amplifier comprises,

a cascode stage comprising a pair of cascode legs,

a circuit for altering a current balance in the cascode legs in response to said voltage representative of said normalized summed input current,

a circuit for altering the current balance in the cascode legs in response to a gain control signal, and

a circuit for generating a noise current with a decaying envelope for altering the current balance in the cascode legs.

13. A neuron for use in a neural network comprising:

a current normalizer for normalizing an input current,

a current to voltage converter for converting said input current into a voltage representative of said input current, and

an output cascode mixing amplifier having a first input for receiving said voltage representative of said input current, a second input for receiving a gain anneal signal for settling said neuron using the Mean Field Approximation and a noise input for receiving a noise signal and a noise anneal input for providing a decaying envelope for the noise signal to settle the neuron using the simulated annealing.

14. The neuron of claim 13 wherein said current normalizer is a coarse current normalizer.