Matrix-structured neural network with learning circuitry

• US 5,083,285 A
• Filed: 10/11/1989
• Issued: 01/21/1992
• Est. Priority Date: 10/11/1988
• Status: Expired due to Term

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Shima, Takeshi, Kamatani, Yukio
• Primary Examiner(s): NOT, DEFINED
• Assistant Examiner(s): NOT, DEFINED
• Application Number:
  US07/419,768
• Time in Patent Office: 832 Days
• Field of Search: 364/513, 364/715, 364/807, 364/602, 307/201, 307/494
• US Class Current: 706/33
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

A multi-layer perceptron circuit device using integrated configuration which is capable of incorporating self-learning function and which is easily extendable. The device includes: at least one synapse blocks containing: a plurality of synapses for performing weight calculation on input signals to obtain output signals, which are arranged in planar array defined by a first and a second directions; input signal lines for transmitting the input signals to the synapses, arranged along the first direction; and output signal lines for transmitting the output signal from the synapses, arranged along the second direction not identical to the first direction; at least one input neuron blocks containing a plurality of neurons to be connected with the input signal lines; and at least one output neuron blocks containing a plurality of neurons to be connected with the output signal lines.

Claims

1. A multi-layer perceptron circuit device with a learning function, comprising:

(a) at least one tilting synapse block including

(i) a plurality of synapses for performing a weight calculation on synapse input signals to obtain synapse output signals;

(ii) synapse input signal lines for transmitting the synapse input signals to the synapses;

(iii) synapse output signal lines for transmitting the synapse output signals from the synapses;

(iv) synapse error signal input lines for transmitting synapse error input signals to the synapses;

(v) learning means for updating values of weights utilized in weight calculations by a product of synapse error input signals and the synapse input signals;

(vi) synapse error signal output lines for transmitting synapse error output signals from the synapses; and

(vii) means for calculating the synapse error output signals as a product of the synapse error input signals and the weights;

(b) at least one tiling input neuron block containing a plurality of neurons, each neuron of the at least one tiling input neuron block being connected with one of the synapse input signal lines by its signal output, inputting one of a plurality of perceptron input signals by its signal input, and supplying one of the synapse input signals to one of the synapse input signal lines, each neuron of the tiling input neuron blocks being connected to one of the synapse error signal output lines by its error signal input, receiving one of the synapse error output signals from one of the synapse error signal output lines, and outputting one of a plurality of perceptron error signals by its error signal output, wherein each of the neurons of the tiling input neuron blocks further includes

(i) differential function converter means for converting one of the perceptron input signals into a converted signal using a nonlinear function, the differential function converter means including

first and second transistors, each having a first, second, and third terminal, the first terminals connected to a first voltage source, and their second terminals connected together to form a first output terminal, the first transistor receiving an externally supplied first input signal representing a function at its third terminal and the second transistor receiving an externally supplied second input signal representing a complement of the function such that an output signal at the first output terminal represents a derivative of the function;

load means connected between the first output terminal and a second voltage source for loading the first output terminal;

reference voltage generator means connected between the first voltage source and the second voltage source for generating a DC offset voltage; and

a second output terminal connected to the reference voltage generator means; and

(ii) means for obtaining one of the perceptron error signals from one of the synapse error output signals transmitted through one of the synapse error signal output lines and the converted signal obtained by the differential function converter means; and

(c) at least one tiling output neuron block containing a plurality of neurons, each neuron of the tiling output neuron blocks being connected with one of the synapse output signal lines by its signal input, and receiving one of the synapse output signals from one of the synapse output signal lines, and outputting a perceptron output signal by its signal output, each neuron of the tiling output neuron blocks being connected to one of the synapse error signal input lines by its error signal output, inputting an externally supplied teacher signal by its error signal input, and supplying one of the synapse error input signals to one of the synapse error signal input lines.

2. The multi-layer perceptron circuit device of claim 1, wherein each of the synapses comprises means for carrying out weight calculations on the synapse input signals transmitted through the synapse input signal lines to obtain the synapse output signals outputted through the synapse output signal lines including multiplier means for multiplying one of the synapse input signals transmitted through one of the synapse input signal lines by a multiplication factor;

and

voltage source means, connected to the multiplier means, for providing a voltage corresponding to the multiplication factor to multiplier means.

3. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse input signal lines in each tiling synapse block feeds one of the synapse input signals to the synapse through means for converting a current flowing through said each of the synapse input signal lines to a corresponding voltage representing said one of the synapse input signals.

4. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse output signal lines in each tiling synapse block is fed with one of the synapse output signals by the synapse through means for performing an impedance conversion in order to compensate for dissipation of said one of the synapse output signals.

5. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the tiling input neuron blocks contains function converter means for converting one of the perceptron input signals into one of the synapse input signals using a monotonous and nonlinear function.

6. The multi-layer perceptron circuit device of claim 5, wherein the function converter means comprises a differential amplifier.

7. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the tiling output neuron blocks contains function converter means for converting one of the synapse output signals into one of the perceptron output signals using a monotonous and nonlinear function.

8. The multi-layer perceptron circuit device of claim 7, wherein the function converter means comprises a differential amplifier.

9. The multi-layer perceptron circuit device of claim 1, wherein each of the synapses comprises:

means for carrying out weight calculations on the synapse input signals transmitted through the synapse input signal lines to obtain the synapse output signals outputted through the synapse output signal lines including multiplier means for multiplying one of the synapse input signals transmitted through one of the synapse input signal lines by a multiplication factor; and

programmable voltage source means, connected with the multiplier means, for providing a voltage corresponding to the multiplication factor, the voltage being controllably variable.

10. The multi-layer perceptron circuit device of claim 9, wherein the programmable voltage source means includes a pair of RC circuit configurations, connected to the multiplier means, for controlling the voltage corresponding to the multiplication factor.

11. The multi-layer perceptron circuit device of claim 9, wherein the programmable voltage source means includes a pair of combinations of capacitors and a switch inserted between the capacitors, which are connected to the multiplier means, for controlling the voltage corresponding to the multiplication factor.

12. The multi-layer perceptron circuit device of claim 9, wherein the programmable voltage source means includes a pair of combinations of capacitors connected through a switch, and a pair of integrator circuits in which each integrator circuit is connected to one of the pair of combinations through a switch, for controlling the voltage corresponding to the multiplication factor.

13. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse error signal input lines in each tiling synapse block feeds one of the synapse error input signals to the synapse through means for converting a current flowing through said each of the synapse error signal input lines to a corresponding voltage representing said one of the synapse error input signals.

14. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse error signal output lines in each tiling synapse block is fed with one of the synapse error output signals by the synapse through means for performing an impedance conversion in order to compensate for dissipation of said one of the synapse error output signals.

15. The multi-layer perceptron circuit device of claim 1, wherein the first and second input signals are obtained as a pair of complementary output signals of a differential amplifier.

16. The multi-layer perceptron circuit device of claim 1, wherein the reference voltage generator means comprises:

third and fourth transistors, each having a first, second and third terminal, the third and fourth transistors having their first terminals connected to the first voltage source, and their second terminals connected to each other as well as to the second output terminal, wherein the third terminals of the third and fourth transistors receive first and second reference voltages; and

additional load means connected between the second output terminal and the second voltage source for loading the second output terminal.

17. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the output tiling neuron blocks further comprises:

differential function converter means for converting one of the synapse output signals transmitted through one of the synapse output signal lines into a converted signal using a monotonous and nonlinear function; and

means for obtaining one of the synapse error input signals from one of the perceptron output signals, the teacher signals, and the converted signal obtained by the differential function converter means.

18. The multi-layer perceptron circuit device of claim 17, wherein the differential function converter means comprises:

first and second transistors, each having first, second, and third terminals, the first and second transistors having their first terminals connected to a first voltage source, and their second terminals connected together to form a first output terminal, the first transistor receiving an externally supplied first input signal representing a function at its third terminal and the second transistor receiving an externally supplied second input signal representing a complement of the function such that an output signal at the first output terminal represents a derivative of the function;

load means connected between the first output terminal and a second voltage source for loading the first output terminal;

reference voltage generator means connected between the first voltage source and the second voltage source for generating DC offset voltage; and

a second output terminal connected to the reference voltage generator means.

19. The multi-layer perceptron circuit device of claim 18, wherein the first and second input signals are obtained as a pair of complementary output signals of a differential amplifier.

20. The multi-layer perceptron circuit device of claim 18, wherein the reference voltage generator means comprises:

third and fourth transistors, each having a first, second, and third terminal, the third and fourth transistors having their first terminals connected to the first voltage source, and their second terminals connected to each other as well as to the second output terminal, wherein the third terminals of the third and fourth transistors receive first and second reference voltages; and

additional load means connected between the second output terminal and the second voltage source for loading the second output terminal.