Multi-layer perceptron circuit device

• US 5,220,641 A
• Filed: 08/02/1991
• Issued: 06/15/1993
• Est. Priority Date: 10/11/1988
• Status: Expired due to Term

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Kamatani, Yukio, Shima, Takeshi
• Primary Examiner(s): Fleming, Michael R.
• Assistant Examiner(s): Davis, George
• Application Number:
  US07/739,505
• Time in Patent Office: 683 Days
• Field of Search: 364/513, 364/807, 364/602, 307/201, 395/24, 395/25, 382/37
• US Class Current: 706/39
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

A multi-layer perceptron circuit device using integrated configuration which is capable of incorporating self-learning function and which is easily extendable. The device includes: at least one synapse blocks containing: a plurality of synapses for performing weight calculation on input signals to obtain output signals, which are arranged in planar array defined by a first and a second directions; input signal lines for transmitting the input signals to the synapses, arranged along the first direction; and output signal lines for transmitting the output signal from the synapses, arranged along the second direction not identical to the first direction; at least one input neuron blocks containing a plurality of neurons to be connected with the input signal lines; and at least one output neuron blocks containing a plurality of neurons to be connected with the output signal lines.

Claims

1. A multi-layer perceptron circuit device with a learning function, comprising:

at least one tiling synapse block including;

a plurality of synapses for performing weight calculation on synapse input signals to obtain output signals;

synapse input signal lines for transmitting the synapse input signals to the synapses;

synapse output signal lines for transmitting the synapse output signal from the synapses;

learning means for updating values of weights to be utilized in weight calculation by a multiplication of synapse error input signals and the synapse input signals;

synapse error signal input lines for transmitting the synapse error input signals to the synapses;

synapse error signal output lines for transmitting the synapse error output signals from the synapses; and

means for calculating the synapse error output signals by a multiplication of the synapse error input signals and the weights;

at least one tiling input neuron block containing a plurality of neurons, each neuron of the at least one tiling input neuron block being connected with one of the synapse input signal lines by its signal output, inputting one of a plurality of perceptron input signals by its signal input, and supplying one of the synapse input signals to one of the synapse input signal lines, each neuron of the tiling input neuron blocks being connected to one of the synapse error signal output lines by its error signal input, receiving one of the synapse error output signals from one of the synapse error signal output lines, and outputting one of a plurality of perceptron error signals by its error signal output, each of the neurons of the tiling input neuron blocks further comprises;

differential function converter means for converting one of the perceptron input signals into a converted signal using a nonlinear function;

means for obtaining one of the perceptron error signals from one of the synapse error output signals transmitted through one of the synapse error signal output lines and the converted signal obtained by the differential function converter means;

function converter means for converting one of the perceptron input signals inputted through said each of the neurons into one of the synapse input signals using a nonlinear function; and

gain control circuit means for controlling a gain of the function converter means and the differential function converter means wherein said gain control circuit means further comprises means for continuously changing the gain of the function converter means and the differential function converter means in a process of learning by the learning means from a value different from that required for a desired result to a value required for the desired result; and

at least one tiling output neuron block containing a plurality of neurons, each neuron of the tiling output neuron blocks being connected with one of the synapse output signal lines by its signal input, and receiving one of the synapse output signals from one of the synapse output signal lines, and outputting a perceptron output signal by its signal output, each neuron of the tiling output neuron blocks being connected to one of the synapse error signal input lines by its error signal output, inputting an externally supplied teacher signal by its error signal input, and supplying one of the synapse error input signals to one of the synapse error signal input lines.

2. The multi-layer perceptron circuit device of claim 1, wherein each of the synapses contains means for carrying out the weight calculations on the synapse input signals transmitted through the synapse input signal lines to obtain the synapse output signals outputted through the synapse output signal lines including:

multiplier means for multiplying one of the synapse input signals transmitted through one of the synapse input signal lines by a multiplication factor; and

voltage source means, connected with the multiplier means, for providing a voltage corresponding to the multiplication factor to the multiplier means.

3. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse input signal lines in each tiling synapse block feeds one of the synapse input signals to the synapse through means for converting a current flowing through said each of the synapse input signal lines to a corresponding voltage representing said one of the synapse input signals.

4. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse output signal lines in each tiling synapse block is fed with one of the synapse output signals by the synapse through means for performing an impedance conversion in order to compensate a dissipation of said one of the synapse output signals.

5. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the tiling input neuron blocks contains function converter means for converting one of the perceptron input signals into one of the synapse input signals using a monotonous and nonlinear function.

6. The multi-layer perceptron circuit device of claim 5, wherein the function converter means comprises a differential amplifier.

7. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the tiling output neuron blocks contains function converter means for converting one of the synapse output signals into one of the perceptron output signals using a monotonous and nonlinear function.

8. The multi-layer perceptron circuit device of claim 7, wherein the function converter means comprises a differential amplifier.

9. The multi-layer perceptron circuit device of claim 1, wherein each of the synapses comprises means for carrying out weight calculations on the synapse input signals transmitted through the synapse input signal lines to obtain the synapse output signals outputted through the synapse output signal lines including:

multiplier means for multiplying one of the synapse input signals transmitted through one of the synapse input signal lines by a multiplication factor; and

programmable voltage source means, connected with the multiplier means, for providing a voltage corresponding to the multiplication factor to the multiplier means, the voltage being controllably variable.

10. The multi-layer perceptron circuit device of claim 9, wherein the programmable voltage source means includes a pair of RC circuit configurations, connected to the multiplier means, for controlling the voltage corresponding to the multiplication factor.

11. The multi-layer perceptron circuit device of claim 9, wherein the variable voltage source means includes a pair of combinations of capacitors with a switch inserted between the capacitors, which are connected to the multiplier means, for controlling the voltage corresponding to the multiplication factor.

12. The multi-layer perceptron circuit device of claim 9, wherein the programmable voltage source means includes a pair of combinations of capacitors connected through a switch, and a pair of integrator circuits in which each integrator circuit is connected to one of the pair of combinations through a switch, for controlling the voltage corresponding to the multiplication factor.

13. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse error signal input lines in each tiling synapse block feeds one of the synapse error input signals to the synapse through means for converting a current flowing through said each of the synapse error signal input lines to a corresponding voltage representing said one of the synapse error input signals.

14. The multi-layer perceptron circuit device of claim 1, wherein each of the synapse error signal output lines in each tiling synapse block is fed with one of the synapse error output signals by the synapse through means for performing an impedance conversion in order to compensate a dissipation of said one of the synapse error output signals.

15. The multi-layer perceptron circuit device of claim 1, wherein the gain control circuit means comprises two RC circuits, connected to different voltage sources, for controlling the gain of the function converter means and the differential function converter means, and a switch for selecting one of the two RC circuits.

16. The multi-layer perceptron circuit device of claim 1, wherein the gain control circuit means comprises counter means for counting numbers, and a converter means for converting the numbers counted by the counter means into a distinct voltage for controlling the gain of the function converter means and the differential function converter means.

17. The multi-layer perceptron circuit device of claim 1, wherein the function converter means has a Gilbert gain cell configuration.

18. The multi-layer perceptron circuit device of claim 1, wherein each of the neurons of the output tiling neuron blocks further comprises:

differential function converter means for converting one of the synapse output signals transmitted through one of the synapse output signal lines into a converted signal using a nonlinear function; and

means for obtaining one of the synapse error input signals from one of the perceptron output signals, the teacher signals, and the converted signal obtained by the differential function converter means.

19. The multi-layer perceptron circuit device of claim 18, wherein each of the neurons of the tiling output neuron blocks further comprises:

function converter means for converting one of the synapse output signals transmitted through one of the synapse output signal lines into one of the perceptron output signals using a monotonous and nonlinear function; and

gain control circuit means for controlling a gain of the function converter means and the differential function converter means such that the gain is continuously changed in a process of learning by the learning means from a value different from that required for a desired result to a value required for the desired result.

20. The multi-layer perceptron circuit device of claim 19, wherein the gain control circuit means of each of the neurons of the tiling output neuron blocks comprises two RC circuits, connected to different voltage sources, for controlling the gain of the function converter means and the differential function converter means, and a switch for selecting one of the two RC circuits.

21. The multi-layer perceptron circuit device of claim 19, wherein the gain control circuit means of each of the neurons of the tiling output neuron blocks comprises counter means for counting numbers, and a converter means for converting the numbers counted by the counter means into a distinct voltage for controlling the gain of the function converter means and the differential function converter means.

22. The multi-layer perceptron circuit device of claim 19, wherein the function converter means of each of the neurons of the tiling output neuron blocks has a Gilbert gain cell configuration.

23. The multi-layer perceptron circuit device of claim 19, wherein each neuron of the tiling output neuron blocks further includes a differential amplifier, having an emitter degeneration resistor and a current source transistor for supplying currents to the differential amplifier, for controlling transmission of the synapse input signals and synapse output signals from said each neuron.

24. The multi-layer perceptron circuit device of claim 23, wherein the gain of the function converter means and the differential function converter means are controlled by changing a resistance of the emitter degeneration resistor.

25. The multi-layer perceptron circuit device of claim 23, wherein the gain of the function converter means and the differential function converter means are controlled by changing a current supplied from the current source transistor.

26. The multi-layer perceptron circuit device of claim 19, wherein the gain control circuit means of each of the neurons of the tiling output neuron blocks controls the gain of the function converter means such that a slope of the nonlinear function used by the function converter means is continuously increased.

27. The multi-layer perceptron circuit device of claim 1, wherein each neuron of the tiling input neuron blocks further includes a differential amplifier, having an emitter degeneration resistor and a current source transistor for supplying currents to the differential amplifier, for controlling transmission of the synapse input signals and synapse output signals from said each neuron.

28. The multi-layer perceptron circuit device of claim 27, wherein the gain of the function converter means and the differential function converter means are controlled by changing a resistance of the emitter degeneration resistor.

29. The multi-layer perceptron circuit device of claim 27, wherein the gain of the function converter means and the differential function converter means are controlled by changing a current supplied from the current source transistor.

30. The multi-layer perceptron circuit device of claim 1, wherein the gain control circuit means controls the gain of the function converter means such that a slope of the nonlinear function used by the function converter means is continuously increased.