Neural network integrated circuit device having self-organizing function

• US 5,293,457 A
• Filed: 05/01/1992
• Issued: 03/08/1994
• Est. Priority Date: 05/15/1989
• Status: Expired due to Term

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Tomioka, Ichiro, Arima, Yutaka, Hanibuchi, Toshiaki
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): DAVIS, GEORGE B
• Application Number:
  US07/877,514
• Time in Patent Office: 676 Days
• Field of Search: 395/23, 395/24
• US Class Current: 706/34
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

An extension directed integrated circuit device having a learning function on a Boltzmann model, includes a plurality of synapse representing units arrayed in a matrix, a plurality of neuron representing units, a plurality of educator signal control circuits, and a plurality of buffer circuits. Each synapse representing unit is connected to a pair of axon signal transfer lines and a pair of dendrite signal transfer lines. Each synapse representing unit includes a learning control circuit which derives synapse load change value data in accordance with predetermined learning rules in response to a first axon signal Si and a second axon signal Sj, a synapse load representing circuit which corrects a synapse load in response to the synapse load change valued data and holds the corrected synapse load value Wij, a first synapse coupling operating circuit which derives a current signal indicating a product Wij·Si from the synapse load Wij and the first axon signal Si and transfers the same to a first dendrite signal line, and a second product signal indicating a product Wij·Sj from the synapse load Wij and the second axon signal Sj and transfers the same onto a second dendrite signal line.

Claims

1. An integrated circuit device having a learning function, being modelled on vital cells, said integrated circuit device comprising a plurality of nerve cell units performing functions of the bodies of said nerve cells respectively, a plurality of axon signal transfer lines provided in correspondence of respective nerve cell units for transferring axon signals indicating states of corresponding nerve cell units, a plurality of dendrite signal lines provided in correspondence to said respective nerve cell units for transferring signals to corresponding nerve cell units, and a plurality of synapse load representing circuits provided on respective crosspoints between said plurality of axon signal transfer lines and said plurality of dendrite signal lines for joining specific synapse loads to signal potentials on corresponding axon signal transfer lines and transferring the same onto corresponding dendrite signal lines, said synapse loads begin adjustable in learning of said integrated circuit device,

each synapse load representing circuit comprising:

learning control means for receiving a first axon signal Si and a second axon signal Sj and outputting a change value of each synapse load in accordance with a predetermined learning rule;

synapse load representing circuit means for outputting a changed synapse load value Wij in accordance with said change value received from said learning control means;

first synapse coupling operating circuit means for transferring a signal indicating a product of said each synapse load from said synapse load representing circuit means and said first axon signal to a first dendrite signal line; and

second synapse coupling operating circuit means for transferring a signal indicating a product of said each synapse load from said synapse load representing circuit means and said second axon signal onto a second dendrite signal line.

2. An integrated circuit device in accordance with claim 1, wherein

said plurality of nerve cell units of said integrated circuit device has a learning function and include input nerve cell units forming an input layer for receiving input data from the exterior and output nerve cell units forming an output layer for deriving output data to the exterior, and

each learning control means comprises:

means for operating a product of said first axon signal Si and said second axon signal Sj;

first and second storage means for holding the outputs of said product operating means, said first storage means storing a product S⁺ i·

S⁺ j operated in a first learning mode in which both axon signals of said input and output nerve cell units are fixed at a value of educator data to be learned, said second storage means storing a result S^- i·

S^- j of product operation in a second learning mode in which only axon signals of said input nerve cell units are fixed at the value of educator data, said first and second storage means storing results of product operation in corresponding said first and second learning modes in response to externally supplied control signals; and

circuit means for outputting a synapse load change value Δ

Wij through said values stored in said first and second storage means in accordance with the following relational expression;

space="preserve" listing-type="equation">Δ

Wij=η

·

(S.sup.+ i·

S.sup.30 j-S.sup.- i·

S.sup.- j)

where η

represents a predetermined learning coefficient.

3. An integrated circuit device in accordance with claim 2, wherein

sad second storage means comprises first register means for receiving the output of said product operating means said first register means latching and outputting supplied data in response to a first control signal,

said first storage means comprises second register means for latching and outputting the output of said first register means in response to a second control signal and third register means for latching and outputting the output of said second register means in response to said first control signal,

said circuit means for outputting a synapse load change value Δ

Wij comprises circuit means for detecting a mismatch of output data from said first and third storage means, and circuit means for passing an externally supplied pulse signal defining said learning coefficient η

in response to a mismatch detecting output received from said mismatch detection circuit and outputting a signal indicating the magnitude of said change value,

data stored in said first storage means also indicating the sign of said change value.

4. An integrated circuit device in accordance with claim 1, wherein

said plurality of nerve cell units include input nerve cell units forming an input layer for receiving input data and output nerve cell units forming an output layer for deriving output data, and

each of said learning control means comprises:

means for operating a product of said first axon signal Si and said second axon signal Sj;

means for storing the output of said product operating means;

circuit means for outputting the following synapse load change value;

space="preserve" listing-type="equation">Δ

Wij=η

·

Si·

Sj

in said storage means; and

means for setting the sign of said synapse load change value in response to an externally supplied learning mode designating signal, said learning mode designating signal designating either a first learning mode in which both axon signals of said input and output nerve cell units are fixed at the value of educator data to be learned or a second learning mode in which only axon signals of said input nerve cell units are fixed at the value of said educator data, said sign setting means making the sign of said synapse load change value positive in said first learning mode while making the sign of said synapse load change value negative in said second learning mode.

5. An integrated circuit device in accordance with claim 4, wherein

said synapse load change value output circuit means comprises means for passing a pulse signal train defining said learning coefficient in response to a signal stored in said storage means.

6. An integrated circuit device having a learning function, being modelled on vital cells, said integrated circuit device comprising a plurality of nerve cell units performing functions of the bodies of said vital cells, a plurality of axon signal transfer lines provided in correspondence to respective nerve cell units for transferring axon signals indicating states of corresponding nerve cell units, a plurality of dendrite signal lines provided in correspondence to respective nerve cell units for transferring signals to said corresponding nerve cell units, a plurality of synapse load representing circuits provided in correspondence to respective crosspoints between said plurality of axon signal transfer lines and said plurality of dendrite signal lines for joining specific weights to signal potentials on corresponding axon signal transfer lines and transferring the same onto corresponding dendrite signal lines, and learning control circuits provided in correspondence to respective synapse load representing circuits and coupled to transfer signal lines for first and second axon signals Si and Sj for outputting signals indicating change values of weights of corresponding synapse load representing circuits, said specific weights being adjustable in a learning mode, said change value indicating signals being formed by increment/decrement indicating signals and a pulse signal train indicating values thereof,

each synapse load representing circuit including:

counter means, being capable of performing count-up and count-down operations, for counting a number of pulse signals corresponding to the value of a synapse load change value generated by a corresponding learning control circuit in response to said increment/decrement indicating signal derived from said corresponding learning control circuit, said counter means being capable of counting a value having a bit number larger than a prescribed bit number required for representing a specific weight and adapted to derive a signal value indicating the value and sign of said specific weight; and

means for receiving said signal value indicating the value of said specific weight received from said counter means for holding the same at a limit value of a corresponding specific weight and outputting the same when a detection means detects a count value being out of said range between predetermined maximum and minimum limit values of said specific weight, received from said counter means when said detection means detects that the count value of said counter means is within said range between predetermined maximum and minimum limit values.

7. An integrated circuit device in accordance with claim 6, wherein

each synapse load representing circuit further includes prevention means for monitoring the count value of said counter means and neglecting a count direction in excess of a maximum or minimum value specific to said counter means itself, that is received from said corresponding learning control circuit, when said count value reaches said maximum or minimum value, thereby preventing said counter means from overflow/underflow.

8. An integrated circuit device in accordance with claim 7, wherein

said prevention means comprises:

underflow/overflow detection means for detecting match/mismatch of all output bits of said counter means, and

means for inhibiting passage of pulse signals to said counter means in response to a match detecting signal received from said underflow/overflow detection means.

9. An integrated circuit device in accordance with claim 6, wherein

said counter means comprises an updown counter having a most significant bit outputting a signal indicating the sign of said specific weight, a plurality of less significant bits outputting a signal indicating the value of said specific weight, and at least one redundant bit between said most significant bit and said plurality of less significant bits,

said detection means comprises means for detecting match/mismatch of the output of said most significant bit and the least significant bit within said at least one redundant bit, and said means for receiving includes

signal passage means provided for respective ones of said plurality of less significant bits for passing either a corresponding bit output of said updown counter or said least significant redundant bit output in response to the output of said match/mismatch detection means.

10. An integrated circuit device having a learning function, being modelled on vital cells, said integrated circuit device comprising a plurality of nerve cell units performing functions of the bodies of said nerve cells respectively, a plurality of axon signal transfer lines provided in correspondence to respective ones of said nerve cells for transferring axon signals indicating states of corresponding nerve cell units, a plurality of dendrite signal lines provided in correspondence to respective ones of said nerve cell units for transferring signals to said correspondence nerve cell units, and synapse load representing circuits provided in correspondence to respective crosspoints between said plurality of axon signal transfer lines and said plurality of dendrite signal lines for generating synapse load information indicating degrees of coupling between corresponding axon signal transfer lines and corresponding dendrite signal lines,

said integrated circuit device further comprising synapse operation representing circuit means provided on said respective crosspoints between said plurality of axon signal transfer lines and said plurality of dendrite signal lines for transferring signals on said corresponding axon signal lines onto said corresponding dendrite signal lines in accordance with said synapse load information received from said synapse load representing circuits,

each synapse operation representing circuit means including:

first gate voltage selection circuit means for selecting either a first reference voltage or a second reference voltage in response to a signal potential on a corresponding axon signal transfer line;

second gate voltage selection circuit means for selecting and outputting either an output voltage from said first gate voltage selection circuit or said second reference voltage in response to an output signal indicating the value of synapse load information received from a corresponding synapse load representing circuit;

third gate voltage selection circuit means for selectively outputting either said second reference voltage or said first reference voltage in response to a signal potential on said corresponding axon signal transfer line and a signal potential indicating the sign of said synapse load information received from said corresponding synapse load representing circuit, said third gate voltage selection circuit means selecting said second reference voltage only when said signal potential on said axon signal transfer line indicates that a corresponding nerve cell unit is in an excitatory state and said signal indicating the sign of said synapse load information received from said corresponding synapse load representing circuit indicates negative synapse load information while selecting and outputting said first reference voltage in other case;

first current supply means for transferring a current corresponding to a signal indicating the value of said synapse load information to a corresponding dendrite signal line in response to the output of said second gate voltage selection circuit means; and

second current supply means for supplying a current responsive to positively/negatively of said synapse load information in response to the output of said third gate voltage selection circuit means,

a current indicating a product of said signal potential indicating a state of said nerve cell unit on said corresponding axon signal transfer line and a corresponding synapse load information transferred onto said corresponding dendrite signal line.

11. An integrated circuit device in accordance with claim 10, wherein

each synapse load representing circuit includes means for expressing the value of said synapse load in N bits and expressing the sign thereof in one bit,

said second gate voltage selection circuit means includes:

a plurality of first selective passage means provided in correspondence to respective ones of said N bits for selectively passing the output of said first gate voltage selection circuit means in response to corresponding synapse load bit data, and

a plurality of second selective passage means provided in correspondence to respective ones of said N bits for passing said second reference voltage in response to corresponding synapse load bit date, said second selective passage means being adapted to complementarily operate with said first selective passage means, outputs of said first and second selective passage means being connected to a common node,

said first current supply means includes a plurality of first current conversion means provided in correspondence to respective ones of said N bits for converting said second reference voltage to current signals and transferring the same to related dendrite signal lines in response to signal potentials of said common node of corresponding said first and second selective passage means, current suppliability of said plurality of first current conversion means being set in proportion to corresponding synapse load bit positions, and

said second current supply means includes second current conversion means for converting said second reference potential to current signals in response to the output of said third gate voltage selection circuit means and transferring the same onto said corresponding dendrite signal lines, said second current conversion means being larger in current suppliability than any of said first current conversion means.

12. An integrated circuit device in accordance with claim 11, wherein

said first and second current supply means comprise a plurality of first insulated gate field effect transistors connected between signal lines for transferring said second reference voltage and said corresponding dendrite signal lines in parallel with each other for receiving outputs of corresponding gate voltage selection circuit means at gates thereof, current suppliability of each said transistors being set by conductances thereof.

13. An integrated circuit device in accordance with claim 12, further comprising a plurality of second insulated gate field effect transistors provided between respective said insulated gate field effect transistors and said corresponding dendrite signal lines for receiving a third reference voltage at gates thereof.

14. An integrated circuit device having a learning function, being modelled on vital cells, said integrated circuit device including a plurality of nerve cell units performing functions of the bodies of said vital cells and dendrite signal lines provided in correspondence to respective nerve cell units for transferring signals to corresponding nerve cell units,

each nerve cell unit including:

comparison means providing a function of a body of a nerve cell, said comparison means having a first input coupled to a corresponding dendrite signal line; and

means coupled to a second input of said comparison means for generating a comparative reference voltage of said comparison means, said comparative reference voltage generating means including means responsive to by an externally supplied logical binary voltage for generating a voltage being vibratingly attenuated with a time constant and a cycle predetermined particular to said comparative reference voltage generating means with respect to a reference bias potential determined specific to said comparative reference voltage generating means and transferring the same to said second input of said comparison means.

15. An integrated circuit device having a learning function, being modelled on vital cells, said integrated circuit device including a plurality of nerve cell units performing functions of the bodies of said vital cells, said plurality of nerve cell units being formed by visible nerve cell units forming an input layer for receiving input data and an output layer for deriving output data, and hidden nerve cell units, said integrated circuit device further comprising:

educator signal control circuit means provided in correspondence to respective nerve cell units for deriving educator signals to be learned by said integrated circuit device in a learning mode,

each educator signal control circuit means comprising;

first storage means for holding and outputting externally supplied data defining an attribute of a corresponding nerve cell unit in an operation mode for defining the attribute of each nerve cell unit while holding and outputting educator data externally supplied through the same terminal as that receiving said attribute defining data in response to a first control signal in an operation mode for setting an educator signal and said learning mode of said integrated circuit device;

second storage means coupled to the output of said first storage means for holding and outputting said data defining an attribute for said nerve cell unit in response to a second control signal in said operation mode for defining the attribute of said nerve cell unit, said operation mode for setting said educator signal and said learning mode of said integrated circuit device;

third storage means coupled to the output of said first storage means for holding said data defining an attribute for said nerve cell unit received from said first storage means in said operation mode for defining the attribute of said nerve cell unit while holding and outputting said educator data received from said first storage means in response to a third control signal in said operation mode for setting said educator signal and said learning mode of said integrated circuit device; and

selection circuit means for selecting one of the output of said third storage means, predetermined first fixed data and data representing a state of a corresponding nerve cell unit in response to an educator data validation/invalidation indicating signal generated in said learning mode and said data defining an attribute for said nerve cell unit held in and outputted from said second storage means and transferring the same onto an axon signal line provided in correspondence to said nerve cell unit,

said data defining an attribute for said nerve cell unit and said educator data being serially transferred to first storage means of an adjacent next-stage educator signal control circuit through said first and third storage means in said operation mode for defining the attribute of each nerve cell unit and said operation mode for setting an educator signal.

16. An integrated circuit device having a learning function, comprising a plurality of axon signal transfer lines, a plurality of dendrite signal lines arranged in a direction for intersecting with said axon signal transfer lines, synapse load representing circuits provided on respective crosspoints between said plurality of axon signal transfer lines and said plurality of dendrite signal lines for joining specific weights to signal potentials on corresponding axon signal transfer lines and transferring the same onto corresponding dendrite signal lines, and learning control circuits provided in correspondence to respective synapse load representing circuits for receiving first axon signals Si on first axon signal transfer liens and second axon signals Sj on second axon signal transfer lines and outputting pulse signals defining amounts of change of weights represented by corresponding synapse load representing circuits with increments thereof in accordance with predetermined learning rules, said specific weights being adjuatable in a learning mode,

each synapse load representing circuit includes:

means for storing synapse load values, said means for storing synapse load values including first capacitor means for storing a synapse load value representing excitatory coupling in the form of charges and second capacitor means for storing a synapse load value representing inhibitory coupling in the form of charges;

means for charging said synapse load values stored in said means for storing synapse load values in response to said pulse signals received from said learning control circuits; and

means for joining weights corresponding to said synapse load values stored in said means for storing synapse load values to an axon signal on a corresponding axon signal transfer line and transferring the same onto a corresponding dendrite signal line, said means for joining and transferring including means for supplying a current signal being proportionate to a product of said axon signal and a respective storage synapse load value onto said corresponding dendrite signal line,

said means for changing synapse load values including means for transferring said pulse signals to said first capacitor means in response to a synapse load increase direction received from a respective learning control circuit through capacitive coupling and transferring said pulse signals to said second capacitor means in response to a decrement direction received from said respective learning control circuit through capacitive coupling.

17. An integrated circuit device in accordance with claim 16, wherein

said transfer means comprises:

first current supply means for converting a first reference potential to a current signal being proportionate to a charging potential for said first capacitor means in response to said charging potential for said first capacitor means and said axon signal on said corresponding axon signal transfer line and transferring the same onto said corresponding dendrite signal line, and

second current supply means for converting said first reference potential to a current signal being proportionate to a charging potential for said second capacitor means in response to said charging potential for said second capacitor means and said axon signal on said corresponding axon signal transfer line and transferring the same onto said corresponding dendrite signal line.

18. An integrated circuit device having a learning function, including a plurality of axon signal transfer lines, a plurality of dendrite signal lines arranged to intersect with said plurality of axon signal transfer lines, and self-organizable synapse load representing circuits provided in correspondence to respective crosspoints between said plurality of axon signal transfer lines and said plurality of axon signal transfer lines and said plurality of dendrite signal lines for joining synapse loads of specific weights to signal potentials on corresponding axon signal transfer lines and transferring the same onto corresponding dendrite signal lines,

each self-organizable synapse load representing circuit comprising:

learning control circuit means coupled to a first axon signal transfer line for transferring a first axon signal Si and a second axon signal transfer line for transferring a second axon signal Sj (i≠

j) and outputting a pulse signal defining the amount of change of a synapse load of a corresponding self-organizable synapse load representing circuit in accordance with a predetermined learning rule, said pulse signal being provided by logically processing said first axon signal Si and said second axon signal Sj along said predetermined learning rules;

synapse load value storage means including at least one capacitor means for storing a synapse load value in the form of charges;

load value change means for changing said synapse load value stored in said synapse load value storage means in response to said pulse signal received from said learning control circuit means; and

means for joining a weight corresponding to said synapse load value stored in said synapse load value storage means to an axon signal on a corresponding axon signal transfer line and transferring the same onto a corresponding dendrite signal line

said means for joining and transferring including means for supplying a current being proportionate to a product of a signal potential on said corresponding axon signal transfer line and the stored said synapse load value.

19. An integrated circuit device in accordance with claim 18, wherein

said learning control circuit means includes:

means for operating a product of said first axon signal Si and said second axon signal Sj, and

means for selectively supplying a pulsing signal to said load value change means in response to the output of said means for operating a product and an externally supplied learning mode designating signal, said pulsing signal being supplied to change said synapse load value stored in at least one capacitor means included in said synapse load value storage means, said learning mode designating signal designating either a first learning mode in which axon signals of visible neurons are fixed at a value of educator data to be learned or a second learning mode in which axon signals of only input neurons in said visible neurons are fixed at said value of said educator data,

said selection means including means for generating synapse load change control pulses representing;

space="preserve" listing-type="equation">Δ

Wij=η

·

Si·

Sj

in said first learning mode while representing;

space="preserve" listing-type="equation">Δ

Wij=-η

·

Si·

Sj

in said second learning mode, where η

corresponds to the number of pulse signals.

20. An integrated circuit device in accordance with claim 18, wherein

said load value change means includes:

a first diode and a second diode being connected in series between a first potential and said synapse load value storage means, and

capacitor means for transferring said pulse signal received from said learning control circuit means to the junction of said first and second diodes by capacitive coupling,

a series body formed by said first and second diodes being provided by an insulated gate field effect transistor having a substrate, first and second conduction terminals and a control terminal, said first conduction terminal providing a first electrode of said series body of said first and second diodes, said substrate providing a second electrode of said series body of said first and second diodes, and a common junction of said second conduction terminal and said control terminal providing the junction of said first and second diodes.

21. A method of forming an integrated circuit device having a learning function, being modelled on vital nerve cells, comprising:

a step of forming an assembly of synapse representing units arrayed in the form of a matrix on a semiconductor chip, educator signal control circuits arranged along a first side of said matrix of said synapse representing units and a second side being adjacent to said first side, and buffer circuits arranged along a third side of said matrix of said synapse representing units and a fourth side being adjacent to said first side,

each of a plurality of neuron representing units comprising the function of the body of a nerve cell, and interconnecting a pair of neuron representing units through a specific synapse load,

said buffer circuits comprising interface functions of transferring signal between the interior and the exterior of said semiconductor chip,

educator signal control circuits comprising functions of deriving information to be learned in learning as well as deriving information defining attributes of related neuron representing units,

said matrix of said synapse representing units having a rectangular configuration divided into a pair of right triangles; and

a step of arranging a plurality of axon signal lines connected to said buffer circuits for transferring axon signals indicating states of said neuron representing units and a plurality of dendrite signal lines for transferring dendrite signals, being input signals to said neuron representing units,

one synapse representing unit being connected to a different pair of axon signal lines and a different pair of dendrite signal lines.

22. A method in accordance with claim 21, wherein said buffer circuits are so formed as to transfer axon signals from the exterior to the interior of said semiconductor chip, and

said step of arranging axon and dendrite signal lines includes a step of non-connecting synapse representing units being arranged along a diagonal line of said rectangle of said synapse representing unit matrix and interconnecting synapse representing units and neuron representing units in a full connection manner on respective ones of a first right-triangular region formed by said first side, said fourth side and said diagonal line of said rectangle and a second right-triangular region formed by said third side and said diagonal line.

23. A method in accordance with claim 21, wherein

said buffer circuits are so arranged as to transfer signals from the exterior to the interior of said semiconductor chip, and

said step of arranging a plurality of axon signal lines and a plurality of dendrite signal lines is adapted to arrange axon signal lines and dendrite signal lines in parallel with rows and columns of said matrix of said synapse representing units, said dendrite signal lines being connected to bonding pads provided around said chip while being non-connected to said synapse representing units.

24. A method in accordance with claim 21, wherein

said buffer circuits are so provided as to transfer signals from the interior to the exterior of said semiconductor chip,

each neuron representing unit includes a buffer circuit means for transferring an output signal from either a corresponding neuron representing unit of a related educator signal control circuit onto a corresponding axon signal line on said semiconductor chip, and

said step of arranging a plurality of signal line and a plurality of dendrite signal lines includes a step of non-connecting synapse representing units being arranged along a diagonal line of said rectangle of said matrix of said synapse representing units and forming interconnection of a full connection mode with respect to synapse representing units and neuron representing units on respective ones of a first right-triangular region formed by said first side, said fourth side and said diagonal line and a second right-triangular region formed by said second side, said third side and said diagonal line.

25. A method in accordance with claim 21, wherein

said buffer circuits are so provided as to transfer signals from the interior to the exterior of said semiconductor chip, and each neuron representing unit includes a buffer circuit means for transferring an output signal from either a related neuron representing unit or related educator signal control circuit into a corresponding axon signal line on said semiconductor chip, and

said step of arranging axon and dendrite signal lines is adapted to arrange axon signal lines and dendrite signal lines in parallel with rows and columns of said matrix of said synapse representing units thereby interconnected said synapse representing units on said first side with said buffer circuits on said third side, and said plurality of neuron representing units on said second side with said buffer circuits on said fourth side.

26. A method in accordance with claim 21, wherein

said step of arranging axon and dendrite signal lines includes a step of connecting said dendrite signal lines to bonding pads which are arranged around said semiconductor ship to be adjacent to said plurality of neuron representing units.

27. A method in accordance with claim 21, wherein

each synapse representing unit includes:

a learning control circuit being activated in said learning for receiving a first axon signal Si and a second axon signal Sj and deriving change value information for a synapse load represented by said each synapse representing unit in accordance with a predetermined learning rules,

a synapse load representing circuit for changing said synapse load in accordance with said change value information received from said learning control circuit and holding changed synapse load Wij,

a first synapse coupling operating circuit for transferring a first product signal indicating a product transferring a first product signal indicating a product Wij·

Si of said first axon signal Si and said synapse load Wij to a first dendrite signal line in response to said synapse load Wij held in said synapse load representing circuit and said first axon signal Si, and

a second synapse coupling operating circuit for transferring a second product signal indicating a product of said second axon signal Sj and said synapse load value Wij to a second dendrite signal line in response to said synapse load Wij held in said synapse load representing circuit and said second axon signal Sj.