Neural network circuit and learning method for neural network circuit

• US 9,792,547 B2
• Filed: 03/09/2015
• Issued: 10/17/2017
• Est. Priority Date: 03/18/2014
• Status: Active Grant

• Current Assignee: Piece Future Pte., Ltd.
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Nishitani, Yu, Ueda, Michihito, Kaneko, Yukihiro
• Primary Examiner(s): Misir, Dave
• Application Number:
  US14/641,835
• Publication Number:
  US 20150269483A1
• Time in Patent Office: 953 Days
• Field of Search: 706 25- 26
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/065 Analogue means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A neural network circuit includes an error calculating circuit that generates an error voltage signal having a magnitude in accordance with a time difference between an output signal and a teaching signal corresponding to the output signal. A weight change pulse voltage signal is input to a synapse circuit of a neural network circuit element including a neuron circuit that output the weight change pulse voltage signal, and a switching pulse voltage signal is input to a synapse circuit of a neural network circuit element other than the neural network circuit element including the neuron circuit that output the switching pulse voltage signal. The neural network circuit element changes the amplitude of the weight change pulse voltage signal on the basis of the error voltage signal generated by the error calculating circuit.

Claims

1. A neural network circuit comprising:

a plurality of neural network circuit elements;

an error calculating circuit;

at least one input signal terminal; and

at least one output signal terminal,

wherein at least one output signal output from the at least one output signal terminal is obtained from an input signal input to the at least one input signal terminal,

wherein the error calculating circuit receives the at least one output signal and teaching signals equal in number to a number of the at least one output signal terminal and generates an error voltage signal representing a voltage signal having an amplitude in accordance with a time difference between an output signal and a teaching signal corresponding to one of the least one output signal,

wherein each of the neural network circuit elements includes at least one synapse circuit and a neuron circuit,

wherein the synapse circuit includes a variable resistive element having a resistance value that varies when a pulse voltage is applied,

wherein the neuron circuit includes a waveform generating circuit, and the waveform generating circuit generates a weight change pulse voltage signal having a predetermined first waveform that rises from a reference value to a predetermined peak value and then falls again to the reference value as time passes and a switching pulse voltage signal having a predetermined second waveform that determines a predetermined duration,

wherein the weight change pulse voltage signal is input to the synapse circuit of the neural network circuit element including the neuron circuit that outputs the weight change pulse voltage signal,

wherein the switching pulse voltage signal is input to the synapse circuit of the neural network circuit element other than the neural network circuit element including the neuron circuit that outputs the switching pulse voltage signal,

wherein the neural network circuit element changes an amplitude of the weight change pulse voltage signal on the basis of the error voltage signal generated by the error calculating circuit, and

wherein for the predetermined duration of the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the synapse circuit, the synapse circuit changes the resistance value of the variable resistive element of the synapse circuit using a voltage in accordance with a time difference between the switching pulse voltage signal and the weight change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the synapse circuit.

2. The neural network circuit according to claim 1, wherein if the time difference between the teaching signal and the output signal is zero, the teaching signal has a reference potential,

wherein if the time difference between the teaching signal and the output signal is within a predetermined range and a center potential is defined as the reference potential, the potential difference of the teaching signal from the center potential increases with increasing time difference in a bipolar manner, and

wherein if the time difference between the teaching signal and the output signal is outside the predetermined range, an amplitude of the teaching signal is maintained at a maximum value of the potential difference obtained in the predetermined range in a bipolar manner.

3. The neural network circuit according to claim 1, wherein the error calculating circuit includes time difference calculating circuits equal in number to the number of the output signal terminals and a summing circuit,

wherein each of the time difference calculating circuits generates the error voltage signal in accordance with a time difference between the output signal output from the corresponding output signal terminal and the teaching signal corresponding to the output signal and inputs the error voltage signal to the neuron circuit included in the neural network circuit element having an output signal that is the same as the output signal output from the output signal terminal,

wherein the summing circuit generates a sum voltage signal obtained by summing the error voltage signals generated by the time difference calculating circuits and inputs the sum voltage signal to the neuron circuit included in the neural network circuit element having an output signal that is not the same as the output signal output from the output signal terminal.

4. The neural network circuit according to claim 1, wherein the variable resistive element includes a first terminal, a second terminal, and a third terminal,

wherein a constant voltage based on the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistive element is applied between the first terminal and the second terminal,

wherein a voltage in accordance with a time difference between the switching pulse voltage signal and the weight change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the variable resistive element is applied between the first terminal and the third terminal for the predetermined duration of the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistive element, and

wherein a resistance value between the first terminal and the second terminal varies in accordance with a potential difference between the first terminal and the third terminal.

5. The neural network circuit according to claim 4, wherein the synapse circuit includes a first switch that connects and disconnects the third terminal of the variable resistive element from a terminal to which the weight change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the variable resistive element is input, and

wherein the first switch controls the connection and disconnection on the basis of the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistive element.

6. The neural network circuit according to claim 4, wherein the variable resistive element is a ferroelectric memristor.

7. The neural network circuit according to claim 6, wherein the ferroelectric memristor includes a control electrode formed on a substrate, a ferroelectric layer in contact with the control electrode, a semiconductor layer formed on the ferroelectric layer, and a first electrode and a second electrode formed on the semiconductor layer, and

wherein a resistance value between the first electrode and the second electrode varies in accordance with a potential difference between the first electrode and the control electrode.

8. The neural network circuit according to claim 1, wherein the neuron circuit includes an integration circuit that integrates a value of an electric current flowing in the variable resistive element of the synapse circuit and a waveform generating circuit that generates the first waveform and the second waveform in accordance with the electric current integrated by the integration circuit, and

wherein the waveform generating circuit includes a multiplier circuit that multiplies a magnitude of the first waveform by a magnitude of the error voltage signal.

9. The neural network circuit according to claim 4, wherein the synapse circuit includes a second switch having one end connected to a first reference voltage source and the other end connected to the first terminal of the variable resistive element, and

wherein the second switch connects the first reference voltage source to the first terminal for the predetermined duration of the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the synapse circuit.

10. A learning method for use in the neural network circuit according to claim 1, comprising:

changing an amplitude of the weight change pulse voltage signal output from the neuron circuit of a first neural network circuit element representing the neural network circuit element having an output signal serving as the output signal output from the output signal terminal on the basis of the error voltage signal; and

subsequently changing an amplitude of the weight change pulse voltage signal output from the neuron circuit of a second neural network circuit element representing the neural network circuit element having an output signal that is not the output signal output from the output signal terminal on the basis of the error voltage signal.

11. The learning method according to claim 10,

wherein a first step includes changing the resistance value of the variable resistive element of the first neural network circuit element with the resistance value of the variable resistive element of the second neural network circuit element remaining unchanged by inputting a signal having a potential that is the same as the reference value to the synapse circuit of the second neural network circuit element instead of inputting the weight change pulse voltage signal,

wherein a second step includes changing the resistance value of the variable resistive element of the second neural network circuit element by causing the synapse circuit of the second neural network circuit element to generate the weight change pulse voltage signal having the first waveform generated by the neuron circuit of the neural network circuit element including the synapse circuit, and

wherein the first step and the second step are repeated until a time difference between the teaching signal and the corresponding output signal reaches a predetermined value or less.

12. A neural network circuit comprising:

an error calculating circuit to which an output signal and a teaching signal are input, the error calculating circuit generating an error signal having a voltage according to a time difference between the output signal and the teaching signal;

first one or more neural network circuit elements included in an intermediate layer of the neural network circuit; and

second one or more neural network circuit elements included in an output layer of the neural network circuit, the output layer outputting the output signal,

wherein each of the first one or more neural network circuit elements includes first one or more synapse circuits and a first neuron circuit,

wherein each of the second one or more neural network circuit elements includes second one or more synapse circuits and a second neuron circuit,

wherein each of the first one or more synapse circuits and the second one or more synapse circuits includes a variable resistive element having a resistance value that varies in accordance with a voltage value of a pulse voltage applied to the variable resistive element,

wherein each of the first neuron circuit and the second neuron circuit includes a waveform generating circuit that generates a weight change pulse voltage signal having a waveform that rises from a reference value to a peak value and then falls again to the reference value as time passes and a switching pulse voltage signal,

wherein a first weight change pulse voltage signal representing the weight change pulse voltage signal generated by the first neuron circuit is input to each of the first one or more synapse circuits,

wherein a second weight change pulse voltage signal representing the weight change pulse voltage signal generated by the second neuron circuit is input to each of the second one or more synapse circuits,

wherein a first switching pulse signal representing the switching pulse signal generated by the first neuron circuit is input to each of the second one or more synapse circuits,

wherein an amplitude of the first weight change pulse voltage signal and an amplitude of the second weight change pulse voltage signal are determined on the basis of the error signal, and

wherein the resistance value of the variable resistive element included in each of the second one or more synapse circuits is varied on the basis of a duration of the first switching pulse voltage signal and a voltage in accordance with a time difference between the first switching pulse voltage signal and the second weight change pulse voltage signal.

13. The neural network circuit according to claim 12, wherein the switching pulse voltage signal generated by the second neuron circuit is the output signal.