Neural network processing system using semiconductor memories

• US 20020032670A1
• Filed: 12/20/2000
• Published: 03/14/2002
• Est. Priority Date: 01/24/1990
• Status: Active Grant

• Current Assignee: Renesas Electronics Corporation
• Original Assignee: Hitachi, Ltd.
• Inventors: Kawajiri, Yoshiki, Kimura, Katsutaka, Watanabe, Takao, Itoh, Kiyoo
• Granted Patent: US 7,043,466 B2
• Time in Patent Office: Days
• US Class Current: 706/33
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

Herein disclosed is a data processing system having a memory packaged therein for realizing a large-scale and high-speed parallel distributed processing and, especially, a data processing system for the neural network processing. The neural network processing system according to the present invention comprises: a memory circuit for storing neuron output values, connection weights, the desired values of outputs, and data necessary for learning; an input/output circuit for writing or reading data in or out of said memory circuit; a processing circuit for performing a processing for determining the neuron outputs such as the product, sum and nonlinear conversion of the data stored in said memory circuit, a comparison of the output value and its desired value, and a processing necessary for learning; and a control circuit for controlling the operations of said memory circuit, said input/output circuit and said processing circuit. The processing circuit is constructed to include at least one of an adder, a multiplier, a nonlinear transfer function circuit and a comparator so that at least a portion of the processing necessary for determining the neutron output values such as the product or sum may be accomplished in parallel. Moreover, these circuits are shared among a plurality of neutrons and are operated in a time sharing manner to determine the plural neuron output values. Still moreover, the aforementioned comparator compares the neuron output value determined and the desired value of the output in parallel.

Claims

1. A data processing system comprising:

a memory (A, B, TG) for storing data;

an input/output circuit (IO) for performing at least one of the operation of writing data in said memory and the operation of reading data from said memory;

an arithmetic circuit (12, 12a, 12b) for arithmetics by using the data stored in said memory; and

a control circuit (CNT, 13, 13A, 13B) for controlling the operations of said memory, said input/output circuit and said arithmetic circuit,

wherein said arithmetic circuit has;

a function to calculate the updated value of the output value of a neuron, which is stored in said memory, by using said neuron output value and the connection weight between neurons; and

a function to calculate the distance (or similarity) between the desired value of the neuron output value stored in said memory and the neuron output value obtained.

2. A data processing system according to claim 1, wherein said memory is stored with a plurality of input data having different characteristics.

3. A data processing system according to claim 2, wherein said memory is stored with a desired value of the output value of the neuron.

4. A data processing system according to claim 2, wherein said memory is constructed of a plurality of blocks, of which:

the first block (A) is stored with the neuron output value (Vis); and

the second block (B) is stored with the connection weight (Tji) between the neurons.

5. A data processing system according to claim 4, wherein said memory further has a third block (TG) stored with the desired value of the neuron output value.

6. A data processing system according to claim 5, wherein said memory is stored with at least one of said neuron output value and said connection weight between the neurons by using a plurality of bits.

7. A data processing system according to claim 6, wherein the plural bits expressing the neuron output value or the connection weight between the neurons are partially used to express the codes of said neuron 5 output value and said connection weight between the neurons.

8. A data processing system according to claim 7, wherein said memory is constructed of memory cells each composed of one transistor and one capacitor.

9. A data processing system comprising:

a memory (A, B, TG) for storing data;

an input/output circuit (IO) for performing at least one of the operation of writing data in said memory and the operation of reading data from said memory;

an arithmetic circuit (12, 12a, 12b) for arithmetics by using the data stored in said memory; and

a control circuit (CNT, 13, 13A, 13B) for controlling the operations of said memory, said input/ output circuit and said arithmetic circuit,

wherein said memory includes a memory cell array having;

a plurality of data lines;

a plurality of word lines (WA) arranged to intersect with said data lines; and

memory cells (MC) arranged at the desired ones of said intersections, so that the data stored in the different plural memory cells can be read out to the different plural data lines intersecting with said word lines by selecting at least one of said word lines,

wherein said arithmetic circuit has a function to calculate the updated value of the neuron output value by using the data read out from said memory, and

wherein said input/output circuit has a function to write said updated value in said memory.

10. A data processing system according to claim 9,

wherein said memory cell array is stored with the desired value of said neuron output values, and

wherein said arithmetic circuit includes means for calculating the distance (or similarity) between said desired value and the obtained neuron output value.

11. A data processing system according to claim 9, wherein said memory cell array is stored with the neuron output value and the connection weight between the neurons.

12. A data processing system according to claim 9, wherein said memory cell array is stored with a plurality of input data having different characteristics.

13. A data processing system according to claim 9, wherein said data processing system is formed over one chip.

14. A data processing system according to claim 10, wherein said memory is constructed of a plurality of blocks, of which:

the first block (A) is stored with the neuron output value; and

the second block (B) is stored with the connection weight between the neurons.

15. A data processing system according to claim 14, wherein said memory is stored with at least one of said neuron output value and said connection weight between the neurons by using a plurality of bits.

16. A data processing system according to claim 15, wherein the plural bits expressing the neuron output value or the connection weight between the neurons are partially used to express the codes of said neuron output value and said connection weight between the neurons.

17. A data processing system according to claim 16, wherein said memory is constructed of memory cells each composed of one transistor and one capacitor.

18. A data processing system according to claim 14, wherein said memory further has a third block (TG) stored with the desired value of the neuron output value.

19. A data processing system according to claim 18, wherein said data processing system is formed over one chip.

20. A data processing system comprising:

a memory (A, B, TG) for storing data;

an input/output circuit (IO) for performing at least one of the operation of writing data in said memory and the operation of reading data from said memory;

an arithmetic circuit (12, 12a, 12b) for arithmetics by using the data stored in said memory; and

a control circuit (CNT, 13, 13A, 13B) for controlling the operations of said memory, said input/ output circuit and said arithmetic circuit,

wherein said arithmetic circuit has;

a function to calculate the updated value of the output value of a neuron, which is stored in said memory, by using said neuron output value and the connection weight between neurons; and

a function to calculate the distance (or similarity) between the desired value of the neuron output value stored in said memory and the neuron output value obtained, and

wherein said data processing system is formed over one chip.

21. A data processing system according to claim 20, wherein said memory is stored with at least one of said neuron output value and said connection weight between the neurons by using a plurality of bits.

22. A data processing system according to claim 21, wherein the plural bits expressing the neuron output value or the connection weight between the neurons are partially used to express the codes of said neuron output value and said connection weight between the neurons.

23. A data processing system according to claim 22, wherein said memory is constructed of memory cells each composed of one transistor and one capacitor.

24. A data processing system according to claim 23, wherein said arithmetic circuit is an analog circuit.