Configurable precision neural network with differential binary non-volatile memory cell structure

• US 10,643,705 B2
• Filed: 05/16/2019
• Issued: 05/05/2020
• Est. Priority Date: 07/24/2018
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Inventors: Choi, Won Ho, Chiu, Pi-Feng, Ma, Wen, Lueker-Boden, Martin
• Primary Examiner(s): Hoang, Huan
• Application Number:
  US16/414,143
• Publication Number:
  US 20200035305A1
• Time in Patent Office: 355 Days
• Field of Search: 365189011
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G11C 11/54 using elements simulating biological cells, e.g. neuron
• CPC Class Codes: G11C 13/0028 Word-line or row circuits
• CPC Class Codes: G11C 13/003 Cell access
• CPC Class Codes: G11C 13/004 Reading or sensing circuits or methods
• CPC Class Codes: G11C 13/0069 Writing or programming circuits or methods
• CPC Class Codes: G11C 16/08 Address circuits; Decoders; Word-line control circuits
• CPC Class Codes: G11C 2013/0073 Write using bi-directional cell biasing
• CPC Class Codes: G11C 2213/71 Three dimensional array
• CPC Class Codes: G11C 2213/75 Array having a NAND structure comprising, for example, memory cells in series or memory elements in series, a memory element being a memory cell in parallel with an access transistor
• CPC Class Codes: G11C 2213/77 Array wherein the memory element being directly connected to the bit lines and word lines without any access device being used
• CPC Class Codes: G11C 27/00 Electric analogue stores, e.g. for storing instantaneous values
• CPC Class Codes: G11C 7/16 Storage of analogue signals in digital stores using an arrangement comprising analogue/digital [A/D] converters, digital memories and digital/analogue [D/A] converters

Abstract

Use of a non-volatile memory array architecture to realize a neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a neural network is formed by a differential memory cell of two individual memory cells, such as a memory cells having a programmable resistance, each connected between a corresponding one of a word line pair and a shared bit line. An input is applied as a pattern of voltage values on word line pairs connected to the unit synapses to perform the multiplication of the input with the weight by determining a voltage level on the shared bit line. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a summation circuit. The approach can be extended from binary weights to multi-bit weight values by use of multiple differential memory cells for a weight.

Claims

1. A non-volatile memory circuit, comprising:

an array of one or more storage units connected along one or more first input lines and each configured to store an N bit weight of neural network, where N is an integer greater than 1, each of the storage units comprising N non-volatile binary storage elements connected to a corresponding output line and configured to store one bit of the N bit weight; and

one or more control circuits connected to the array of storage units, the one or more control circuits configured to;

apply a first input of a neural network to the one or more first input lines to generate an output voltage level on each of the output lines connected to a first of the storage units in response thereto;

individually weight the output voltage levels generated in response to the first input on each of the output lines according to a significance of the bit of the weight stored in the binary storage element of the first storage unit corresponding to the output line; and

determine a multi-bit value for the response of the weight stored in the first storage unit to the first input from a combination of the individually weighted output voltages to thereby perform an in-array multiplication of the first input with the weight stored in the first storage unit.

2. The non-volatile memory circuit of claim 1, wherein:

the array further comprises a second storage unit connected along one or more second input lines and configured to store an N bit weight of neural network, the second storage unit comprising N non-volatile binary storage elements connected to a corresponding output line and configured to store one bit of the N bit weight, wherein for each of the binary storage elements of a given significance of the second storage unit is connected to the same output line as the binary storage elements of a same significance of the first storage unit; and

the one or more control circuits are further configured to;

apply a second input of the neural network to the one or more second input lines to generate an output voltage level on each of the output lines connected to the second storage unit in response thereto;

individually weight the output voltage levels generated in response to the second input on each of the output lines according to the significance of the bit of the weight stored in the binary storage element of the second storage unit corresponding to the output line; and

determine a multi-bit value for the response of the weight stored in the second storage unit to the second input from a combination of the individually weighted output voltages generated in response to the second input to thereby perform an in-array multiplication of the second input with the weight stored in the second storage unit.

3. The non-volatile memory circuit of claim 2, wherein the one or more control circuits are further configured to:

accumulate the multi-bit value for the response of the weight stored in the first storage unit to the first input and the multi-bit value for the response of the weight stored in the second storage unit to the second input.

4. The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to:

apply the second input of the neural network to the second input lines concurrently with applying the first input of the neural network to the first input lines.

5. The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to:

apply the second input of the neural network to the second input lines sequentially with applying the first input of the neural network to the first input lines.

6. The non-volatile memory circuit of claim 3, wherein the one or more control circuits are further configured to:

concurrently with applying the first input to the first input lines to generate the output voltage level on each of the output lines connected to the first of the storage units in response thereto, apply the first input to the first input lines to generate an output voltage level on each of the output lines connected to a second of the storage units in response thereto;

individually weight the output voltage levels generated in response to the first input on each of the output lines according to a significance of the bit of the weight stored in the binary storage element of the second storage unit corresponding to the output line; and

determine a multi-bit value for the response of the weight stored in the second storage unit to the first input from a combination of the individually weighted output voltages to thereby perform an in-array multiplication of the first input with the weight stored in the second storage unit.

7. The non-volatile memory circuit of claim 1, wherein:

one or more first input lines are a pair of first word lines;

each of the binary storage elements includes a pair of memory cells each connected between the corresponding output line and one of the first word lines;

and the first input is applied to the pair of first word lines as one of a plurality of voltage patterns.

8. The non-volatile memory circuit of claim 7, wherein the memory cells include phase change memory (PCM) based memory cells.

9. The non-volatile memory circuit of claim 7, wherein the array of storage is formed according to a cross-point architecture as part of a monolithic three-dimensional semiconductor memory device where the memory cells are arranged in multiple physical levels above a substrate.

10. A method, comprising:

receiving a plurality of input values for a neural network;

translating each of the plurality of input values into a corresponding first voltage pattern, each first voltage pattern being one of a plurality of voltage patterns comprising a pair of voltage values;

applying the plurality of first voltage patterns to one or more pairs of word lines, each pair of word lines connected to a first set of N pairs of non-volatile memory cells storing an N-bit weight value of the neural network, each pair of memory cells of each first set storing a bit of the weight value and comprising a first memory cell connected between a first word line of the word line pair and a corresponding bit line and a second memory cell connected between a second word line of the word line pair and the corresponding bit line;

weighting an output level on each of the bit lines corresponding to the pair of memory cells of each first set in response to the applied first voltage patterns according to a significance of the bit stored in the corresponding pair of memory cells; and

combining the weighted output levels to determine a multi-bit value for one or more products of the one or more input values with the one or more weight values.

11. The method of claim 10, wherein:

applying the plurality of first voltage patterns to one or more pairs of word lines includes applying the plurality of first voltage patterns sequentially to a corresponding plurality of the word line pairs; and

weighting the output levels on the bit lines in response to the applied plurality of first voltage patterns sequentially.

12. The method of claim 10, wherein:

applying the plurality of first voltage patterns to one or more pairs of word lines includes applying the plurality of first voltage patterns concurrently to a corresponding plurality of the word line pairs; and

weighting the output levels on the bit lines in response to the applied plurality of first voltage patterns concurrently.

13. The method of claim 10, wherein each pair of word lines is further connected to a second set of N pairs of non-volatile memory cells storing an N-bit weight value of the neural network, each pair of memory cells of each second set storing a bit of the weight value and comprising a first memory cell connected between a first word line of the word line pair and a corresponding bit line and a second memory cell connected between a second word line of the word line pair and the corresponding bit line, the method further comprising:

weighting the output levels on the bit lines corresponding to the pair of memory cells of each second set in response to the applied first voltage patterns according to a significance of the bit stored in the corresponding pair of memory cells.

14. An apparatus, comprising:

an array of non-volatile memory cells, including a plurality of N first bit lines, a first word line pair and a first group of N non-volatile memory cell pairs, a first memory cell of each pair of the first group connected between a first word line of the first word line pair and a corresponding one of the first bit lines and a second memory cell of each pair of the first group connected between a second word line of the first word line pair and the corresponding one of the first bit lines; and

one or more control circuits connected to the array of non-volatile memory cells, the one or more control circuits configured to;

receive a first input and apply a first voltage pattern corresponding to the first input to the first word line pair;

individually weight an output level on each of the N first bit lines in response to applying the first voltage pattern to the first word line pair;

combine the individually weighted output levels to form a weighted response of the first group of memory cells to the first input; and

determine a multi-bit value for the weighted response of the first group of memory cells to the first input.

15. The apparatus of claim 14, wherein the output levels are current levels and the one or more control circuits include a current mirror configured to weight the output level on each of the N first bit lines.

16. The apparatus of claim 15, wherein the current mirror is configured to implement weighting of the output level through relative sizing of transistors in the current mirror.

17. The apparatus of claim 14, wherein the output levels are voltage levels and the one or more control circuits include a voltage summation circuit to combine the individually weighted output levels.

18. The apparatus of claim 14, wherein the array of non-volatile memory cells is formed according to a cross-point architecture as part of a monolithic three-dimensional semiconductor memory device where the memory cells are arranged in multiple physical levels above a substrate.

19. The apparatus of claim 14, wherein:

the array of non-volatile memory cells further includes a second word line pair and a second group of N non-volatile memory cell pairs, a first memory cell of each pair of the second group connected between a first word line of the second word line pair and a corresponding one of the first bit lines and a second memory cell of each pair of the second group connected between a second word line of the second word line pair and the corresponding one of the first bit lines; and

the one or more control circuits are further configured to;

receive a second input and apply a second voltage pattern corresponding to the second input to the second word line pair;

individually weight an output level on each of the N first bit lines in response to applying the second voltage pattern to the second word line pair;

combine the individually weighted output levels to form a weighted response of the second group of memory cells to the second input;

determine a multi-bit value for the weighted response of the second group of memory cells to the second input; and

accumulate the multi-bit value for the weighted response of the second group of memory cells to the second input with the multi-bit value for the weighted response of the first group of memory cells to the first input.

20. The apparatus of claim 19, wherein the one or more control circuits are further configured to:

apply the second voltage pattern to the second word line pair concurrently with applying the first voltage pattern to the first word line pair; and

individually weight the output level on each of the N first bit lines in response to concurrently applying the second voltage pattern to the second word line pair and of the first voltage pattern to the first word line pair.