VECTOR-BY-MATRIX MULTIPLIER MODULES BASED ON NON-VOLATILE 2D AND 3D MEMORY ARRAYS

• US 20190213234A1
• Filed: 01/11/2019
• Published: 07/11/2019
• Est. Priority Date: 01/11/2018
• Status: Active Grant

• Current Assignee: Mentium Technologies Inc.
• Original Assignee: Mentium Technologies Inc.
• Inventors: Bayat, Farnood Merrikh, Prezioso, Mirko
• Granted Patent: US 10,552,510 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/065 Analogue means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G11C 11/54 using elements simulating biological cells, e.g. neuron
• CPC Class Codes: G11C 16/0425 comprising cells containing a merged floating gate and select transistor
• CPC Class Codes: G11C 16/0466 comprising cells with charge storage in an insulating layer, e.g. metal-nitride-oxide-silicon [MNOS], silicon-oxide-nitride-oxide-silicon [SONOS] G11C16/0483, G11C16/0491 take precedence
• CPC Class Codes: G11C 16/0483 comprising cells having several storage transistors connected in series
• CPC Class Codes: G11C 16/10 Programming or data input circuits
• CPC Class Codes: G11C 16/26 Sensing or reading circuits; Data output circuits
• CPC Class Codes: G11C 7/1006 Data managing, e.g. manipulating data before writing or reading out, data bus switches or control circuits therefor

Abstract

There are disclosed devices, system and methods for a vector-by-matrix multiplier (VMM) module having a three-dimensional memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input and an output signal in a range that is based on a stored charge and an input signal during assertion of the activation signal. The memory devices are arranged in two dimensional (XY) layers that are vertically disposed along (Z) columns. The activation inputs of each layer are connected to a same activation signal, the memory devices of rows in a first dimension (X) of each layer have signal inputs connected to different input signals and have signal outputs connected in series to a common output. The memory devices of rows in a second dimension (Y) of each layer have signal inputs connected to a set of the same inputs along the first dimension.

Claims

1. A vector-by-matrix multiplier (VMM) module comprising:

a three-dimensional (3D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input to receive an input signal and signal output to output an output signal when the activation input receives an activation signal,

the output signal being in a range that is based on a charge stored in the charge storage, an input applied to the activation input, and an input signal received at the signal input;

the nonvolatile memory devices arranged in a plurality of two dimensional (2D) (XY) layers that are vertically disposed along a plurality of (Z) columns, the activation inputs of memory devices of each layer connected to a same activation input signal, the memory devices of each layer having signal inputs connected to signal outputs of memory devices in the layer above along a third dimension (Z) and having signal outputs connected to the signal inputs of memory devices in the layer below along a third dimension (Z).

8. The module of claim 1, wherein VMM module inputs are signal inputs of a top 2D layer of the memory devices connected together along the second dimension (Y), and VMM module outputs are signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X).

9. The module of claim 8, wherein each two-dimensional (2D) (XY) layer that is vertically disposed along a plurality of (Z) columns can be selected by the application of a proper input to the same activation input of that layer and the application of a larger input to the same activation inputs of other layers.

10. The module of claim 9, wherein when the same activation input of a selected layer receives an activation signal, the VMM module outputs of the bottom layer output signals that are the vector-by-matrix multiplication of weights stored in the nonvolatile memory devices of the selected layer which are proportional to the charges stored in their charge storages and the input signals received at the VMM module inputs of the top layer.

11. The module of claim 9, wherein the module outputs add the signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of bottom layer;

and

wherein the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of the bottom layer from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the first dimension (X) of that layer.

12. The module of claim 11, wherein the VMM module inputs are drain terminals of a top 2D layer of the memory devices connected together along bit lines (BLs) along the second dimension (Y), and the VMM module outputs are source terminals of a bottom 2D layer of the memory devices connected together along source lines (SLs) along the first dimension (X).

13. The module of claim 11, further comprising:

activation circuitry to provide the activation signal to bias the FGTs in a read mode so there would be no change in the charge storage of the FGTs and so that the FGTs operate in analog mode to have output signals in a linear range with respect to the dot products of the weight stored in the charge storage and the input signal received at the signal input of each of the memory devices.

14. The module of claim 9, further comprising:

input circuitry to input each of a set of input signals into corresponding ones of the set of the same VMM module inputs along the first dimension (X) of memory devices; and

the output circuitry to connect the VMM module signal outputs connected in series of the rows of the memory devices in the first dimension (X) into a neural path outputs of a neural network.

15. The module of claim 9, wherein the nonvolatile memory devices are floating gate transistors (FGTs);

the stored charges are floating gate charges;

the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs.

16. The module of claim 8, wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storages based on parameters of a neural network implemented using the VMM module and trained with a training dataset.

2-7. -7. (canceled)

17. A vector-by-matrix multiplier (VMM) module comprising:

a three-dimensional (3D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input to receive an input signal and a signal output to output an output signal when the activation input receives an activation signal;

the output signal being in a range that is based on a charge stored in the charge storage, an input applied to the activation input, and an input signal received at the signal input;

the nonvolatile memory devices arranged in a plurality of two dimensional (2D) (XY) layers that are vertically disposed along a plurality of (Z) columns, the activation inputs of nonvolatile memory devices along a first dimension (X) of each layer connected to a same activation input (WL), the memory devices of each layer having signal inputs connected to signal outputs of memory devices in the layer above along a third dimension (Z) and having signal outputs connected to the signal inputs of memory devices in the layer below along a third dimension (Z).

18. The module of claim 17, wherein VMM module outputs are signal outputs of a top 2D layer of the memory devices connected together along the second dimension (Y).

19. The module of claim 17, wherein each two-dimensional (2D) (XY) layer that is vertically disposed along a plurality of (Z) columns can be selected by the application of a proper input to the activation inputs (WLs) of that layer and the application of a larger input to the activation inputs (WLs) of other layers.

20. The module of claim 17, further comprising:

activation circuitry to provide the signals to bias the nonvolatile memory devices in a selected layer in a read mode during operation so there would be no change in the charge storage of the nonvolatile memory devices and so that the nonvolatile memory devices operate in analog mode to have output signals which are dot products of weights stored in the charge storages and VMM module input signals.

21. The module of claim 20, wherein the input activation circuitry converts the VMM module input signals to voltage with a diode-connected nonvolatile memory device to bias the activation inputs (WLs) of the selected layer (called gate-coupled) and bias the signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X) with a fixed voltage;

or

wherein the input activation circuitry converts the VMM module input signals to voltage with an operational amplifier with a nonvolatile memory device feedback to bias the signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X) (called source-coupled) and bias the activation inputs (WLs) of a nonvolatile memory devices in a selected layer connected together along the first dimension (X) with a fixed voltage;

22. The module of claim 17, wherein when the activation inputs (WLs) of a selected layer receive activation signals, VMM module outputs of the top layer output signals that are the vector-by-matrix multiplication of the weight stored in the nonvolatile memory devices of the selected layer proportional to the charges stored in their charge storages and the input signals received at VMM module inputs.

23. The module of claim 17, wherein module outputs add the signal outputs of the nonvolatile memory devices of rows in the second dimension (Y) of the top layer;

and

wherein the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the second dimension (Y) of the top layer from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the second dimension (Y) of that layer.

24. The module of claim 17, wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storages based on parameters of a neural network implemented using the VMM module and trained with a training dataset.

25. The module of claim 17, wherein the nonvolatile memory devices are floating gate transistors (FGTs);

the stored charges are floating gate charges;

the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs.

26. The module of claim 25, wherein VMM module outputs are drain terminals of a top 2D layer of the memory devices connected together along bit lines (BLs) along the second dimension (Y).

27. The module of claim 17, further comprising:

input circuitry to input each of a set of input signals into corresponding ones of a set of same VMM module inputs along the second dimension (Y) of memory devices; and

the output circuitry to connect VMM module signal outputs connected in series of the rows of the memory devices in the first dimension (Y) into neural path outputs of a neural network.

28. A vector-by-matrix multiplier (VMM) module comprising:

a two-dimensional (2D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input and signal output to output an output signal when the activation input receives an activation signal;

the output signal being in a range that is based on a charge stored in the charge storage, an input applied to the activation input, and an input signal received at the signal input;

the nonvolatile memory devices arranged in a two dimensional (XY) layer that has horizontal rows in a first dimension (X) and vertical columns in a second dimension (Y), the activation inputs of memory devices of each row connected to a same activation input signal, the memory devices of the columns having signal inputs connected to signal outputs of memory devices in the row above and having signal outputs connected to the signal inputs of memory devices in the row below.

29. The module of claim 28, wherein each row can be selected by the application of proper input to its activation input (WL) and the application of larger input to the activation inputs (WLs) of other rows.

30. The module of claim 28, wherein when the activation input of a row receives an activation signal, the memory devices operate in one of a linear mode or a subthreshold mode to cause the output signals from the memory devices in that row to be dot products of the state and an input signal received at each of the memory devices.

31. The module of claim 28, further comprising:

activation circuitry to provide the signals to bias the nonvolatile memory devices in a selected row in a read mode during operation so there would be no change in the charge storage of the nonvolatile memory devices and so that the nonvolatile memory devices operate in analog mode to have output signals which are dot products of the weight stored in the charge storage and the VMM module input signals; and

output circuitry to measure the VMM module output signals.

32. The module of claim 31, wherein the VMM module inputs are the different signal inputs of the nonvolatile memory devices in the first row along the first dimension (X) biased by input activation circuitry (called linear mode) and the output circuitry is connected to the signal outputs of the memory devices of the last row connected together along the first dimension (X);

or

wherein the VMM module output is a signal inputs of the memory devices of the first row connected together along the first dimension (X), input activation circuitry converts the VMM module input signals to voltage with an operational amplifier with a nonvolatile memory device feedback to bias the signal outputs of a last row of the memory devices connected together along the first dimension (X) (called source-coupled) and bias the activation inputs (WLs) of a nonvolatile memory devices in a selected row connected together along the first dimension (X) with a fixed voltage.

33. The module of claim 32, wherein for the case of source-coupled, the module outputs add the signal inputs of the nonvolatile memory devices of the first row along the first dimension (X).

34. The module of claim 32, wherein for the case of linear mode, the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of the selected row from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the first dimension (X) of that row.

35. The module of claim 28, wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storage based on the parameters of the neural network implemented based on the VMM module and trained with training dataset.

36. The module of claim 28, wherein the nonvolatile memory devices are floating gate transistors (FGTs);

the stored charges are floating gate charges;

the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs.