Vector-by-matrix multiplier modules based on non-volatile 2D and 3D memory arrays

• US 10,552,510 B2
• Filed: 01/11/2019
• Issued: 02/04/2020
• Est. Priority Date: 01/11/2018
• Status: Active Grant

• Current Assignee: Mentium Technologies Inc.
• Original Assignee: Mentium Technologies Inc.
• Inventors: Bayat, Farnood Merrikh, Prezioso, Mirko
• Primary Examiner(s): Lebouef, Jerome
• Application Number:
  US16/246,353
• Publication Number:
  US 20190213234A1
• Time in Patent Office: 389 Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/065 Analogue means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G11C 11/54 using elements simulating biological cells, e.g. neuron
• CPC Class Codes: G11C 16/0425 comprising cells containing a merged floating gate and select transistor
• CPC Class Codes: G11C 16/0466 comprising cells with charge storage in an insulating layer, e.g. metal-nitride-oxide-silicon [MNOS], silicon-oxide-nitride-oxide-silicon [SONOS] G11C16/0483, G11C16/0491 take precedence
• CPC Class Codes: G11C 16/0483 comprising cells having several storage transistors connected in series
• CPC Class Codes: G11C 16/10 Programming or data input circuits
• CPC Class Codes: G11C 16/26 Sensing or reading circuits; Data output circuits
• CPC Class Codes: G11C 7/1006 Data managing, e.g. manipulating data before writing or reading out, data bus switches or control circuits therefor

Abstract

Systems and methods for a vector-by-matrix multiplier (VMM) module having a three-dimensional memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input and an output signal in a range that is based on a stored charge and an input signal during assertion of the activation signal. The memory devices are arranged in two dimensional (XY) layers that are vertically disposed along (Z) columns. The activation inputs of each layer are connected to a same activation signal, the memory devices of rows in a first dimension (X) of each layer have signal inputs connected to different input signals and have signal outputs connected in series to a common output. The memory devices of rows in a second dimension (Y) of each layer have signal inputs connected to a set of the same inputs along the first dimension.

Claims

1. A vector-by-matrix multiplier (VMM) module comprising:

a three-dimensional (3D) memory matrix of nonvolatile memory devices each having a charge storage, an activation input, a signal input to receive an input signal and signal output to output an output signal when the activation input receives an activation signal,

the output signal being in a range that is based on a charge stored in the charge storage, the activation signal applied to the activation input, and the input signal received at the signal input;

the nonvolatile memory devices arranged in a plurality of two dimensional (2D) (XY) layers that are vertically disposed along a plurality of (Z) columns, the activation inputs of memory devices of each layer connected to a same activation input signal, the memory devices of each layer having signal inputs connected to signal outputs of memory devices in the layer above along a third dimension (Z) and having signal outputs connected to the signal inputs of memory devices in the layer below along the third dimension (Z), wherein;

VMM module inputs are signal inputs of a top 2D layer of the memory devices connected together along the second dimension (Y);

VMM module outputs are signal outputs of a bottom 2D layer of the memory devices connected together along the first dimension (X); and

each two-dimensional (2D) (XY) layer that is vertically disposed along a plurality of (Z) columns can be selected by the application of a proper input to the same activation input of that layer and the application of a larger input to the same activation inputs of other layers.

2. The module of claim 1, wherein when the same activation input of a selected layer receives the activation signal, the VMM module outputs of the bottom layer output signals that are the vector-by-matrix multiplication of weights stored in the nonvolatile memory devices of the selected layer which are proportional to the charges stored in their charge storages and the input signals received at the VMM module inputs of the top layer.

3. The module of claim 1, wherein the module outputs add the signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of bottom layer;

and

wherein the output circuitry subtracts the added signal outputs of every other column of the nonvolatile memory devices of rows in the first dimension (X) of the bottom layer from the added signal outputs of the prior column of the nonvolatile memory devices of rows in the first dimension (X) of that layer.

4. The module of claim 3, wherein the VMM module inputs are drain terminals of a top 2D layer of the memory devices connected together along bit lines (BLs) along the second dimension (Y), and the VMM module outputs are source terminals of a bottom 2D layer of the memory devices connected together along source lines (SLs) along the first dimension (X).

5. The module of claim 3, further comprising:

activation circuitry to provide the activation signal to bias the FGTs in a read mode so there would be no change in the charge storage of the FGTs and so that the FGTs operate in analog mode to have output signals in a linear range with respect to the dot products of the weight stored in the charge storage and the input signal received at the signal input of each of the memory devices.

6. The module of claim 1, further comprising:

input circuitry to input each of a set of input signals into corresponding ones of the set of the same VMM module inputs along the first dimension (X) of memory devices; and

the output circuitry to connect the VMM module signal outputs connected in series of the rows of the memory devices in the first dimension (X) into a neural path outputs of a neural network.

7. The module of claim 1, wherein the nonvolatile memory devices are floating gate transistors (FGTs);

the stored charges are floating gate charges;

the input signals are drain inputs, the output signals are source outputs and the activation inputs are gate or world-line inputs.

8. The module of claim 1, wherein the VMM module further comprises programming circuitry to write the charges stored into the charge storages based on parameters of a neural network implemented using the VMM module and trained with a training dataset.