Storage of neural networks

• US 10,552,251 B2
• Filed: 12/12/2017
• Issued: 02/04/2020
• Est. Priority Date: 09/06/2017
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Qin, Minghai, Vucinic, Dejan, Sun, Chao
• Primary Examiner(s): Leibovich, Yair
• Application Number:
  US15/839,521
• Publication Number:
  US 20190073259A1
• Time in Patent Office: 784 Days
• Field of Search: 714 25
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G11C 13/0002 using resistive RAM [RRAM] elements
• CPC Class Codes: G11C 13/0004 comprising amorphous/crystalline phase transition cells
• CPC Class Codes: G11C 2029/0411 Online error correction
• CPC Class Codes: G11C 29/70 Masking faults in memories by using spares or by reconfiguring

Abstract

Disclosed include a device and a method for storing a neural network. The device includes a plurality of memory cells configured to store weights of the neural network. The plurality of memory cells may include one or more faulty cells. The device further includes a processor coupled to the plurality of memory cells. The processor is configured to construct the neural network based on a structure of the neural network and a subset of the weights stored by the plurality of memory cells. The subset of the weights may exclude another subset of the weights stored by one or more memory cells comprising the one or more faulty cells.

Claims

1. A device comprising:

a plurality of memory cells configured to store weights of a neural network, the plurality of memory cells comprising one or more faulty cells; and

a processor coupled to the plurality of memory cells, the processor configured to;

construct a neural network based on a stored structure of the neural network and a subset of the weights stored by a subset of the plurality of memory cells, without another subset of the weights stored by one or more memory cells comprising the one or more faulty cells.

2. The device of claim 1, wherein the plurality of memory cells comprise one of resistive memory cells and phase change memory cells.

3. The device of claim 1, wherein the one or more faulty cells comprise one or more stuck-at-fault cells.

4. The device of claim 1, wherein the processor is configured to:

perform a cyclic redundancy check on the plurality of memory cells, and

detect the one or more faulty cells from the plurality of memory cells based on the cyclic redundancy check.

5. The device of claim 1, wherein the processor is configured to:

construct the neural network by substituting zero for the another subset of the weights stored by the one or more memory cells comprising the one or more faulty cells.

6. The device of claim 5, wherein the processor is configured to:

construct the neural network by substituting the zero for the another subset of the weights stored by the one or more memory cells, in response to determining that a ratio between a number of the one or more faulty cells and a number of the plurality of memory cells is less than a predetermined threshold.

7. The device of claim 1, wherein the processor is configured to:

retrain the weights stored by the plurality of memory cells, in response to detecting the one or more faulty cells.

8. The device of claim 1, wherein the processor is configured to:

retrain the another subset of the weights stored by the one or more memory cells and bypass retraining the subset of the weights stored by the subset of the plurality of memory cells, in response to detecting the one or more faulty cells.

9. The device of claim 1, wherein the plurality of memory cells are non-erasable.

10. The device of claim 1, wherein the processor is further configured to:

receive input data indicating an image, audio, or a combination of the image and the audio, and

determine a feature in the input data according to the neural network constructed based on the stored structure of the neural network and the subset of the weights stored by the subset of the plurality of memory cells.

11. The device of claim 1, wherein the stored structure of the neural network is stored by a first memory device, and the weights are stored by the plurality of memory cells of a second memory device different from the first memory device.

12. The device of claim 1, wherein the processor is configured to bypass error correction code processing while constructing the neural network without the another subset of the weights stored by the one or more memory cells comprising the one or more faulty cells.

13. A device comprising:

a plurality of memory cells configured to store weights of a neural network, each weight stored by a corresponding vector of memory cells of the plurality of memory cells, each vector associated with a corresponding indicator indicative of whether the vector stores an incorrect weight; and

a processor coupled to the plurality of memory cells, the processor configured to;

detect a vector of memory cells of the plurality of memory cells storing the incorrect weight based on the corresponding indicator associated with the detected vector,

substitute a default value for the incorrect weight stored by the detected vector, and

construct the neural network based on the substituted default value.

14. The device of claim 13, wherein each indicator comprises a single bit.

15. The device of claim 13, wherein the processor is further configured to:

perform a cyclic redundancy check on the detected vector of the memory cells and the corresponding indicator associated with the detected vector; and

detect whether the detected vector of the memory cells stores the incorrect weight based on the cyclic redundancy check.

16. The device of claim 13, wherein each indicator indicates whether a corresponding cell is a stuck-at-fault cell.

17. The device of claim 13, wherein the plurality of memory cells are memory cells of a resistive non-volatile memory device.

18. The device of claim 13, wherein the processor is configured to:

construct the neural network by substituting the default value, in response to determining that a ratio between a number of memory cells storing incorrect weights and a number of the plurality of memory cells is less than a predetermined threshold.

19. A method comprising:

detecting a subset of a plurality of memory cells, the subset of the plurality of memory cells including stuck-at-fault cells;

substituting a default value for a set of weights of a neural network, the set of the weights stored at least in part on the stuck-at-fault cells; and

constructing the neural network based on a stored structure of the neural network, another set of the weights of the neural network stored by another subset of the plurality of memory cells, and the substituted default value.

20. The method of claim 19, further comprising:

substituting the default value for the set of the weights, in response to determining that a ratio between a number of the stuck-at-fault cells and a number of the plurality of memory cells is less than a predetermined threshold.

21. A device comprising:

means for detecting a subset of a plurality of memory cells, the subset of the plurality of memory cells including stuck-at-fault cells;

means for substituting a default value for a set of weights of a neural network, the set of the weights stored by the stuck-at-fault cells; and

means for constructing the neural network based on a stored structure of the neural network, another set of the weights of the neural network stored by another subset of the plurality of memory cells, and the substituted default value.