Storing neural net works and weights for neural networks

• US 11,436,482 B2
• Filed: 10/29/2018
• Issued: 09/06/2022
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Qin, Minghai
• Primary Examiner(s): Geib, Benjamin P
• Application Number:
  US16/174,014
• Publication Number:
  US 20200134443A1
• Time in Patent Office: 1,408 Days
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/065 Analogue means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/10 Interfaces, programming languages or software development kits, e.g. for simulating neural networks

Abstract

Systems and methods are disclosed for storing neural networks and weights for neural networks. In some implementations, a method is provided. The method includes storing a plurality of weights of a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. Each weight of the plurality of weights is associated with a connection of the plurality of connections. The neural network comprises a binarized neural network. The method also includes receiving input data to be processed by the neural network. The method further includes determining whether a set of weights of the plurality of weights comprises one or more errors. The method further includes refraining from using the set of weights to process the input data using the neural network in response to determining that the set of weights comprises the one or more errors.

Claims

1. A method, comprising:

storing a plurality of weights of a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes, wherein;

each weight of at least some of the plurality of weights is associated with a connection of the plurality of connections; and

the neural network comprises a binarized neural network;

receiving input data to be processed by the neural network;

performing a parity check to determine whether there is one or more errors in one or more weights in a set of weights of the plurality of weights, wherein the set of weights comprises multiple weights, and the multiple weights share the parity check; and

selectively refraining from using the set of weights to process the input data using the neural network in response to determining that the set of weights comprises the one or more errors.

2. The method of claim 1, further comprising:

using the set of weights to process the input data using the neural network in response to determining that the set of weights does not comprises the one or more errors.

3. The method of claim 1, wherein each weight of the plurality of weights comprises one of two values.

4. The method of claim 1, wherein:

the set of weights are associated with a filter; and

the filter is configured to generate a feature map.

5. The method of claim 4, wherein refraining from using the set of weights comprises:

refraining from generating the feature map.

6. The method of claim 4, wherein refraining from using the set of weights comprises:

ignoring the feature map generated by the filter.

7. The method of claim 1, wherein determining whether the set of weights of the plurality of weights comprises the one or more errors is based on a single parity check.

8. The method of claim 7, wherein:

the set of weights comprises multiple weights; and

the single parity check is shared among the multiple weights.

9. The method of claim 1, further comprising:

constructing the neural network based on a structure of the neural network, wherein the structure of the neural network indicates interconnections between the plurality of nodes.

10. The method of claim 1, further comprising:

receiving training data; and

training the neural network based on the training data, wherein the plurality of weights are generated during the training of the neural network.

11. The method of claim 1, wherein the plurality of weights are stored on a resistive non-volatile memory.

12. A device, comprising:

a network interface; and

a processing device configured to;

receive, via the network interface, a plurality of weights of a neural network, wherein;

the neural network comprises a plurality of nodes and a plurality of connections between the plurality of nodes;

each weight of at least some of the plurality of weights is associated with a connection of the plurality of connections; and

each weight of at least some of the plurality of weights comprises one of two values;

perform a parity check to determine whether there is one or more errors in one or more weights in a set of weights of the plurality of weights, wherein the set of weights comprises multiple weights, and the multiple weights share the parity check; and

selectively refrain from using the set of weights to process input data using the neural network in response to determining that the set of weights comprises the one or more errors.

13. The device of claim 12, wherein the processing device is further configured to:

use the set of weights to process the input data using the neural network in response to determining that the set of weights does not comprises the one or more errors.

14. The device of claim 12, wherein the neural network comprises a binarized neural network.

15. The device of claim 12, wherein:

the set of weights are associated with a filter; and

the filter is configured to generate a feature map.

16. The device of claim 15, wherein the processing device is further configured to, in refraining from using the set of weights:

refrain from generating the feature map.

17. The device of claim 15, wherein the processing device is further configured to, in refraining from using the set of weights:

ignore the feature map generated by the filter.

18. The device of claim 12, wherein determining whether the set of weights of the plurality of weights comprises the one or more errors is based on a single parity check.

19. The device of claim 18, wherein:

the set of weights comprises multiple weights; and

the single parity check is shared among the multiple weights.

20. An apparatus, comprising:

a processing device configured to;

store a plurality of weights of a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes, wherein;

each weight of the plurality of weights is associated with a connection of the plurality of connections; and

the neural network comprises a binarized neural network;

determine whether a set of weights of the plurality of weights comprises one or more errors based on a parity check, wherein;

the set of weights comprises multiple weights; and

the multiple weights share the parity check; and

selectively refrain from using the set of weights to process input data using the neural network in response to determining that the set of weights comprises the one or more errors.