EFFICIENT AND SCALABLE SYSTEMS FOR CALCULATING NEURAL NETWORK CONNECTIVITY IN AN EVENT-DRIVEN WAY

• US 20190318224A1
• Filed: 06/26/2019
• Published: 10/17/2019
• Est. Priority Date: 10/01/2014
• Status: Active Application

• Current Assignee: Thalchemy Corporation
• Original Assignee: Thalchemy Corporation
• Inventors: Lipasti, Mikko H., Nere, Andrew, Hashmi, Atif, Wakerly, John F.
• Application Number:
  US16/453,876
• Publication Number:
  US 20190318224A1
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Systems and methods achieving scalable and efficient connectivity in neural algorithms by re-calculating network connectivity in an event-driven way are disclosed. The disclosed solution eliminates the storing of a massive amount of data relating to connectivity used in traditional methods. In one embodiment, a deterministic LFSR is used to quickly, efficiently, and cheaply re-calculate these connections on the fly. An alternative embodiment caches some or all of the LFSR seed values in memory to avoid sequencing the LFSR through all states needed to compute targets for a particular active neuron. Additionally, connections may be calculated in a way that generates neural networks with connections that are uniformly or normally (Gaussian) distributed.

Claims

1. A scalable system for recalculating, in an event-driven manner, property parameters including connectivity parameters of a neural network, the system comprises:

an input component that receives a time varying input signal;

a storage component for storing the property parameters of the neural network;

a state machine capable of recalculating property parameters of the neural network, wherein the property parameters include connectivity among neurons of the neural network; and

an output component that generates output signals reflective of the calculated property parameters of the neural network and the input signal.

2. The system of claim 1, wherein the state machine is capable of generating a unique identifying number for each neuron in the neural network.

3. The system of claim 1, wherein the state machine comprises a Linear Feedback Shift Register (LFSR).

4. The system of claim 3, wherein the LFSR is configured to generate certain property parameters including connectivity.

5. The system of claim 1, wherein the input component converts the received time varying input signal into a sequence of spikes.

6. The system of claim 1, wherein the state machine recalculates connectivity of a neuron currently being evaluated using a predefined initial value corresponding to the neuron currently being evaluated only when the neuron currently being evaluated fires in response to the input signal.

7. The system of claim 1, wherein the storage component includes a cache for storing predefined initial values used by the state machine for recalculating certain property parameters.

8. The system of claim 1, wherein the state machine is also used to generate a connection type for each neuron of the network.

9. The system of claim 1, wherein the property parameters further include neural delays of each neuron in the network.

10. The system of claim 1, further comprises:

a plurality of processing elements, wherein each processing element has a state machine and is capable of calculating property parameters of a subset of neurons of the neural network.

11. A computer-implemented method for recalculating network property parameters of a neural network including connectivity parameters in an event-driven manner, the method comprises:

initializing property parameters of the neural network;

receiving, at an evaluating neuron of the neural network, a neural input corresponding to a time varying input signal to the neural network;

recalculating by a state machine of the neural network at least some of the property parameters of the evaluating neuron, wherein the property parameters are random but determined after initialization;

determining whether the evaluating neuron is to generate a neural output to its target neurons in the neural network; and

if the evaluating neuron is determined to generate a neural output to its target neurons in the neural network, propagating the output of the evaluating neuron to its target neurons.

12. The method of claim 11, wherein the property parameters of the neural network include one or more parameters of the group consisting of maximum number of neurons in the neural network, one or more random number generation seed values, neural axonal and dendritic delay values, positive connectivity strength values, negative connectivity strength values, neuron refractory period, decay rate of neural membrane potential, neuron membrane potential, and neuron membrane leakage parameter.

13. The method of claim 11, wherein the determining comprises:

calculating current membrane potential of the evaluating neuron based on the neural input and the connectivity parameters of the evaluating neuron;

comparing the calculated membrane potential to a firing threshold value of the evaluating neuron; and

reporting that the evaluating neuron is to generate an output if the calculated membrane potential exceeds the firing threshold value.

14. The method of claim 11, wherein the recalculating comprises:

using a pseudo-random number generator with a pre-defined start value to calculate the property parameters.

15. The method of claim 14, wherein the pseudo-random number generator comprises a Linear Feedback Shift Register (LFSR).

16. The method of claim 11, wherein the recalculating comprises:

retrieving a stored pre-defined initial value corresponding to the evaluating neuron; and

calculating connectivity parameters of the evaluating neuron using the retrieved seed value.

17. The method of claim 11, wherein the recalculating comprises calculating connectivity of a neuron currently being evaluated only when the neuron currently being evaluated fires in response to the input signal.

18. The method of claim 11, further comprises retrieving an initial value of the state machine for recalculating the connectivity parameters of the evaluating neuron from a cache coupled to the state machine.

19. The method of claim 11, further comprises:

maintaining a list of future firing neurons, wherein the updating at each time step is conducted only on neurons identified on the list;

for each target neuron of a neuron that fires at a current time step, comparing the current membrane potential of that target neuron to a corresponding predefined firing threshold of that target neuron;

adding an identity of a target neuron to the list of future firing neurons if the current membrane potential of that target neuron exceeds the corresponding predefined firing threshold; and

removing an identity of a target neuron from the list of future firing neurons if the current membrane potential of that target neuron is below the corresponding predefined firing threshold.

20. The method of claim 11, further comprises:

taking sum of the state machine results to form a uniform distribution; and

adding an offset to center the normalized distribution in calculating addresses of the target neurons.