Unsupervised, supervised and reinforced learning via spiking computation

• US 9,245,223 B2
• Filed: 08/16/2012
• Issued: 01/26/2016
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Modha, Dharmendra S.
• Primary Examiner(s): Chang, Li-Wu
• Application Number:
  US13/587,424
• Publication Number:
  US 20140250039A1
• Time in Patent Office: 1,258 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

The present invention relates to unsupervised, supervised and reinforced learning via spiking computation. The neural network comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of edges interconnects the plurality of neural modules. Each edge interconnects a first neural module to a second neural module, and each edge comprises a weighted synaptic connection between every neuron in the first neural module and a corresponding neuron in the second neural module.

Claims

1. A method comprising:

controlling spiking in a neural network comprising a plurality of neural modules interconnected via a plurality of weighted synaptic connections in an interconnection network, wherein each neural module comprises multiple digital neurons, and each neuron in each neural module is connected to a corresponding neuron in another neural module via a weighted synaptic connection; and

in response to a neuron in a neural module spiking, updating a synaptic weight associated with a weighted synaptic connection connected to the neuron by selectively applying one of a first learning rule for learning false negatives and a second learning rule for unlearning false positives;

wherein controlling the spiking in the neural network comprises;

generating signals that define a set of time steps for operation of each neuron in each neural module; and

at each time step, for each neuron in each neural module, selectively generating an outgoing firing event based on an operational state of the neuron in response to receiving an incoming firing event as an input signal from a corresponding neuron in another neural module via a weighted synaptic connection connected to the neuron, wherein the input signal is weighted by a synaptic weight associated with the weighted synaptic connection.

2. The method of claim 1, further comprising:

at each time step, for each neuron in each neural module, updating an operational state of the neuron; and

adapting a synaptic weight associated with a weighted synaptic connection as a function of one or more firing events generated by one or more interconnected neurons.

3. The method of claim 2, further comprising:

updating a learning rule associated with a weighted synaptic connection, wherein the weighted synaptic connection is reconfigurable for one or more of the following learning rules;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP.

4. The method of claim 2, further comprising:

applying the first learning rule for learning false negatives to a weighted synaptic connection when a neuron interconnected via the weighted synaptic connection generates an outgoing firing event.

5. The method of claim 4, further comprising:

applying the second learning rule for unlearning false positives to a weighted synaptic connection when a neuron interconnected via the weighted synaptic connection generates an outgoing firing event.

6. The method of claim 5, further comprising:

for at least one neuron in at least one neural module, receiving one or more input events from an input interface to the neural network; and

for at least one neuron in at least one neural module, generating and sending one or more output events to an output interface of the neural network.

7. The method of claim 5, further comprising:

for at least one neuron in at least one neural module, generating and sending one or more output events to an evaluation module in the neural network, wherein, based on a type of the one or more output events, the one or more output events are fed to at least one other neuron in at least one other neural module by the evaluation module.

8. The method of claim 5, further comprising:

updating one or more learning rules associated with one or more weighted synaptic connections such that the neural network operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.

9. The method of claim 5, further comprising:

adaptively updating one or more synaptic weights associated with one or more weighted synaptic connections in the interconnection network as a function one or more firing events of one or more interconnected neurons.

10. The method of claim 5, wherein:

for a first set of neurons in a first neural module of the plurality of neural modules, information flows from each neuron in the first set to a corresponding neuron in a second neural module of the plurality of neural modules; and

for a second set of neurons in the second neural module, information flows from each neuron in the second set to a corresponding neuron in the first neural module.

11. A system comprising a computer processor, a computer-readable hardware storage device, and program code embodied with the computer-readable hardware storage device for execution by the computer processor to implement a method comprising:

controlling spiking in a neural network comprising a plurality of neural modules interconnected via a plurality of weighted synaptic connections in an interconnection network, wherein each neural module comprises multiple digital neurons, and each neuron in each neural module is connected to a corresponding neuron in another neural module via a weighted synaptic connection; and

in response to a neuron in a neural module spiking, updating a synaptic weight associated with a weighted synaptic connection connected to the neuron by selectively applying one of a first learning rule for learning false negatives and a second learning rule for unlearning false positives;

wherein controlling the spiking in the neural network comprises;

generating signals that define a set of time steps for operation of each neuron in each neural module; and

at each time step, for each neuron in each neural module, selectively generating an outgoing firing event based on an operational state of the neuron in response to receiving an incoming firing event as an input signal from a corresponding neuron in another neural module via a weighted synaptic connection connected to the neuron, wherein the input signal is weighted by a synaptic weight associated with the weighted synaptic connection.

12. The system of claim 11, the method further comprising:

at each time step, for each neuron in each neural module, updating an operational state of the neuron; and

adapting a synaptic weight associated with a weighted synaptic connection as a function of one or more firing events generated by one or more interconnected neurons.

13. The system of claim 12, the method further comprising:

updating a learning rule associated with a weighted synaptic connection, wherein the weighted synaptic connection is reconfigurable for one or more of the following learning rules;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP.

14. The system of claim 12, the method further comprising:

applying the first learning rule for learning false negatives to a weighted synaptic connection when a neuron interconnected via the weighted synaptic connection generates an outgoing firing event.

15. The system of claim 14, the method further comprising:

applying the second learning rule for unlearning false positives to a weighted synaptic connection when a neuron interconnected via the weighted synaptic connection generates an outgoing firing event.

16. The system of claim 15, the method further comprising:

for at least one neuron in at least one neural module, receiving one or more input events from an input interface to the neural network; and

for at least one neuron in at least one neural module, generating and sending one or more output events to an output interface of the neural network.

17. The system of claim 15, the method further comprising:

for at least one neuron in at least one neural module, generating and sending one or more output events to an evaluation module in the neural network, wherein, based on a type of the one or more output events, the one or more output events are fed to at least one other neuron in at least one other neural module by the evaluation module.

18. The system of claim 15, the method further comprising:

updating one or more learning rules associated with one or more weighted synaptic connections such that the neural network operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.

19. The system of claim 15, the method further comprising:

adaptively updating one or more synaptic weights associated with one or more weighted synaptic connections in the interconnection network as a function of one or more firing events of one or more interconnected neurons.

20. A computer program product comprising a computer-readable hardware storage device having program code embodied therewith, the program code being executable by a computer to implement a method comprising:

controlling spiking in a neural network comprising a plurality of neural modules interconnected via a plurality of weighted synaptic connections in an interconnection network, wherein each neural module comprises multiple digital neurons, and each neuron in each neural module is connected to a corresponding neuron in another neural module via a weighted synaptic connection; and

in response to a neuron in a neural module spiking, updating a synaptic weight associated with a weighted synaptic connection connected to the neuron by selectively applying one of a first learning rule for learning false negatives and a second learning rule for unlearning false positives;

wherein controlling the spiking in the neural network comprises;

generating signals that define a set of time steps for operation of each neuron in each neural module; and

at each time step, for each neuron in each neural module, selectively generating an outgoing firing event based on an operational state of the neuron in response to receiving an incoming firing event as an input signal from a corresponding neuron in another neural module via a weighted synaptic connection connected to the neuron, wherein the input signal is weighted by a synaptic weight associated with the weighted synaptic connection.