Unsupervised, supervised and reinforced learning via spiking computation

• US 10,445,642 B2
• Filed: 05/23/2016
• Issued: 10/15/2019
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Modha, Dharmendra S.
• Primary Examiner(s): Chang, Li Wu
• Application Number:
  US15/162,361
• Publication Number:
  US 20170213133A1
• Time in Patent Office: 1,240 Days
• Field of Search: None
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

The present invention relates to unsupervised, supervised and reinforced learning via spiking computation. The neural network comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of edges interconnects the plurality of neural modules. Each edge interconnects a first neural module to a second neural module, and each edge comprises a weighted synaptic connection between every neuron in the first neural module and a corresponding neuron in the second neural module.

Claims

1. A method, comprising:

training a neural network for reinforcement learning by;

receiving output comprising firing events generated by a neuron population of the neural network;

determining a type of the output; and

propagating the output through one or more other neural populations of the neural network based on the type of the output, wherein the output is copied and propagated through a first set of neuron populations for the neural network to learn the output in response to determining the type of the output is a first type, and the output is propagated through a second set of neuron populations different from the first set of populations for the neural network to unlearn the output in response to determining the type of the output is a second type different from the first type.

2. The method of claim 1,

wherein the first type represents positive output including false negatives for the neural network to learn, and the second type represents negative output including false positives for the neural network to unlearn.

3. The method of claim 2, wherein the propagating comprises:

in response to determining the type of the output is the first type, providing a copy of the output to the first set of neuron populations, wherein the output is learned by the neural network as the copy of the output propagates through the first set of neuron populations via a first set of pathways.

4. The method of claim 3, wherein the propagating further comprises:

in response to determining the type of the output is the first type, applying Hebb learning based on firing activity of the first set of neuron populations to adjust synaptic weights of the first set of pathways.

5. The method of claim 2, wherein the propagating comprises:

in response to determining the type of the output is the second type, providing a copy of the output to the second set of neuron populations, wherein the output is unlearned by the neural network as the copy of the output propagates through the second set of neuron populations via a second set of pathways.

6. The method of claim 5, wherein the propagating further comprises:

in response to determining the type of the output is the second type, applying anti-Hebb learning based on firing activity of the second set of neuron populations to adjust synaptic weights of the second set of pathways.

7. A system comprising a computer processor, a computer-readable hardware storage medium, and program code embodied with the computer-readable hardware storage medium for execution by the computer processor to implement a method comprising:

training a neural network for reinforcement learning by;

receiving output comprising firing events generated by a neuron population of the neural network;

determining a type of the output; and

propagating the output through one or more other neural populations of the neural network based on the type of the output, wherein the output is copied and propagated through a first set of neuron populations for the neural network to learn the output in response to determining the type of the output is a first type, and the output is propagated through a second set of neuron populations different from the first set of populations for the neural network to unlearn the output in response to determining the type of the output is a second type different from the first type.

8. The system of claim 7,

wherein the first type represents positive output including false negatives for the neural network to learn, and the second type represents negative output including false positives for the neural network to unlearn.

9. The system of claim 8, wherein the propagating comprises:

in response to determining the type of the output is the first type, providing a copy of the output to the first set of neuron populations, wherein the output is learned by the neural network as the copy of the output propagates through the first set of neuron populations via a first set of pathways.

10. The system of claim 9, wherein the propagating further comprises:

in response to determining the type of the output is the first type, applying Hebb learning based on firing activity of the first set of neuron populations to adjust synaptic weights of the first set of pathways.

11. The system of claim 8, wherein the propagating comprises:

in response to determining the type of the output is the second type, providing a copy of the output to the second set of neuron populations, wherein the output is unlearned by the neural network as the copy of the output propagates through the second set of neuron populations via a second set of pathways.

12. The system of claim 11, wherein the propagating further comprises:

in response to determining the type of the output is the second type, applying anti-Hebb learning based on firing activity of the second set of neuron populations to adjust synaptic weights of the second set of pathways.

13. A computer program product comprising a computer-readable hardware storage device having program code embodied therewith, the program code being executable by a computer to implement a method comprising:

training a neural network for reinforcement learning by;

receiving output comprising firing events generated by a neuron population of the neural network;

determining a type of the output; and

propagating the output through one or more other neural populations of the neural network based on the type of the output, wherein the output is copied and propagated through a first set of neuron populations for the neural network to learn the output in response to determining the type of the output is a first type, and the output is propagated through a second set of neuron populations different from the first set of populations for the neural network to unlearn the output in response to determining the type of the output is a second type different from the first type.

14. The computer program product of claim 13,

wherein the first type represents positive output including false negatives for the neural network to learn, and the second type represents negative output including false positives for the neural network to unlearn.

15. The computer program product of claim 14, wherein the propagating comprises:

in response to determining the type of the output is the first type, providing a copy of the output to the first set of neuron populations, wherein the output is learned by the neural network as the copy of the output propagates through the first set of neuron populations via a first set of pathways.

16. The computer program product of claim 15, wherein the propagating further comprises:

in response to determining the type of the output is the first type, applying Hebb learning based on firing activity of the first set of neuron populations to adjust synaptic weights of the first set of pathways.

17. The computer program product of claim 14, wherein the propagating comprises:

in response to determining the type of the output is the second type, providing a copy of the output to the second set of neuron populations, wherein the output is unlearned by the neural network as the copy of the output propagates through the second set of neuron populations via a second set of pathways.

18. The computer program product of claim 17, wherein the propagating further comprises:

in response to determining the type of the output is the second type, applying anti-Hebb learning based on firing activity of the second set of neuron populations to adjust synaptic weights of the second set of pathways.