EVENT-DRIVEN UNIVERSAL NEURAL NETWORK CIRCUIT

• US 20150371131A1
• Filed: 08/24/2012
• Published: 12/24/2015
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Modha, Dharmendra S.
• Granted Patent: US 9,292,788 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

The present invention provides an event-driven universal neural network circuit. The circuit comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of digital synapses interconnects the neural modules. Each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module. Corresponding neurons in the first neural module and the second neural module communicate via the synapses. Each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module. A control module generates signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnection network.

Claims

1. :

A method comprising;

performing event-driven spiking computation in a neural network circuit comprising a plurality of neural modules, wherein each neural module includes multiple digital neurons, each neuron in each neural module is connected to a corresponding neuron in an other neural module via a synapse of an interconnect network of synapses, and the synapse has a learning rule associating the neuron in the neural module with the corresponding neuron in the other neural module;

wherein the event-driven spiking computation comprises;

generating signals that define a set of time steps for event-driven operation of each neuron and event communication via the interconnect network of synapses; and

at each time step, for each neuron, updating an operational state of the neuron and, based on the operational state, determining whether to generate an outgoing firing event in response to receiving an incoming firing event as an input signal from a corresponding neuron in an other neural module via a synapse connecting the neuron to the corresponding neuron, wherein the input signal is weighted by a synaptic weight of the synapse.

2. :

The method of claim 1, further comprising;

adapting a synaptic weight of a synapse of the interconnect network of synapses as a function of one or more outgoing firing events generated by one or more interconnected neurons.

3. :

The method of claim 2, further comprising;

updating a learning rule for a synapse of the interconnect network of synapses, wherein the synapse is reconfigurable for one or more of the following learning rules;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP;

applying a first learning rule for learning false negatives to a first synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron; and

applying a second learning rule for unlearning false positives to a second synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron.

4. :

The method of claim 3, further comprising;

a first set of neurons in the neural network circuit receiving one or more input events from an input interface to the neural network circuit;

a second set of neurons in the neural network circuit generating and sending one or more output events to an output interface of the neural network circuit; and

a third set of neurons in the neural network circuit generating and sending one or more output events to an evaluation module of the neural network circuit, wherein, based on a type of the one or more output events, the one or more output events are fed to a fourth set of neurons in the neural network circuit.

5. :

The method of claim 3, further comprising;

updating one or more learning rules for one or more synapses of the interconnect network of synapses, such that the neural network circuit operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.

6. :

A system comprising a computer processor, a computer-readable hardware storage device, and program code embodied with the computer-readable hardware storage device for execution by the computer processor to implement a method comprising;

performing event-driven spiking computation in a neural network circuit comprising a plurality of neural modules, wherein each neural module includes multiple digital neurons, each neuron in each neural module is connected to a corresponding neuron in an other neural module via a synapse of an interconnect network of synapses, and the synapse has a learning rule associating the neuron in the neural module with the corresponding neuron in the other neural module;

wherein the event-driven spiking computation comprises;

generating signals that define a set of time steps for event-driven operation of each neuron and event communication via the interconnect network of synapses; and

at each time step, for each neuron, updating an operational state of the neuron and, based on the operational state, determining whether to generate an outgoing firing event in response to receiving an incoming firing event as an input signal from a corresponding neuron in an other neural module via a synapse connecting the neuron to the corresponding neuron, wherein the input signal is weighted by a synaptic weight of the synapse.

7. :

The system of claim 6, further comprising;

adapting a synaptic weight of a synapse of the interconnect network of synapses as a function of one or more outgoing firing events generated by one or more interconnected neurons.

8. :

The system of claim 7, further comprising;

updating a learning rule for a synapse of the interconnect network of synapses, wherein the synapse is reconfigurable for one or more of the following learning rules;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP;

applying a first learning rule for learning false negatives to a first synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron; and

applying a second learning rule for unlearning false positives to a second synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron.

9. :

The system of claim 8, further comprising;

a first set of neurons in the neural network circuit receiving one or more input events from an input interface to the neural network circuit;

a second set of neurons in the neural network circuit generating and sending one or more output events to an output interface of the neural network circuit; and

a third set of neurons in the neural network circuit generating and sending one or more output events to an evaluation module of the neural network circuit, wherein, based on a type of the one or more output events, the one or more output events are fed to a fourth set of neurons in the neural network circuit.

10. :

The system of claim 9, further comprising;

updating one or more learning rules for one or more synapses of the interconnect network of synapses, such that the neural network circuit operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.

11. :

A computer program product comprising a computer-readable hardware storage device having program code embodied therewith, the program code being executable by a computer to implement a method comprising;

performing event-driven spiking computation in a neural network circuit comprising a plurality of neural modules, wherein each neural module includes multiple digital neurons, each neuron in each neural module is connected to a corresponding neuron in an other neural module via a synapse of an interconnect network of synapses, and the synapse has a learning rule associating the neuron in the neural module with the corresponding neuron in the other neural module;

wherein the event-driven spiking computation comprises;

generating signals that define a set of time steps for event-driven operation of each neuron and event communication via the interconnect network of synapses; and

at each time step, for each neuron, updating an operational state of the neuron and, based on the operational state, determining whether to generate an outgoing firing event in response to receiving an incoming firing event as an input signal from a corresponding neuron in an other neural module via a synapse connecting the neuron to the corresponding neuron, wherein the input signal is weighted by a synaptic weight of the synapse.

12. :

The computer program product of claim 11, further comprising;

adapting a synaptic weight of a synapse of the interconnect network of synapses as a function of one or more outgoing firing events generated by one or more interconnected neurons.

13. :

The computer program product of claim 12, further comprising;

updating a learning rule for a synapse of the interconnect network of synapses, wherein the synapse is reconfigurable for one or more of the following learning rules;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP;

applying a first learning rule for learning false negatives to a first synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron; and

applying a second learning rule for unlearning false positives to a second synapse of the interconnect network of synapses in response to an outgoing firing event generated by an interconnected neuron.

14. :

The computer program product of claim 13, further comprising;

a first set of neurons in the neural network circuit receiving one or more input events from an input interface to the neural network circuit;

a second set of neurons in the neural network circuit generating and sending one or more output events to an output interface of the neural network circuit; and

a third set of neurons in the neural network circuit generating and sending one or more output events to an evaluation module of the neural network circuit, wherein, based on a type of the one or more output events, the one or more output events are fed to a fourth set of neurons in the neural network circuit.

15. :

The computer program product of claim 14, further comprising;

updating one or more learning rules for one or more synapses of the interconnect network of synapses, such that the neural network circuit operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.