EVENT-DRIVEN UNIVERSAL NEURAL NETWORK CIRCUIT

• US 20130073494A1
• Filed: 09/16/2011
• Published: 03/21/2013
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Modha, Dharmendra S.
• Granted Patent: US 8,738,554 B2
• Time in Patent Office: Days
• US Class Current: 706/25
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

The present invention provides an event-driven universal neural network circuit. The circuit comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of digital synapses interconnects the neural modules. Each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module. Corresponding neurons in the first neural module and the second neural module communicate via the synapses. Each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module. A control module generates signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnection network.

Claims

1. A neural network circuit, comprising:

a plurality of neural modules, wherein each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module; and

an interconnection network comprising a plurality of digital synapses that interconnect the plurality of neural modules, wherein each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module such that corresponding neurons in the first neural module and the second neural module communicate via the synapses, wherein each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module; and

a control module that generates signals which define a set of time steps for event driven operation of the neurons and event communication via the interconnection network.

2. The neural network circuit of claim 1, wherein each synapse has a weight, a delay, and a synapse type, wherein each synapse type comprises a different set of learning rules.

3. The neural network circuit of claim 2, wherein:

each neuron in a neural module has an operational state; and

based on its operational state, said neuron generates a firing event in response to one or more firing events received as input signals from corresponding neurons in other neural modules via connected synapses, wherein each said input signal is weighted by said weight of the synapse communicating said input signal to said neuron.

4. The neural network circuit of claim 3, wherein at each time step:

each neuron in each neural module updates its operational state and computes whether to generate a firing event; and

each neural module generates a firing event vector indicating which of its neurons generated a firing event, and transmits the firing event vector via a connected synapse to communicate the firing activity to one or more neurons via the interconnection network.

5. The neural network circuit of claim 4, wherein:

a synapse interconnecting the first neural module and the second neural module receives;

a first firing event vector from the first neural module indicating which neurons in the first neural module generated a firing event; and

a second firing event vector from the second neural module indicating which neurons in the second neural module generated a firing event.

6. The neural network circuit of claim 5, wherein:

in response to the first vector, said synapse forwards the first vector to the second neural module and applies learning rules associating each firing neuron in the first neural module with a corresponding neuron in the second neural module; and

in response to the second vector, said synapse forwards the second vector to the first neural module and applies learning rules associating each firing neuron in the second neural module with its corresponding neuron in the first neural module.

7. The neural network circuit of claim 6, wherein:

in response to the first vector, said synapse further forwards to the second neural module said weight, said delay, and a previously forwarded weight of said synapse;

in response to the second vector, said synapse further forwards to the first neural module said weight, said delay, and a previously forwarded weight of said synapse.

8. The neural network circuit of claim 7, wherein:

the weights of all synapses in the interconnection network are adaptive as a function of the history of the firing event vectors from the first and the second neural modules.

9. The neural network circuit of claim 8, further comprising:

an interface module configured for updating learning rules in one or more synapses in the interconnection network, wherein each synapse is reconfigurable for learning rules including one or more of;

Hebbian, anti-Hebbian, spike-timing-dependent-plasticity (STDP), and anti-STDP.

10. The neural network circuit of claim 9, wherein:

a first learning rule for learning false negatives is applied to a first synapse in the interconnection network when a neuron interconnected with the first synapse generates a firing event; and

a second learning rule for unlearning false positives is applied to a second synapse in the interconnection network when a neuron interconnected with the second synapse generates a firing event.

11. The neural network circuit of claim 10, wherein:

the neural modules are organized into multiple populations of neural modules, wherein;

a first set of neurons within at least one population of neural modules is configured to receive input events from an input interface to the neural network circuit;

a second set of neurons within at least one population of neural modules is configured to generate output events to an output interface of the neural network circuit; and

a third set of neurons within at least one population of neural modules is configured to generate output events to an evaluation module of the neural network circuit, wherein the evaluation module, based on the type of the output events, feeds the output events to a fourth set of neurons within other populations of neural modules.

12. The neural network circuit of claim 11, wherein each neural module comprises four neurons, and each neuron comprises one of:

a neuron in a learning, bottom-up pathway;

a neuron in a learning, top-down pathway;

a neuron in an unlearning, bottom-up pathway; and

a neuron in an unlearning, top-down pathway.

13. The neural network circuit of claim 12, wherein:

the interface module updates the learning rules such that the neural network operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.

14. The neural network circuit of claim 1, wherein:

each neuron comprises a reconfigurable digital complementary metal-oxide-semiconductor (CMOS) circuit for logic and memory elements for its operational state;

each synapse between two neural modules comprises a reconfigurable digital CMOS circuit for logic and memory elements for its operational state; and

each synapse between two neural modules comprises a communication link implemented via a combination of logical and physical primitives.

15-19. -19. (canceled)

20. A computer program product on a computer-readable medium for an efficient, event-driven design for a neural network circuit comprising a plurality of neural modules, wherein each neural module includes multiple digital neurons such that every neuron in a first neural module is connected to a corresponding neuron in a second neural module via an interconnect network of synapses, wherein each synapse includes a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module, the computer program product comprising instructions which when executed on a computer cause the computer to perform operations including:

generating signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnect network;

at each time step, each neuron updating its operational state and determining whether to generate a firing event in response to firing events received as input signals from corresponding neurons in other neural modules, wherein each said input signal is weighted by a synaptic weight of the synapse communicating said input signal to said neuron;

adapting the synaptic weight associated with each synapse as a function of the firing events of the interconnected neurons;

applying a first learning rule for learning false negatives to a first synapse when a neuron interconnected via the first synapse generates a firing event, and applying a second learning rule for unlearning false positives to a second synapse when a neuron interconnected via the second synapse generates a firing event;

a set of neurons generating output events to an evaluation module, wherein based on the type of the output events generated, the output events are fed to another set of neurons; and

updating the learning rules such that the neural network operates as one or more of;

an auto-associative system, a hetero-associative system, and a reinforcement learning system.