Elementary network description for neuromorphic systems with plurality of doublets wherein doublet events rules are executed in parallel

• US 9,104,973 B2
• Filed: 09/21/2011
• Issued: 08/11/2015
• Est. Priority Date: 09/21/2011
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm Technologies Incorporated (Qualcomm, Inc.)
• Inventors: Nageswaran, Jayram Moorkanikara, Petre, Csaba, Szatmary, Botond, Izhikevich, Eugene M., Piekniewski, Filip
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Brown, Jr., Nathan
• Application Number:
  US13/239,123
• Publication Number:
  US 20130073495A1
• Time in Patent Office: 1,420 Days
• Field of Search: 706/25
• US Class Current: 1/1
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 3/105 Shells for specifying net layout
• CPC Class Codes: G06N 99/007 Molecular computers, i.e. using inorganic molecules using biomolecules G06N3/002

Abstract

A simple format is disclosed and referred to as Elementary Network Description (END). The format can fully describe a large-scale neuronal model and embodiments of software or hardware engines to simulate such a model efficiently. The architecture of such neuromorphic engines is optimal for high-performance parallel processing of spiking networks with spike-timing dependent plasticity. Neuronal network and methods for operating neuronal networks comprise a plurality of units, where each unit has a memory and a plurality of doublets, each doublet being connected to a pair of the plurality of units. Execution of unit update rules for the plurality of units is order-independent and execution of doublet event rules for the plurality of doublets is order-independent.

Claims

1. A system configured to provide a neuronal network, the system comprising:

one or more integrated circuits having a neuronal network embodied therein, the neuronal network comprising;

a plurality of units, each having a memory updated according to a unit update rule; and

a plurality of doublets, each doublet being connected to a pair of the plurality of units and configured to update the memory of a postsynaptic one of the pair of units in response to an event received from the presynaptic other of the pair of units according to a doublet event rule;

wherein execution of unit update rules for the plurality of units is order-independent and execution of doublet event rules for the plurality of doublets is order-independent;

wherein the doublet event rules are executable in parallel; and

wherein the unit update rules are executable in parallel.

2. The system of claim 1, wherein for the each doublet, the execution of the corresponding doublet event rule is triggered in response to a change in the memory of the presynaptic unit.

3. The system of claim 2 wherein, for at least one doublet, there is a delay between occurrence of changes in the memory of its corresponding presynaptic unit and updates to the memory of its corresponding postsynaptic unit.

4. The system of claim 2, wherein the doublet event rule for each doublet is triggered when its corresponding presynaptic unit transmits an event to the each doublet.

5. The system of claim 1, wherein each doublet has a memory that is updated in response to events received from its corresponding presynaptic unit, and wherein the memory of the postsynaptic unit corresponding to the each doublet is updated based on the content of the memory of the each doublet.

6. The system of claim 5, wherein updates to unit memories and doublet memories are performed within a period of a time determined by a system clock, and wherein all of the updates are completed before the period of time expires.

7. The system of claim 5, wherein the doublet event rule for each doublet includes a timing event rule that controls modification of the memory of the each doublet.

8. The system of claim 7, wherein the timing event rule comprises pre-post event rule that controls modification of memory of the each doublet and that depends on the difference between the timing of events transmitted by the presynaptic unit and the timing of at least one subsequent event transmitted by the postsynaptic unit.

9. The system of claim 7, wherein the timing event rule comprises post-pre event rule that controls modification of memory of the each doublet and that depends on the difference between the timing of events transmitted by the presynaptic unit and the timing of at least one preceding pulse transmitted by the postsynaptic unit.

10. The system of claim 5, wherein each doublet is configured to update its memory according to a predefined post event rule that is triggered by its corresponding postsynaptic unit.

11. The system of claim 10, wherein the post event rule depends on the difference between the timing of pulse of the postsynaptic unit and the timing of at least one preceding pulse of the presynaptic unit.

12. The system of claim 1, wherein the memory of each unit is updated by atomic addition.

13. The system of claim 1, wherein the neuronal network further comprises a plurality of triplets each having a memory, each triplet being configured to access the memory of two of the plurality of units, wherein each triplet is operable to update its own triplet memory and the memory at least one of the two units in accordance with a triplet rule, wherein execution of triplet update rules for the plurality of triplets is order-independent.

14. The system of claim 13, wherein the neuronal network is represented as a directed graph.

15. The system of claim 14, wherein the representation of a neuronal network is mapped to the interconnected plurality of units independently of the physical and electrical structure of the units, the doublets and the triplets.

16. The system of claim 15, wherein the neuronal network is expressed in a specification provided in non-transitory storage and coded according to a hardware description language.

17. The system of claim 16, wherein the specification is operable to reconfigure a semiconductor integrated circuit.

18. A method of implementing neuronal networks embodied in one or more integrated circuits, the method comprising:

interconnecting a plurality of units embodied in one or more integrated circuits using doublets configured to modify a memory of a postsynaptic unit responsive to an event received from a presynaptic unit;

configuring doublet event rules that determine how the doublets respond to events; and

configuring a unit update rule for each unit that controls the response of the each unit to memory updates initiated by a doublet, wherein execution of the doublet event rules is order-independent and execution of the unit update rules is order-independent;

wherein configuring doublet event rules includes configuring each of the doublet event rules to be executed within a time step having a duration determined by a system clock after receiving an event during the time step.

19. The method of claim 18, wherein each presynaptic unit maintains an event condition that controls transmission of events to a doublet.

20. The method of claim 18, further comprising configuring an after-event unit update rule for each presynaptic unit to be executed in association with the transmission of the events, wherein the after-event unit update rule causes modification of memory of the each presynaptic unit.

21. The method of claim 18, wherein each doublet modifies the memory of the postsynaptic unit memory by atomic addition.

22. The method of claim 18, wherein the doublet event rule for each doublet includes a timing event rule that controls modification of a memory of the each doublet based on timing of pulses associated with one or more of the plurality of units.

23. The method of claim 22, wherein the timing event rule comprises pre-post event rule that controls modification of memory of the each doublet and that depends on the difference between the timing of pulses of a presynaptic unit and the timing of pulses of at least one subsequent pulse of the postsynaptic unit.

24. The method of claim 22, wherein the timing event rule comprises post-pre event rule that controls modification of memory of the each doublet and that depends on the difference between the timing of pulses of the presynaptic unit and the timing of pulses of at least one preceding pulse of the postsynaptic unit.

25. The method of claim 22, wherein execution of the doublet event rule for each doublet is triggered by an event received from a presynaptic unit.

26. The method of claim 18, further comprising

interconnecting a second plurality of units using triplets configured to access memory of a pair of triplet-connected units;

and

configuring a triplet update rule for each triplets that controls updates to the memory of the each triplets and the memory of at least one of its corresponding pair of triplet-connected units, wherein execution of triplet update rules is order-independent.

27. The method of claim 26, wherein the update rule for each triplet responds to information maintained in memories of the pair of units connected by the each triplet.

28. The method of claim 26, wherein triplets update memories of units by atomic addition.

29. The method of claim 26, wherein each step of interconnecting pluralities of units, and each step of configuring rules are performed in accordance with a representation of a desired neuronal network.

30. The method of claim 29, wherein the desired neuronal network is represented as a directed graph.

31. The method of claim 29, wherein the representation of a neuronal network is mapped to the interconnected plurality of units independently of:

the physical and electrical structure of the plurality of units;

the physical and electrical structure of doublets used to connect the plurality of units; and

of

the physical and electrical structure of the triplets used to the plurality of units.

32. The method of claim 29, wherein the pluralities of units, the doublets and the triplets are embodied in a semiconductor integrated circuit.

33. The method of claim 32, further comprising expressing the neuronal network in a specification using a hardware description language.

34. The method of claim 33, wherein the specification is operable to reconfigure a semiconductor integrated circuit.

35. The method of claim 33, wherein the specification is used to manufacture a semiconductor integrated circuit.

36. The method of claim 35, wherein the first number of units includes certain of the second number of units.

37. The method of claim 36, wherein the doublet event rules are executable in parallel, the triplet update rules are executable in parallel, and the unit update rules are executable in parallel.

38. The method of claim 37, wherein the triplet update rules and the unit update rules are executed once within a time step having a duration determined by a system clock.

39. The method of claim 36, wherein the doublet event rule comprises pulse delivery rule that modifies postsynaptic unit memory based on the content of its associated doublet memory.

40. The method of claim 36, wherein each doublet is configured to respond to events communicated by its corresponding postsynaptic unit by initiating updates of doublet memory according to a predefined post event rule configured for the each doublet.

41. The method of claim 40, wherein the post event rule depends on the difference between the timing of pulse of the postsynaptic unit and the timing of at least one preceding pulse of the presynaptic unit.

42. A system configured to provide a neuronal network, the system comprising:

one or more integrated circuits having a neuronal network embodied therein, the neuronal network comprising;

a plurality of units, each unit comprising a non-transitory unit memory that is periodically updated according to a unit update rule associated with the each unit; and

a plurality of doublets, each doublet connecting a pair of units, wherein each doublet is configured to respond to a stimulation by one of the pair of units by initiating update of the memory of the other unit according to a predefined doublet event rule associated with the each doublet;

wherein updates to unit memories are performed within a period of a time determined by a system clock;

wherein all of the updates are completed before the period of time expires;

wherein each pair of units connected by a doublet comprises a presynaptic unit and a postsynaptic unit; and

wherein the connecting doublet is configured to modify the memory of the postsynaptic unit by atomic addition.

43. The system of claim 42, wherein the doublet event rule comprises a pulse delivery rule that modifies postsynaptic unit memory based on the content of a memory of the connecting doublet.

44. The system of claim 42, wherein the execution of the doublet event rule of the connecting doublet is triggered by an event generated by the presynaptic unit.

45. The system of claim 42, wherein the doublet event rules for at least two doublets are executable in parallel, and wherein the unit update rules for at least two of the units are executable in parallel.

46. The system of claim 45, wherein the post event rule depends on the difference between the timing of pulse of the postsynaptic unit and the timing of at least one preceding pulse of the presynaptic unit.

47. The system of claim 42, wherein, for at least one doublet, there is a delay between stimulation by the one of the pair of units and the initiation of the memory update of the other unit.

48. The system of claim 42, wherein each unit of the plurality of units is configured to generate an event based on content of memory of the each unit, and wherein each unit is further configured to communicate the event to a doublet connected to the each unit.

49. The system of claim 48, wherein an after-event unit update rule is configured for each of the plurality of units that is executed after the event has been generated, wherein the after-event unit update rule modifies memory of the each unit.

50. The system of claim 42, wherein each doublet comprises a memory and wherein the doublet event rule for each doublet includes a timing event rule that controls modification of a memory of the each doublet based on timing of events associated with one or more of the plurality of units.

51. The system of claim 50, wherein the timing event rule comprises pre-post event rule that controls modification of memory of the doublet and that depends on the difference between the timing of pulses of the presynaptic unit and the timing of pulses of at least one subsequent pulse of the postsynaptic unit.

52. The system of claim 50, wherein the timing event rule comprises post-pre event rule that controls modification of memory of the doublet and that depends on the difference between the timing of pulses of the presynaptic unit and the timing of pulses of at least one preceding pulse of the postsynaptic unit.

53. The system of claim 42, wherein the neuronal network further comprises a plurality of triplets, each triplet configured to access the memory of two of the units, wherein each triplet is configured to initiate updates of the memory of at least one of the two units according to a triplet update rule associated with the each triplet.

54. The system of claim 53, wherein one or more of the plurality of units is connected to both a doublet and a triplet.

55. The system of claim 53, wherein at least one of the units has a triplet update rule that is executable in parallel with the triplet update rule of another unit.

56. The system of claim 53, wherein each pair of units connected by a triplet comprises a presynaptic unit and a postsynaptic unit, wherein the connecting triplet is configured to modify the memory of at least one of the presynaptic unit and the postsynaptic unit by atomic addition.

57. The system of claim 56, wherein the triplet update rule for the connecting triplet operates based on information maintained in the memories of the presynaptic unit and the postsynaptic unit.

58. The system of claim 57, wherein the triplet update rule for the connecting triplet is configured to cause modification of a memory of the connecting triplet.

59. The system of claim 57, wherein the connecting triplet is configured to modify the memories of one or more of the presynaptic and postsynaptic units by atomic addition.

60. The system of claim 53, wherein the neuronal network is represented as a directed graph.

61. The system of claim 60, wherein the representation of a neuronal network is mapped to the interconnected plurality of units independently of the physical and electrical structure of the units, the doublets and the triplets.

62. The system of claim 61, wherein the neuronal network is expressed in a specification provided in non-transitory storage and coded according to a hardware description language.

63. The system of claim 62, wherein the specification is operable to reconfigure a semiconductor integrated circuit.

64. The system of claim 62, wherein the specification controls manufacture of a semiconductor integrated circuit.

65. The system of claim 61, wherein the neuronal network is expressed in a specification provided in non-transitory storage and coded in a programming language.

66. The system of claim 42, wherein each doublet is configured to respond by initiating update of doublet memory according to a predefined post event rule associated with the each doublet triggered by an event communicated by the postsynaptic unit.

67. The system of claim 42, wherein the events emulate pulses of excitable neurons.