NEUROMORPHIC ARCHITECTURE FOR UNSUPERVISED PATTERN DETECTION AND FEATURE LEARNING

• US 20180075346A1
• Filed: 09/13/2016
• Published: 03/15/2018
• Est. Priority Date: 09/13/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: International Business Machines Corporation
• Inventors: Pantazi, Angeliki, Wozniak, Stanislaw Andrzej
• Granted Patent: US 10,650,307 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

Embodiments relate to a neuromorphic architecture for unsupervised feature learning using memristive synapses realized using phase-change devices. A spiking neural network architecture for unsupervised pattern learning and a spike-based learning algorithm compatible with phase-change synapses is described, and a feature-learning algorithm capable of performing a sequence of set operations on input patterns is provided. A learning rule for the extraction of certain features of the input that is compatible with spiking neurons and synapses with spike-based plasticity is also provided. The system enables enhanced pattern- and feature-extraction capabilities in neuromorphic systems.

Claims

1. An apparatus for computation comprising:

a first neuronal module comprising a first plurality of neurons, wherein each neuron in the first plurality of neurons operates with a different learning mechanism selected from a first set of learning mechanisms;

a second neuronal module comprising a second plurality of neurons, wherein each neuron in the second plurality of neurons operates with a different learning mechanism selected from a second set of learning mechanisms; and

an arbitration mechanism operatively connected to the first neuronal module and the second neuronal module.

2. The apparatus of claim 1, wherein the first set of learning mechanisms is the same as the second set of learning mechanisms.

3. The apparatus of claim 1, wherein the first set of learning mechanisms is different from the second set of learning mechanisms.

4. The apparatus of claim 1, wherein a learning mechanism in the first set of learning mechanisms comprises spike-timing dependent plasticity with selective potentiation.

5. The apparatus of claim 1, wherein a learning mechanism in the first set of learning mechanisms comprises spike-timing dependent plasticity with selective depression.

6. The apparatus of claim 1, wherein a learning mechanism in the first predefined set of learning mechanisms comprises inverted spike-timing dependent plasticity.

7. The apparatus of claim 1, wherein the arbitration mechanism disables either the first neuronal module or the second neuronal module.

8. The apparatus of claim 1, wherein the first plurality of neurons comprises at least a first level tuned neuron, the second plurality of neurons comprises at least a second level tuned neuron, and the arbitration mechanism comprises a third neuron that provides a level output signal that modifies the behavior of the first level tuned neuron and the second level tuned neuron.

9. The apparatus of claim 1, wherein the first neuronal module comprises at least a first neuron with inhibitory link, the second neuronal module comprises at least a second neuron with inhibitory link, and the arbitration mechanism comprises an inhibitory output that modifies the behavior of the first neuron and the second neuron.

10. The apparatus of claim 1, wherein a spike-timing dependent plasticity comprises an enhancement selected from the group consisting of selective potentiation, selective depression, and inverted spike-timing dependent plasticity, and wherein the enhancement is controlled by a measure indicative of a learning status.

11. The apparatus of claim 10, wherein the measure comprises a cumulative strength of the activated synapses.

12. The apparatus of claim 10, wherein the measure comprises a count of synapses of the neuron that have high weight values.

13. The apparatus of claim 10, wherein the measure comprises the firing of at least one neuron in the first plurality of neurons.

14. A method for pattern detection and feature learning comprising:

training a first neuronal module, wherein the first neuronal module comprises a first neuron having a first learning mechanism;

training a second neuronal module, wherein the second neuronal module comprises a second neuron having a second learning mechanism;

providing an arbitration mechanism connected to the first neuronal module and the second neuronal module; and

applying an input to the first neuronal module and the second neuronal module.

15. The method of claim 14, further comprising training a third neuronal module, wherein the third neuronal module comprises a third neuron having a third learning mechanism, and applying the input to the third neuronal module.

16. The method of claim 14, wherein the first learning mechanism comprises spike-timing dependent plasticity with selective potentiation.

17. The method of claim 14, wherein the first learning mechanism comprises spike-timing dependent plasticity with selective depression.

18. The method of claim 14, wherein the first learning mechanism comprises inverted spike-timing dependent plasticity.

19. The method of claim 14, further comprising enhancing a spike-timing dependent plasticity of the first neuron with a selective potentiation that is controlled by a measure indicative of a learning status.

20. An apparatus for computation comprising:

a first neuronal module comprising a first level tuned neuron, wherein the first level tuned neuron has a first learning mechanism;

a second neuronal module comprising a second level tuned neuron, wherein the second level tuned neuron has a second learning mechanism; and

an arbitration mechanism operatively connected to the first neuronal module and the second neuronal module and comprising a third neuron that provides a level output that modifies the behavior of the first neuronal module and second neuronal module.