Apparatus and methods for backward propagation of errors in a spiking neuron network

• US 9,489,623 B1
• Filed: 10/15/2013
• Issued: 11/08/2016
• Est. Priority Date: 10/15/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Sinyavskiy, Oleg, Polonichko, Vadim
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Buss, Benjamin
• Application Number:
  US14/054,366
• Time in Patent Office: 1,120 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: Y10S 901/50 Miscellaneous

Abstract

Apparatus and methods for developing robotic controllers comprising parallel networks. In some implementations, a parallel network may comprise at least first and second neuron layers. The second layer may be configured to determine a measure of discrepancy (error) between a target network output and actual network output. The network output may comprise control signal configured to cause a task execution by the robot. The error may be communicated back to the first neuron layer in order to adjust efficacy of input connections into the first layer. The error may be encoded into spike latency using linear or nonlinear encoding. Error communication and control signal provision may be time multiplexed so as to enable target action execution. Efficacy associated with forward and backward/reverse connections may be stored in individual arrays. A synchronization mechanism may be employed to match forward/reverse efficacy in order to implement plasticity.

Claims

1. A method of communicating feedback from one layer to an other layer of a multilayer spiking neuron network, the method comprising:

determining the feedback based at least on an output from the one layer and a teaching input into the one layer;

encoding the feedback into a spike latency of a first spike and a second spike;

communicating the first and the second spikes to the other layer;

based on the first spike and the second spike, determining an error associated with an output of the other layer; and

adjusting an efficacy of an input connection into the other layer based on an input to the other layer and the error;

wherein the encoding further comprises a binary encoding comprising the step of determining a latency of the first spike based at least on a first bit value associated with the feedback, and a latency of the second spike based at least on a second bit value associated with the feedback.

2. The method of claim 1, further comprising:

configuring the output of the one layer based on the output of the other layer and an efficacy of at least one connection between the one layer and the other layer; and

determining the output of the other layer based at least on the input to the other layer and the efficacy of the at least one connection between the one layer and the other layer.

3. The method of claim 2, further comprising:

configuring the input to the other layer to communicate information related to one or more objects; and

configuring the teaching input to indicate a target object;

wherein the output of the one layer is indicative of an occurrence of the target object within the one or more objects.

4. The method of claim 3, further comprising:

adjusting the efficacy of the at least one connection between the one and other layers based on the output of the other layer and the feedback.

5. The method of claim 4, wherein the communicating the first and second spikes to the other layer comprises communicating via a feedback connection;

and

the method further comprises adjusting the efficacy of the feedback connection based on the output of the other layer and the feedback.

6. The method of claim 5, wherein:

the other layer comprises a first spiking neuron;

the one layer comprises a second spiking neuron;

the adjusting the efficacy of the at least one connection between the one layer and the other layer comprises adjusting based on an event associated with the second neuron; and

the adjusting the efficacy of the feedback connection comprises adjusting based on an event associated with the first neuron.

7. The method of claim 6, further comprising:

determining the output of the one layer based on a sigmoid operation on a combination of the output of the other layer and the efficacy of the at least one connection between the one and the other layers; and

the determining the output of the other layer comprises determining based on a sigmoid operation on a combination of the input to the other layer and the efficacy of the input connection.

8. The method of claim 1, wherein the encoding further comprises an encoding selected from the group consisting of linear encoding and logarithmic encoding.

9. The method of claim 1, wherein the encoding further comprises a hybrid encoding configured to determine the latency of the first spike based at least on a first function of the feedback, and the latency of the second spike based at least on a second function of the feedback.

10. The method of claim 1, further comprising:

communicating at least one spike payload information comprising a plurality of bits; and

encoding the feedback into the plurality of bits.

11. The method of claim 1, further comprising encoding the feedback into a number of spikes within a plurality of spikes

wherein the feedback comprises a real number characterized by two or more bits.

12. An apparatus comprising a non-transitory computer-readable storage medium comprising a plurality of instructions configured to, when executed by a processor, cause the apparatus to:

determine a feedback based at least on an output from one layer and a teaching input into the one layer;

encode the feedback into a property of at least one spike;

communicate the at least one spike to an other layer;

based on the at least one spike, determine an error associated with an output of the other layer; and

adjust an efficacy of an input connection into the other layer based on an input to the other layer and the error;

wherein;

the at least one spike is configured to communicate payload information comprising a plurality of bits; and

the feedback is encoded into the plurality of bits.

13. The apparatus of claim 12, wherein:

the output of the one layer is determined based on the output of the other layer and an efficacy of at least one connection between the one layer and the other layer; and

the output of the other layer is determined based at least on the input to the other layer and the efficacy.

14. The apparatus of claim 13, wherein:

the input to the other layer is configured to communicate information related to one or more objects;

the teaching input is configured to indicate a target object; and

the output of the one layer is indicative of an occurrence of the target object within the one or more objects.

15. The apparatus of claim 14, wherein the plurality of instructions are further configured, when executed by the processor, to cause the apparatus to:

adjust the efficacy of the at least one connection between the one and another layers based on the output of the other layer and the feedback.

16. The apparatus of claim 15, wherein:

the at least one spike is communicated from the one layer to the other layer via a feedback connection; and

the efficacy of the feedback connection is configured to be adjusted based on the output of the another layer and the feedback.

17. The apparatus of claim 16, wherein:

the other layer comprises a first spiking neuron;

the one layer comprises a second spiking neuron;

the adjustment of the efficacy of the at least one connection between the one layer and the other layer is based on an event associated with the second neuron; and

the adjustment of the efficacy of the feedback connection is configured based on an event associated with the first neuron.

18. The apparatus of claim 17, wherein:

the determination of the output of the one layer is configured based on a sigmoid operation on a combination of the output of the other layer and the efficacy of the at least one connection between the one and the other layers; and

the determination of the output of the other layer is configured based on a sigmoid operation on a combination of the input to the other layer and the efficacy of the input connection.

19. The apparatus of claim 12, wherein:

the property comprises a spike latency; and

the encoding of the feedback is selected from the group consisting of linear encoding and logarithmic encoding.

20. The apparatus of claim 12, wherein:

the at least one spike comprises a first and a second spike;

the property comprises a spike latency; and

the encoding of the feedback comprises a binary encoding configured to determine a latency of the first spike based at least on a first bit value associated with the feedback, and a latency of the second spike based at least on a second bit value associated with the feedback.

21. The apparatus of claim 12, wherein:

the at least one spike comprises a first and a second spike;

the property comprises a spike latency; and

the encoding of the feedback comprises a hybrid encoding configured to determine a latency of the first spike based at least on a first function of the feedback, and a latency of the second spike based at least on a second function of the feedback.

22. The apparatus of claim 12, wherein:

the feedback comprises a real number characterized by two or more bits;

the at least one spike comprises a plurality of spikes; and

the property comprises a number of spikes within the plurality of spikes.

23. A controller apparatus useful for controlling a robot, the apparatus comprising:

a first plurality of spiking neurons configured to receive, via a plurality of input connections, a sensory input associated with an environment of the robot, and to provide a signal conveying information about an object within the environment;

a second plurality of spiking neurons configured to determine a control output based at least on the signal; and

one or more output connections configured to provide the control output for the robot;

wherein the second plurality of neurons is configured to;

determine an error measure based at least on the control output and the teaching input;

communicate the error measure to the first plurality of neurons via a feedback interface, the feedback interface being configured to share access to the neuron with the one or more output connections;

the second plurality of neurons is configured to be updated at a plurality of iterations;

the control output and communication of the error measure by the second plurality of neurons are configured as a time multiplex characterized by at least;

a first portion of the plurality of iterations being configured to provide the control output to the robot; and

a second portion of the plurality of iterations being configured to communicate the feedback to the first plurality of neurons.

24. The apparatus of claim 23, wherein individual iterations within the first portion and the second portion of the plurality of iterations are arranged in an alternating order.

25. The apparatus of claim 24, wherein:

individual iterations within one of the first portion or the second portion of the plurality of iterations comprise even iterations; and

individual iterations within the other one of the first portion or the second portion of the plurality of iterations comprise odd iterations.

26. The apparatus of claim 23, wherein:

individual neurons of the first plurality of spiking neurons are operably coupled to one or more individual neurons of the second plurality of spiking neurons via a plurality of feed forward and a plurality of feedback connections;

individual ones of the plurality of feed forward connections are configured to communicate the signal from the first plurality of spiking neurons to the second plurality of spiking neurons;

individual ones of the plurality of feedback connections are configured to communicate the error measure from the second plurality of spiking neurons to the first plurality of spiking neurons;

individual ones of the plurality of feed forward connections are characterized by a feed-forward efficacy, the feed forward efficacy of a given feed forward connection configured to be adjusted based at least on the error measure and a portion of the signal communicated via the given feed forward connection;

individual ones of the plurality of feedback connections are characterized by a feedback efficacy; and

the feedback efficacy of a given feedback connection is configured to be adjusted based at least on the error measure and a portion of the signal communicated via the given feed-forward connection to a given neuron of the second plurality of neurons.

27. The apparatus of claim 26, wherein:

individual ones of the plurality of input connections are characterized by an input efficacy, the input efficacy of a given input connection configured to be adjusted based on a discrepancy measure and a portion of the sensory input communicated via the given input connection to a given neuron of the first plurality of neurons; and

the discrepancy measure is determined based at least on the feedback received by the given neuron of the first plurality of neurons and the feedback efficacy of the given feedback connection.

28. The apparatus of claim 27, wherein:

the connectivity between the individual neurons of the first plurality of spiking neurons and the individual neurons of the second plurality of spiking neurons is selected from the group consisting of;

(i) all-to-all, (ii) all-to-one, and (iii) some-to-some mapping.

29. The apparatus of claim 28, wherein:

the input efficacy of the given input connection is configured to advance or delay generation of a portion of the signal by the given neuron of the first plurality of neurons; and

the feed-forward efficacy of the given feed forward connection is configured to advance or delay generation of a portion of the control output by the given neuron of the second plurality of neurons.

30. The apparatus of claim 27, wherein:

the error measure communicated via the given feedback connection is configured to be determined by the given neuron of the second plurality of neurons;

the feedback efficacy of the given feedback connection is configured to be adjusted by the given neuron of the first plurality of neurons; and

the discrepancy measure determination is effectuated by the given neuron of the first plurality of neurons.