Increased dynamic range artificial neuron network apparatus and methods

• US 9,436,909 B2
• Filed: 06/19/2013
• Issued: 09/06/2016
• Est. Priority Date: 06/19/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Piekniewski, Filip, Polonichko, Vadim, Izhikevich, Eugene
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US13/922,143
• Publication Number:
  US 20140379624A1
• Time in Patent Office: 1,175 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Apparatus and methods for processing inputs by one or more neurons of a network. The neuron(s) may generate spikes based on receipt of multiple inputs. Latency of spike generation may be determined based on an input magnitude. Inputs may be scaled using for example a non-linear concave transform. Scaling may increase neuron sensitivity to lower magnitude inputs, thereby improving latency encoding of small amplitude inputs. The transformation function may be configured compatible with existing non-scaling neuron processes and used as a plug-in to existing neuron models. Use of input scaling may allow for an improved network operation and reduce task simulation time.

Claims

1. An electronic device having computerized logic, the computerized logic configured to be placed in operation with a network node of a network in order to:

evaluate a value of an input into the network node;

when the evaluation indicates that the input value is above a threshold, generate a scaled input using a concave function of the input; and

when the evaluation indicates that the input value is not above the threshold, generate a bypass input;

wherein the scaled input is characterized by a magnitude that is lower than a magnitude of the input value.

2. The device of claim 1, wherein the generation of the scaled input is configured to increase stability of the network, the stability increase characterized by an increased sensitivity of the network to sparse inputs.

3. The device of claim 1, wherein the generation of the scaled input is configured to increase stability of the network, the stability increase characterized by a reduced pathological synchronized activity within the network.

4. A computer readable apparatus having a non-transitory storage medium with at least one computer program stored thereon, the at least one computer program configured to, when executed:

generate a transformed input of a plurality of spiking inputs of a neuron of a spiking neuron network; and

communicate the transformed input to the neuron;

wherein;

the generation of the transformed input is configured to cause the neuron to encode an input within an expanded range into a latency of a spike output, the expanded range characterized by greater span of input values compared to an input range of the neuron in an absence of a transformation.

5. The computer readable apparatus of claim 4, wherein:

the at least one computer program is further configured to be downloaded from a repository over an operative data link.

6. The computer readable apparatus of claim 5, wherein:

the operative data link comprises one or more of a wireless local area network (WLAN) link and a personal area network (PAN) link.

7. The computer readable apparatus of claim 5, wherein:

the repository comprises a cloud-based entity; and

the download is effected via the operative data link between a user computer and the cloud-based entity;

the operative data link is configured based on an authentication process, the authentication process configured based on at least one of (i) an identity of a user; and

(ii) an identity of the user computer.

8. The computer readable apparatus of claim 5, wherein:

a session is established based at least on a payment transaction to an entity associated with the repository, the transaction based on one or more of a subscription, a pay-per-use, a one-time payment, or a rent payment model.

9. A method of adapting an extant logical network to provide a desired functionality, the method comprising:

placing a logical entity in communication with a node;

where the logical entity is configured to process a plurality of inputs for the node of the extant logical network;

receiving one or more inputs of the plurality of inputs; and

causing the extant logical network to operate in accordance with the desired functionality based at least on a transformation of the one or more inputs by the logical entity;

wherein the rule is effectuated based at least on an evaluation of a value of individual ones of the one or more inputs by the logical entity; and

wherein the logical entity is configured such that;

when the evaluation indicates that the value is within a range, the transformation produces an output equal to the value; and

when the evaluation indicates that the value is outside the range, the transformation produces an output different from the value.

10. The method of claim 9, wherein:

the range is characterized by a lower bound and an upper bound; and

the logical entity is configured such that, when the evaluation indicates the value is below the lower bound, the transformation produces an output greater than the value.

11. The method of claim 10, wherein the logical entity is configured such that, when the evaluation indicates the value being above the upper bound, the transformation produces an output smaller than the value.

12. The method of claim 9, wherein:

the desired functionality is characterized by a response generated by the node, the response having a first latency associated therewith;

the first latency falls within a first latency range configured based on the transformation of the one or more inputs; and

a second latency based on the received one or more inputs is characterized by a second latency range, the second range being narrower than the first range.

13. The method of claim 12, wherein:

the transformation comprises a concave function applied to individual ones of the one or more inputs; and

the first latency is configured based on a combination of transformed ones of the one or more inputs.

14. The method of claim 13, wherein the combination comprises a linear combination.

15. The method of claim 12, wherein:

the transformation comprises a concave function applied to a combination of the one or more inputs; and

the first latency is configured based on the transformation of the combination.

16. The method of claim 9, wherein:

individual ones of the plurality of inputs are received by the node via a plurality of connections;

individual ones of the plurality of connections are characterized by an efficacy configured to advance or delay a time of the response generation; and

the efficacy of a given one of the plurality of connections is configured to be adjusted based on a time interval between the time of the response generation and a time of an input of the plurality of inputs associated with the given connection.

17. The method of claim 9, wherein:

the plurality of inputs comprise a first value, and a second value configured larger than the first value; and

the transformation comprises a concave function of the input, configured to increase the first value to produce a third value and to decrease the second value to produce a fourth value.

18. The method of claim 9, further comprising:

updating a state of the node based on at least a combination of the transformed one or more inputs, the updating configured to cause the node to generate a response based on at least the updated state of the node breaching a threshold;

wherein;

the node state is configured in accordance with a dynamic process configured to be updated at a plurality of time intervals;

a combination of the transformed inputs is configured to produce a combined input associated with a given update interval of the plurality of time intervals;

the node state is configured based on the combined input and at least one other combined input obtained during another update interval of the plurality of time intervals prior to the given update interval, the at least one other combined input configured based on the combination of other scaled inputs received by the node prior to the another update interval.

19. The method of claim 18, further comprising:

decaying the combined input in accordance with a decay process, the decaying configured to reduce a contribution of the combined input for use by a subsequent update of the plurality of updates.

20. The method of claim 19, wherein the decay process is selected from a group consisting of:

an exponential decay characterized by an exponent, a linear decay process characterized by an average rate, a geometric decay, and a polynomial decay.