Dynamically reconfigurable stochastic learning apparatus and methods

• US 9,015,092 B2
• Filed: 06/04/2012
• Issued: 04/21/2015
• Est. Priority Date: 06/04/2012
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Polonichko, Vadim, Sinyavskiy, Oleg
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): OLUDE AFOLABI, OLATOYOSI
• Application Number:
  US13/487,576
• Publication Number:
  US 20130325775A1
• Time in Patent Office: 1,051 Days
• Field of Search: 706/25
• US Class Current: 706/25
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Generalized learning rules may be implemented. A framework may be used to enable adaptive signal processing system to flexibly combine different learning rules (supervised, unsupervised, reinforcement learning) with different methods (online or batch learning). The generalized learning framework may employ average performance function as the learning measure thereby enabling modular architecture where learning tasks are separated from control tasks, so that changes in one of the modules do not necessitate changes within the other. Separation of learning tasks from the control tasks implementations may allow dynamic reconfiguration of the learning block in response to a task change or learning method change in real time. The generalized learning apparatus may be capable of implementing several learning rules concurrently based on the desired control application and without requiring users to explicitly identify the required learning rule composition for that application.

Claims

1. Apparatus comprising a storage medium, said storage medium comprising a plurality of instructions to operate a network, comprising a plurality of spiking neurons, the instructions configured to, when executed:

based at least in part on receiving a task indication, select first group and second group from said plurality of spiking neurons;

operate said first group in accordance with first learning rule, based at least in part on an input signal and training signal; and

operate said second group in accordance with second learning rule, based at least in part on input signal;

wherein;

said task indication comprises at least said first and said second rules;

said first rule comprises at least reinforcement learning rule;

said second rule comprises at least unsupervised learning rule; and

said first rule further comprises first combination of at least said reinforcement learning rule and supervised learning rule.

2. The apparatus of claim 1, wherein:

said first combination further comprises unsupervised learning rule; and

said second rule further comprises second combination of reinforcement, supervised and unsupervised learning rules, said second combination being distinctly different from said first combination.

3. The apparatus of claim 1, wherein neurons in at least said first group are selected from said plurality, based at least in part on a random parameter associated with spatial coordinate of said plurality of neurons within said network;

and

at least one of said first and said second group is characterized by a finite life span, said life span configured during said selecting.

4. The apparatus of claim 1, wherein:

neurons in at least said first group are selected using high level neuromorphic description language (HNLD) statement comprising a tag, configured to identify said neurons; and

said first group and said second group comprise disjoint set pair characterized by an empty intersect.

5. The apparatus of claim 1, wherein:

said first group and said second group comprise overlapping set pair;

neurons in at least said first group are selected using high level neuromorphic description language (HNLD) statement comprising a tag; and

said tag comprises an alphanumeric identifiers adapted to identify a spatial coordinate of neurons within respective groups.

6. The apparatus of claim 1, wherein:

said first rule having target performance associated therewith; and

said operate said first group in accordance with said first rule is configured to produce actual output having actual performance associated therewith such that said actual performance being closer to said target performance, as compared to another actual performance associated with another actual output being generated by said first group operated in absence of said first learning rule.

7. The apparatus of claim 6, wherein:

said training signal further comprises a desired output;

comparison of said actual performance to said target performance is based at least in part on a distance measure between said desired output and actual output such that said actual performance is being closer to said target performance is characterized by a value of said distance measure, determined using said desired output and said actual output, being smaller compared to another value of said distance measure, determined using said desired output and said another actual output.

8. The apparatus of claim 7, wherein said distance measure comprises instantaneous mutual information.

9. The apparatus of claim 7, wherein said distance measure comprises a squared error between (i) a convolution of said actual output with first convolution kernel α

and (ii) a convolution of said desired output with second convolution kernel β

.

10. The apparatus of claim 1, wherein operate said first group in accordance with first learning rule is based at least in part on first input signal;

and

operate said second group in accordance with second learning rule is based at least in part on second input signal, said second signal being different from said first signal.

11. The apparatus of claim 1, wherein operate said first group in accordance with first learning rule is based at least in part on input signal;

and

operate said second group in accordance with second learning rule is based at least in part on said input signal.

12. Computer readable apparatus comprising a storage medium, said storage medium comprising a plurality of instructions to operate a processing apparatus, the instructions configured to, when executed:

based at least in part on first task indication at first instance, operate said processing apparatus in accordance with first stochastic hybrid learning rule configured to produce first learning signal based at least in part on first input signal and first training signal, associated with said first task indication; and

based at least in part on second task indication at second instance, subsequent to first instance operate said processing apparatus in accordance with second stochastic hybrid learning rule configured to produce second learning signal based at least in part on second input signal and second training signal, associated with said second task indication;

wherein;

said first hybrid learning rule is configured to effectuate first rule combination; and

said second hybrid learning rule is configured to effect second rule combination, said second combination distinctly different from said first combination.

13. The apparatus of claim 12, wherein said second task indication is configured based at least in part on a parameter, associated with said operating said processing apparatus in accordance with said first stochastic hybrid learning rule, exceeding a threshold.

14. The apparatus of claim 13, wherein said parameter is selected from the group consisting of:

(i) peak power consumption associated with operating said processing apparatus;

(ii) power consumption associated with operating said processing apparatus, averaged over an interval associated with producing said first learning signal;

(iii) memory utilization associated with operating said processing apparatus; and

(iv) logic resource utilization associated with operating said processing apparatus.

15. The apparatus of claim 12, wherein said second task indication is configured based at least in part on detecting a change in number if input channels associated with said first input signal.

16. The apparatus of claim 12, wherein said second task indication is configured based at least in part on detecting a change in composition of said first training signal, said change comprising any one or more of (i) addition of reinforcement signal;

(ii) removal of reinforcement signal;

(iii) addition of supervisor signal; and

(iv) removal of supervisory signal.

17. A computer-implemented method of operating a computerized spiking network, comprising a plurality of nodes, the method comprising:

based at least in part on first task indication at a first instance, operating said plurality of nodes in accordance with a first stochastic hybrid learning rule configured to produce first learning signal based at least in part on first input signal and first training signal, associated with said first task indication; and

based at least in part on second task indication at second instance, subsequent to first instance;

operating first portion of said plurality of nodes in accordance with second stochastic hybrid learning rule configured to produce second learning signal based at least in part on second input signal and second training signal, associated with said second task indication; and

operating second portion of said plurality of nodes in accordance with third stochastic learning rule configured to produce third learning signal based at least in part on second input signal associated with said second task indication;

wherein;

said first hybrid learning rule is configured to effect first rule combination; and

said second hybrid learning rule is configured to effect second rule combination, said second combination substantially different from said first combination.

18. The method of claim 17, wherein:

said first hybrid learning rule comprises first rule having first combination coefficient associated therewith and second rule having second combination coefficient associated therewith;

said second hybrid learning rule comprises said first rule having third combination coefficient associated therewith and said second rule having fourth combination coefficient associated therewith; and

at said first coefficient different from said third coefficient.

19. The method of claim 17, wherein said first rule is selected from the group consisting of reinforcement and supervised learning rules, said first rule being configured in accordance with said first training and said second training signal, respectively.

20. The method of claim 17, wherein:

said first hybrid learning rule comprises first rule and second rule; and

said second hybrid learning rule comprises said first rule and at least third rule, said third rule different from at least one of said first rule and said second rule.

21. The method of claim 17, wherein said third learning signal is based at least in part on said second learning signal.

22. The method of claim 17, wherein:

said first hybrid learning rule is configured to implement reinforcement and supervised rule combination; and

said second hybrid learning rule is configured to implement at least supervised and unsupervised rule combination.

23. The method of claim 17, wherein said second task indication is based at least in part one or more of (i) a detected a change in network processing configuration;

and (ii) a detected change in configuration of said first input signal.

24. The method of claim 23, wherein said change in network processing configuration comprises any of (i) a failure of one or more of said plurality of nodes;

(ii) an addition of one or more failure of one or more nodes; and

(iii) a failure of one or more of input-output interface associated with said plurality of nodes.