ADAPTIVE PREDICTOR APPARATUS AND METHODS

• US 20140277718A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Izhikevich, Eugene, Sinyavskiy, Oleg, Passot, Jean-Baptiste
• Granted Patent: US 9,764,468 B2
• Time in Patent Office: Days
• US Class Current: 700/250
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/39292 Neural brain based controller based on simplified model of vertebrate nervous system
• CPC Class Codes: G05B 2219/39298 Trajectory learning
• CPC Class Codes: G05B 2219/40499 Reinforcement learning algorithm
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

Apparatus and methods for training and operating of robotic devices. Robotic controller may comprise a predictor apparatus configured to generate motor control output. The predictor may be operable in accordance with a learning process based on a teaching signal comprising the control output. An adaptive controller block may provide control output that may be combined with the predicted control output. The predictor learning process may be configured to learn the combined control signal. Predictor training may comprise a plurality of trials. During initial trial, the control output may be capable of causing a robot to perform a task. During intermediate trials, individual contributions from the controller block and the predictor may be inadequate for the task. Upon learning, the control knowledge may be transferred to the predictor so as to enable task execution in absence of subsequent inputs from the controller. Control output and/or predictor output may comprise multi-channel signals.

Claims

1. A method of predicting a plant control output by an adaptive computerized predictor apparatus, the method comprising:

configuring the predictor apparatus, using one or more processors, to operate in accordance with a learning process based on a teaching input;

at a first time instance, based on a sensory context, causing the predictor apparatus to generate the plant control output;

configuring the predictor apparatus, using one or more processors, to provide the predicted plant control output as the teaching input into the learning process; and

at a second time instance subsequent to the first time instance, causing the predictor apparatus to generate the predicted plant control output based on the sensory context and the teaching input;

wherein the predicted plant control output is configured to cause the plant to perform an action consistent with the sensory context.

2. The method of claim 1, wherein:

the plant comprises a robotic platform;

responsive to the sensory context comprising a representation of an obstacle, the action comprises an avoidance maneuver executed by the robotic platform; and

responsive to the sensory context comprising a representation of an target, the action comprises an approach maneuver executed by the robotic platform.

3. The method of claim 1, wherein the sensory context is based on sensory input into the learning process, a portion of the sensory input comprising a video sensor data and another portion of the sensory input comprising the predicted plant control output.

4. The method of claim 1, wherein the learning process is configured based on a network of computerized neurons configured to be adapted in accordance with the sensory context and the teaching input.

5. The method of claim 4, wherein:

multiple ones of the computerized neurons are interconnected by connections characterized by connection efficacy; and

the adaptation comprises adapting the connection efficacy of individual connections based on the sensory context and the teaching input.

6. The method of claim 4, wherein the adaptation is based on an error measure between the predicted plant control output and the teaching input.

7. The method of claim 4, wherein:

individual ones of the computerized neurons are communicatively coupled to connections characterized by connection efficacy;

individual ones of the computerized neurons are configured to be operable in accordance with a dynamic process characterized by an excitability parameter;

the sensory context is based on input spikes delivered to into the predictor apparatus via a portion of the connections, individual ones of the input spikes being capable of increasing the excitability parameter associated with individual ones of the computerized neurons; and

the teaching input comprises one or more teaching spikes configured to adjust an efficacy of a portion of the connections, the efficacy adjustment for a given connection providing a portion of the input spikes into a given computerized neuron being configured based on one or more events occurring within a plasticity window, the one or more event including one or more of;

(i) a presence of one or more input spikes on the given connection, (ii) an output being generated by the given neuron, or (iii) an occurrence of at least one of the one or more teaching spikes.

8. The method of claim 7, wherein, responsive to the sensory context being updated at 40 ms intervals, the plasticity window duration is selected between 5 ms and 200 ms, inclusive.

9. The method of claim 4, wherein:

a portion of the computerized neurons comprise spiking neurons, individual ones of the spiking neurons being characterized by a neuron excitability parameter configured to determine an output spike generation by a corresponding spiking neuron;

multiple ones of the spiking neurons being interconnected by second connections characterized by second connection efficacy, individual ones of the second connections being configured to communicate one or more spikes from a pre-synaptic spiking neuron to a post-synaptic spiking neuron; and

a portion of the sensory context is based on sensory input into the learning process comprising one or more spikes.

10. The method of claim 9 wherein the predicted plant control output comprises one or more spikes generated based on spike outputs by individual ones of the spiking neurons.

11. The method of claim 9, wherein the sensory input comprises one or more spikes configured to be communicated by a portion of the connections.

12. The method of claim 9, wherein:

the predicted plant control output comprises a continuous signal configured based on one or more spike outputs by individual ones of spiking neurons; and

the continuous signal includes one or more of an analog signal, a polyadic signal with arity greater than one, an n-bit long discrete signal with n-bits greater than one, a real-valued signal, or a digital representation of a real-valued signal.

13. The method of claim 9, wherein:

the sensory input comprises a continuous signal; and

the continuous signal includes one or more of an analog signal, a polyadic signal with arity greater than 1, an n-bit long discrete signal with n-bits greater than 1, or a real-valued signal, or a digital representation of an analog signal.

14. The method of claim 9, wherein the sensory input comprises a binary signal characterized by a single bit.

15. The method of claim 1, wherein:

the learning process is configured to be updated at regular time intervals; and

the adaptation is based on an error measure between (i) the predicted plant control output generated at a given time instance and (ii) the teaching signal determined at another given time instance prior to the given time instance, the given time instance and the other time instance separated by a duration equal to one of the regular time intervals.

16. The method of claim 1, wherein:

the plant comprises at least one motor comprising a motor interface; and

the predicted plant control output comprises one or more instructions to the motor interface configured to actuate the at least one motor.

17. The method of claim 1, wherein the learning process comprises supervised learning process.

18. The method of claim 1, wherein the predicted plant control output comprises a vector of outputs comprising two or more output components.

19. The method of claim 1, wherein the learning process is configured based one or more a look up table, a hash-table, and a data base table configured to store a relationship between a given sensory context, a teaching input, associated with the given sensory context;

and the predicted plant control output generated for the given sensory context during learning.

20. A computerized predictor apparatus comprising a plurality of computer-readable instructions configured to, when executed, generate a predicted control output by:

configuring the predictor apparatus to operate in accordance with a learning process based on a teaching input; and

based on a sensory context, causing the predictor apparatus to generate the predicted control output;

wherein;

the teaching input comprises the predicted control output.

21. A computerized robotic neuron network control apparatus, comprising:

one or more processors configured to execute computer program modules, the computer program modules comprising;

a first logic module configured to receive a sensory input signal and a teaching signal; and

a second logic module configured to generate a predicted output based on the a sensory input signal and a teaching signal;

wherein the teaching signal comprises another predicted output generated prior to the predicted control output based on the sensory input signal.