ROBOTIC CONTROL ARBITRATION APPARATUS AND METHODS

• US 20150094852A1
• Filed: 09/27/2013
• Published: 04/02/2015
• Est. Priority Date: 09/27/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Laurent, Patryk, Passot, Jean-Baptiste, Izhikevich, Eugene
• Granted Patent: US 9,296,101 B2
• Time in Patent Office: Days
• US Class Current: 700/250
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/40494 Neural network for object trajectory prediction, fuzzy for robot path
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: Y10S 901/03 Teaching system
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

Apparatus and methods for arbitration of control signals for robotic devices. A robotic device may comprise an adaptive controller comprising a plurality of predictors configured to provide multiple predicted control signals based on one or more of the teaching input, sensory input, and/or performance. The predicted control signals may be configured to cause two or more actions that may be in conflict with one another and/or utilize a shared resource. An arbitrator may be employed to select one of the actions. The selection process may utilize a WTA, reinforcement, and/or supervisory mechanisms in order to inhibit one or more predicted signals. The arbitrator output may comprise target state information that may be provided to the predictor block. Prior to arbitration, the predicted control signals may be combined with inputs provided by an external control entity in order to reduce learning time.

Claims

1. A controller apparatus of a robot, comprising:

one or more processors configured to execute computer program modules, the computer program modules comprising;

a first predictor module operable in accordance with a first learning process configured to provide a first and a second predicted action indication configured based on a context, the first predicted action indication being configured to cause a robot to execute a first action, the second predicted action indication being configured to cause the robot to execute a second action;

a second predictor module operable in accordance with a second learning process configured to provide a predicted control output configured based on the context and an input signal, the predicted control output being configured to cause a robot to execute one of the first action or the second action; and

an arbitrator module configured to provide one of the first or the second action indication as the input signal to the second predictor module.

2. The apparatus of claim 1, wherein:

the context comprises (i) a sensory signal conveying information associated with an object within an environment of the robot and (ii) the input signal;

the first action is configured to reduce a distance measure between the robot and the object; and

the second action is configured to increase the distance measure.

3. The apparatus of claim 1, wherein

the context comprises a sensory signal conveying information associated with an object within an environment of the robot;

at least one of the first or the second learning process comprises a supervised learning process configured based on a teaching signal, the supervised learning process being updated at a plurality of time instances; and

the teaching signal is determined based on a measure between (i) a combination of a control signal and at least one of the first or the second predicted action indications determined at first time instance of the plurality of time instances; and

(i) the at least one of the first or the second predicted action indications determined at second time instance of the plurality of time instances, the second time instance occurring prior to the first time instance; and

the control signal is configured in accordance with the information, the control signal being provided to the controller apparatus so as to communicate data related to individual ones of the first and the second actions.

4. The apparatus of claim 3, wherein

the first learning process comprises a first supervised learning process configured based on a first teaching signal, the first supervised learning process being updated at a plurality of time instances;

and

the first teaching signal is determined based on a first performance measure between (i) a combination of a control signal and at least one of the first or the second predicted action indications determined at first time instance of the plurality of time instances; and

(ii) the at least one of the first or the second predicted action indications determined at second time instance of the plurality of time instances, the second time instance occurring prior to the first time instance;

the second learning process comprises a second supervised learning process configured based on a second teaching signal, the second supervised learning process being updated at individual ones of the plurality of time instances; and

the first teaching signal is determined based on (i) the predicted control output determined by the second supervised learning process at the first time instance; and

(ii) the predicted control output determined by the second supervised learning process at the second time instance.

5. The apparatus of claim 3, wherein:

the information is configured to convey one or more of object size, shape, color, type, or location.

6. The apparatus of claim 3, wherein the provision of the first or the second action indication as the input signal to the second predictor module is configured in accordance with a winner-takes-all mechanism configured to preclude one of the first or the second action from entering the input signal.

7. The apparatus of claim 1, wherein:

at least one of the first or the second learning process comprises a supervised learning process configured based on a teaching signal, the supervised learning process being updated at a plurality of time intervals; and

the supervised learning process update is based on an error measure between (i) at least one of the first or the second predicted action indications determined at a given time instance and (ii) the teaching signal determined at another time instance prior to the given time instance, the given time instance and the other time instance separated by one of the time intervals.

8. The apparatus of claim 1, wherein the computer program modules further comprise:

a combiner module configured to provide (1) a first combined output based on a first combination of a control input and the first predicted action indication, and (2) a second combined output based on a second combination of the control input and the second predicted action indication;

wherein the first combined output is configured to cause execution if the first action and the second combined output is configured to cause execution of the second action.

9. The apparatus of claim 8, wherein:

the context comprises a sensory signal conveying information associated with an object within an environment of the robot; and

the first action comprises the object approach action;

the second action comprises the object avoid action;

the first combination includes a transform function configured to combine the control input and the first predicted action indication via one or more operations including an additive operation; and

the second combination includes a transform function configured to combine the control input and the second predicted action indication via one or more operations including an additive operation.

10. The apparatus of claim 8, wherein:

the first predictor module is further configured to provide the first predicted action indication and the second predicted action indication based on a teaching signal and the context in accordance with an adaptive process configured to reduce an error measure between one or both of (1) the first combined output and the first predicted action indication or (2) the second combined output and the second predicted action indication;

the teaching signal comprises at least one of the first predicted action indication or the second predicted action indication;

the teaching signal is provided responsive to the first combination and/or the second combination being determined at first time instance;

the first predictor module is further configured to provide the first predicted action indication and the second predicted action indication at a second time instance subsequent to the first time instance; and

the first predictor module is further configured to receive the context at the first time instance and at the second time instance.

11. The apparatus of claim 1, wherein:

based on execution of the first action being in conflict with contemporaneous execution of the second action, the arbitrator module is configured to relay one and only one of the first predicted action indication or the second predicted action indication to the input signal.

12. The apparatus of claim 11, wherein:

the relaying is configured based on gating at least one of the first predicted action indication or the second predicted action indication by the arbitrator module, the gating preventing execution of the respective action; and

the gating is configured based on applying a zero gain to at least one of the first predicted action indication or the second predicted action indication.

13. An apparatus configured to arbitrate multiple predicted outputs, the apparatus comprising:

one or more processors configured to;

provide an inhibitory signal based on (i) a determination of an object characteristic within an input and (ii) a parameter indicative of an association between the characteristic and one or more actions; and

based on the inhibitory signal, prevent relaying of all but one of the multiple predicted outputs to the input;

wherein;

the inhibitory signal is configured to cause execution of a single action associated with the but one of the multiple predicted outputs;

individual ones of the multiple predicted outputs are configured based on the characteristic and are adapted to cause contemporaneous execution of the one or more actions and the single action; and

the parameter is determined prior to the arbitration of the multiple predicted outputs based on one or more training trials characterized by presence of the object characteristic and the but one of the multiple predicted outputs.

14. The apparatus of claim 13, further comprising:

a first spiking neuron and a second spiking neuron configured to convey a spike from an input connection to an output connection; and

an inhibitory connection configured to communicate inhibitory efficacy from the first spiking neuron to the second spiking neuron;

wherein;

the multiple predicted outputs comprise a first spiking signal configured to convey a first spike and a second spiking signal configured to convey a second spike;

the inhibitory signal comprises an inhibitory spike communicated from the first spiking neuron to the second spiking neuron via the inhibitory connection; and

the inhibitory spike is configured to provide the inhibitory efficacy to the second neuron thereby reducing probability of the conveyance of spikes within the second spiking signal from an input connection to an output connection of the second spiking neuron.

15. The apparatus of claim 14, wherein the first spiking neuron is adapted to provide the inhibitory spike using a winner-takes-all mechanism configured based on an occurrence of the first spike within the first spiking signal prior to the second spike within the second spiking signal.

16. The apparatus of claim 14, wherein the first neuron is adapted to provide the inhibitory spike using a winner-takes-all mechanism configured based on an occurrence of the first spike within the first spiking signal prior to the second spike within the second spiking signal.

17. The apparatus of claim 14, wherein the first spiking neuron is configured to provide the inhibitory spike using a winner-takes-all mechanism configured based on an efficacy associated with the first spike within the first spiking signal exceeding an efficacy associated with the second spike within the second spiking signal.

18. The apparatus of claim 14, wherein:

the second spiking neuron is configured to communicate another inhibitory spike to the first spiking neuron based on an occurrence of another spike within the second spiking signal prior to another spike within the first spiking signal; and

the communication of the other inhibitory spike is configured to provide another inhibitory efficacy to the first neuron thereby reducing probability of the conveyance of spikes within the first spiking signal from an input connection to an output connection of the first spiking neuron.

19. The apparatus of claim 14, wherein:

the first spiking signal is configured to cause execution of the single action;

the second spiking signal is configured to cause execution of an action of the one or more actions;

the one or more actions comprise an object approach action and an object avoid action;

the apparatus is configured to be embodied in a mobile robotic platform;

the characteristic comprises a position measure of the object relative the robotic platform; and

individual ones of the multiple predicted outputs are configured based on a learning process configured to be adapted based on the input and a teaching signal indicative of a target action.

20. A robotic apparatus, comprising:

a platform comprising a motor operable in a first mode and a second mode; and

one or more processors configured to execute computer program modules, the computer program modules comprising;

a sensory processing module configured to detect a parameter of an object in a sensory input;

a controller module operable in accordance with a control policy comprising an object approach action and an object avoid action;

a first learning module configured to provide an approach action indication and an avoid action indication;

a second learning module configured to provide a motor control instruction configured to activate the motor; and

an arbitrator module configured to prevent provision of one of the approach action indication or the avoid action indication to the second learning module based on the parameter.