Predictive robotic controller apparatus and methods

• US 9,314,924 B1
• Filed: 06/14/2013
• Issued: 04/19/2016
• Est. Priority Date: 06/14/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Laurent, Patryk, Passot, Jean-Baptiste, Sinyavskiy, Oleg, Ponulak, Filip, Gabardos, Borja Ibarz, Izhikevich, Eugene
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Peche, Jorge
• Application Number:
  US13/918,620
• Time in Patent Office: 1,040 Days
• Field of Search: 700/245, 700/250, 700/255, 700/257, 700/264, 901/1, 901/2, 901/4
• US Class Current: 1/1
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/39271 Ann artificial neural network, ffw-nn, feedforward neural network
• CPC Class Codes: G05B 2219/39289 Adaptive ann controller
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

Robotic devices may be trained by a user guiding the robot along target action trajectory using an input signal. A robotic device may comprise an adaptive controller configured to generate control signal based on one or more of the user guidance, sensory input, performance measure, and/or other information. Training may comprise a plurality of trials, wherein for a given context the user and the robot'"'"'s controller may collaborate to develop an association between the context and the target action. Upon developing the association, the adaptive controller may be capable of generating the control signal and/or an action indication prior and/or in lieu of user input. The predictive control functionality attained by the controller may enable autonomous operation of robotic devices obviating a need for continuing user guidance.

Claims

1. A method of training a computerized robotic apparatus having motorized operational elements to perform a target action based on a sensory context, the training being effectuated via a plurality of iterations, the method comprising:

during a first portion of the plurality of iterations;

causing the apparatus to perform a first action based on a first control signal generated based on a sensory context and a target action indication provided by a user; and

causing the apparatus to adjust a controller learning parameter based on a first performance value determined based on the first action and the target action; and

during a second portion of the plurality of iterations, the second portion being subsequent to the first portion;

causing the apparatus to generate an action signal based on the sensory context and the adjusted controller learning parameter, the action signal generation occurring prior to a subsequent provision of the target action indication associated with a subsequent iteration; and

causing the apparatus to perform a second action based on a second control signal generated based on the sensory context and the action signal but absent the provision of the target action indication by the user; and

wherein;

the action performance based on the second control signal is characterized by a second performance value, where the second performance value is improved compared to the first performance value;

where the subsequent provision of the target action indication by the user during the subsequent iteration of the plurality of iterations is configured to cause execution of another action characterized by a third performance value;

the third performance value is lower than the second performance value;

where the target action, the first action, the second action, and the another action, each comprise rotating of the motorized operational elements of the computerized robotic apparatus; and

where the first, second, and third performance are determined based at least in part on the sensory context.

2. The method of claim 1, wherein:

the third performance value is determined by a controller; and

the third performance value is lower than the first performance value.

3. The method of claim 1, wherein:

the controller learning parameter adjustment is adjusted based on a supervised learning process configured based on the sensory context and a combination of a control signal and a user input; and

the control signal comprises the first control signal or the second control signal.

4. The method of claim 3, wherein the user input comprises the target action indication.

5. The method of claim 1, wherein:

the sensory context comprises an object representation; and

the target action comprises at least one of an object approach maneuver or an object avoidance maneuver.

6. The method of claim 1, wherein:

individual ones of the first portion of the plurality of iterations are characterized by a time interval between an onset of the sensory context and the provision of the user input; and

a time period between the onset of the sensory context during the second portion of the plurality of iterations and the action signal generation is no greater than a minimum value of the time interval.

7. The method of claim 1, wherein:

individual ones of the first portion of the plurality of iterations are characterized by a time interval between an onset of the sensory context and the provision of the user input; and

a time period between the onset of the sensory context during the second portion of the plurality of iterations and the action signal generation is no less than at least one of a mean value of the time interval or a median value of the time interval.

8. The method of claim 1, wherein:

a concluding iteration of the first portion of the plurality of iterations is characterized by a concluding target action signal being provided by the user, the concluding iteration being the last iteration of the first portion of the plurality of iterations, the concluding target action being a target action associated with the concluding iteration; and

the generation of the action signal is characterized by the absence of another target action signal being provided by the user within a time period subsequent to time of the concluding target action signal provision and a time of the action signal generation.

9. The method of claim 1, wherein:

the first control signal comprises a combination of the target action indication provided by the user and a predicted control signal generated by the apparatus;

an action performance during individual ones of the first portion of the plurality of iterations is configured based on the combination of the target action indication provided by the user and the predicted control signal generated by the apparatus; and

performing the action based solely on the target action indication provided by the user or the predicted control signal is characterized by a fourth performance value and a fifth performance value, respectively, the fourth performance value and the fifth performance value being lower than the first performance value.

10. The method of claim 1, wherein:

the first performance value is determined based on a first proximity measure between the first action and a target action; and

the second action is characterized by the second performance value determined based on a second proximity measure between the second action and the target action.

11. The method of claim 1, further comprising:

analyzing individual ones of a plurality of first control signals comprises determining a deviation between the first action and the target action;

wherein;

the target action corresponds to operation of the apparatus based on the user; and

the analysis is configured to cause modification of a controller state in accordance with a learning process, the learning process being configured based on a performance measure.

12. The method of claim 1, further comprising:

determining a predicted control output based on a characteristic of the sensory context the sensory context conveying information associated with one or both of an environment of the robotic apparatus and a platform state;

wherein;

the first control signal and the second control signal are configured based on the sensory signal; and

the first action is configured based on the predicted control output.

13. The method of claim 12, further comprising:

providing a table configured to store a plurality of teaching inputs, a plurality of sensory signal characteristics, and a plurality of predicted control outputs; and

selecting a given predicted control output based on a match between a given characteristic of the sensory context and an individual one of the plurality of sensory signal characteristics.

14. The method of claim 13, wherein the given predicted control output is configured based on a search of the table, the search being configured based on the plurality teaching inputs, individual ones of the plurality of teaching inputs comprising a target action indication provided by the user.

15. The method of claim 12, further comprising determining a combined output based on the given predicted control output and the target action indication provided by the user the combined output being characterized by a transform function configured to combine the predicted control output and the target action indication provided by the user via one or more operations including an additive operation.

16. The method of claim 15, wherein the transform function is configured to combine the predicted control output and the target action indication provided by the user via one or more operations including a union operation.

17. The method of claim 15, wherein:

the adjusting the controller learning parameter is configured based on a supervised learning process configured based on the combined output;

the supervised learning process is configured to be updated at a plurality of time intervals; and

the adjusting the controller learning parameter comprises determining an error measure between (i) the predicted control output generated at a first time instance and (ii) the target action indication provided by the user at second time instance subsequent to the first time instance, the first time instance and the second time instance separated by one of the time intervals.

18. The method of claim 1, wherein:

the user comprises a human trainer;

the second control signal is configured to cause a corrective action by the apparatus, the corrective action being characterized by a lower deviation from the target action;

the corrective action is effectuated based on a cooperative interaction with the human, the cooperative interaction being characterized by a plurality of iterations;

the first control signal corresponds to a first given iteration of the plurality of iterations; and

the second control signal corresponds to a second given iteration of the plurality of iterations, the second given iteration occurring subsequent to the first given iteration.

19. The method of claim 18, further comprising:

determining a plurality of predicted control outputs based on a characteristic of a sensory signal and a given user input;

wherein;

the sensory signal is configured to convey information about one or both of an environment of the robotic apparatus and a plant state;

the first action is configured based on a first predicted control output of the plurality of predicted control outputs, the first predicted control output corresponding to the first target action indication by the user;

the corrective action is configured based on a second predicted control output of the plurality of predicted control outputs, the second predicted control output corresponding to the second target action indication by the user;

the corrective action being characterized by an improved performance as compared to the first action and the target action;

the improved performance being quantified based on a lower deviation of the corrected action from the target action compared to a deviation between the first action and the target action; and

the target action being based solely on the target action indication by the user absent a predicted control output of the plurality of predicted control outputs.

20. The method of claim 1, wherein the second control signal is generated by the controller responsive to a confirmation provided by the user.