Apparatus and methods for programming and training of robotic devices

• US 10,105,841 B1
• Filed: 02/03/2015
• Issued: 10/23/2018
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Szatmary, Botond, Grotmol, Oyvind, Izhikevich, Eugene, Sinyavskiy, Oleg
• Primary Examiner(s): Smith, Jelani A
• Assistant Examiner(s): Alsomiri, Majdi
• Application Number:
  US14/613,237
• Time in Patent Office: 1,358 Days
• Field of Search: 701 36
• CPC Class Codes: B25J 9/0081 with master teach-in means
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/1607 Calculation of inertia, jacobian matrixes and inverses
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/00 Computing arrangements based on biological models
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/03 Teaching system
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Apparatus and methods for training and operating of robotic devices. Robotic controller may comprise a plurality of predictor apparatus configured to generate motor control output. One predictor may be operable in accordance with a pre-configured process; another predictor may be operable in accordance with a learning process configured based on a teaching signal. An adaptive combiner component may be configured to determine a combined control output controller block may provide control output that may be combined with the predicted control output. The pre-programmed predictor may be configured to operate a robot to perform a task. Based on detection of a context, the controller may adaptively switch to use control output of the learning process to perform the given or another task. User feedback may be utilized during learning.

Claims

1. An apparatus for controlling a robot, the apparatus comprising:

a sensor interface configured to receive a first representation of an object associated with an environment of the robot;

a predictor component configured to determine a first control output, the first control output being configured to cause the robot to execute a task in accordance with the first representation of the object;

an input interface configured to receive a teaching input associated with a second representation of the object, the teaching input being configured to convey information related to a target trajectory associated with execution of the task by the robot;

an evaluation component configured to provide a second control output based on an evaluation of the first control output and the teaching input;

a learning predictor component configured to determine a third control output, the third control output being determined based on analysis of the first representation of the object and the second control output; and

a combiner configured to combine the first control output and the third control output to produce a fourth control output, the fourth control output being configured to cause the robot to execute the task in accordance with the first representation of the object;

wherein;

execution of the task based on the fourth control output produces a first trajectory that is closer to the target trajectory compared to a second trajectory associated with execution of the task based on the first control output.

2. The apparatus of claim 1, wherein:

the learning predictor component is operable in accordance with a learning process, the learning process being configured to determine an association between the first representation of the object and the second control output; and

the learning process is characterized by a learning configuration that is adapted based on a first occurrence of the first representation of the object contemporaneously with provisioning of the second control output.

3. The apparatus of claim 2, wherein:

the learning process comprises a look up table comprising a plurality of entries; and

the determination of the third control output by the learning process increments an entry of the plurality of entries based on a second occurrence of the first representation of the object contemporaneously with provisioning of the third control output.

4. The apparatus of claim 2, wherein:

the learning predictor component is further configured to provide to the learning process a confidence information associated with the first control output;

the learning process comprises a look up table comprising a plurality of entries; and

the determination of the third control output by the learning process increments an entry of the plurality of entries based on a second occurrence of the first representation of the object contemporaneously with provisioning of the second control output where the confidence information satisfies a given condition.

5. The apparatus of claim 2, wherein:

the first representation of the object, the first control output, and the second control output comprise a plurality of features of a first type and one or more features of a second type; and

the determination of the third control output is effectuated by the learning process that comprises logic configured to;

determine a subset of features by random selection of a portion of the plurality of features and at least one feature from the second input features;

compare individual features of the subset to corresponding features of a plurality of training feature sets, individual ones of the plurality of training feature sets comprising a number of training features, the number being equal to or greater than the quantity of features within the subset of features;

based on the comparison, determine a similarity measure for a given training set of the plurality of training feature sets, the similarity measure characterizing a similarity between features of the subset and features of the given training set;

responsive to the similarity measure breaching a threshold, select one or more training sets from the plurality of training sets;

determine one or more potential control outputs, individual ones of the one or more potential control outputs being associated with a corresponding training set of the plurality of training sets; and

determine the third control output based on a transformation obtained from the one or more potential control outputs;

wherein;

individual ones of the plurality of training feature sets comprise features of the first type and at least one feature of the second type;

individual ones of the plurality of training feature sets are obtained during training operation of the robot, the training operation being performed responsive to receiving the teaching input from the robot; and

individual ones of the one or more potential control outputs determined based on the teaching input and the features of the given training set.

6. The apparatus of claim 5, wherein the similarity measure is determined based on a difference between values of the features of the subset and values of the features of the given training set.

7. The apparatus of claim 5, wherein the similarity measure is determined based on a distance between individual features of the subset of features and corresponding features of the given training set.

8. The apparatus of claim 1, wherein execution of the task based solely on the third control output is configured to produce a third trajectory that is farther from the target trajectory compared to the second trajectory.

9. The apparatus of claim 1, wherein:

the robot comprises a vehicle, the vehicle comprising a platform configured to move in at least one dimension;

the apparatus is disposed on the platform; and

the teaching input is provided by an entity disposed external to the platform.

10. The apparatus of claim 9, further comprising:

a wireless communications interface configured to receive remote data transmissions from the entity;

wherein;

the teaching input is provided via the remote data transmissions; and

the teaching input is configured based on an evaluation of the second representation of the object, the second representation being distinct from the first representation.

11. The apparatus of claim 1, wherein the evaluation component is configured to produce the second control output based on a discrepancy between the first control signal and the teaching input.

12. The apparatus of claim 11, wherein the combiner is operable in accordance with at least one of an addition or a union operation on the first control output and the third control output.

13. The apparatus of claim 1, wherein the combiner is configured based on a concatenation of the first control output and the third control output.

14. The apparatus of claim 1, further comprising:

another combiner component configured to combine the fourth control output and the teaching input to produce a motor control output, the combination of the fourth control output and the teaching input being characterized by a transfer function;

wherein;

the robot comprises an actuator configured to displace at least a portion of the robot in at least one dimension based on application of the motor control output; and

the transfer function is configured to provide an override combination wherein the a motor control output is configured based solely on the teaching signal satisfying a given condition.

15. A method of determining a combined control output for a task being executed by a robot, the method comprising:

for a sensory context, determining a first instance of a control output using a pre-configured and non-learning prediction process and a second instance of the control output using a learning prediction process;

combining the first instance of the control output with the second instance of the control output using a combination process to produce a combined control output; and

causing the task to be executed responsive to providing the combined control output to the robot;

wherein;

the learning prediction process is configured to determine the second instance of the control output based on a teaching input indicative of a target trajectory associated with the task execution; and

execution of the task by the robot in accordance with the combined control output is configured to produce a trajectory that is closer to the target trajectory compared to task execution based on either the first instance of the control output or the second instance of the control output.

16. The method of claim 15, wherein:

the learning prediction process is configured to associate the sensory context with the target trajectory, the association being based on updating a parameter of the learning prediction process based on a contemporaneous occurrence of the sensory context and the teaching input.

17. The method of claim 16, wherein:

the robot comprises a plurality of actuators characterized by a first operational degree of freedom and a second operational degree of freedom;

the first instance of the control output is configured for operation within the first operational degree of freedom and the second instance of the control output is configured for operation within the second operational degree of freedom; and

the combination process comprises a concatenation operation.

18. A method of determining a control signal for a robot, the method being performed by one or more processors executing instructions stored by a non-transitory computer-readable storage medium, the method comprising:

determining an occurrence of a first context in sensory input;

accessing a learning process configuration, the learning process configuration adapted to convey an association between a given context and a respective action;

determining a first action associated with the first context;

responsive to the first action corresponding to a pre-programmed action, activating a pre-programmed predictor component to produce the control signal based on analysis of the sensory input;

responsive to the first action corresponding to a learned action, activating a learning predictor component to produce the control signal based on analysis of the sensory input and a training input; and

updating the learning process configuration in accordance with the activated learning predictor component.

19. The method of claim 18, wherein:

the robot comprises an autonomous vehicle and a collector apparatus;

the pre-programmed action comprises an object search task configured based on a random exploration of environment by the robot; and

the learned action comprises an object collection task configured based on the training input.