Apparatus and methods for haptic training of robots

• US 9,844,873 B2
• Filed: 03/20/2017
• Issued: 12/19/2017
• Est. Priority Date: 11/01/2013
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Ponulak, Filip, Kazemi, Moslem, Laurent, Patryk, Sinyavskiy, Oleg, Izhikevich, Eugene
• Primary Examiner(s): Jen, Ian
• Application Number:
  US15/464,104
• Publication Number:
  US 20170232613A1
• Time in Patent Office: 274 Days
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/36418 Modify trajectory by operator gesture, gesture force sensed by end effector
• CPC Class Codes: G05B 2219/36425 Move manually, touch surface, record position
• CPC Class Codes: G05B 2219/40499 Reinforcement learning algorithm
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

Robotic devices may be trained by a trainer guiding the robot along a target trajectory using physical contact with the robot. The robot may comprise an adaptive controller configured to generate control commands based on one or more of the trainer input, sensory input, and/or performance measure. The trainer may observe task execution by the robot. Responsive to observing a discrepancy between the target behavior and the actual behavior, the trainer may provide a teaching input via a haptic action. The robot may execute the action based on a combination of the internal control signal produced by a learning process of the robot and the training input. The robot may infer the teaching input based on a comparison of a predicted state and actual state of the robot. The robot'"'"'s learning process may be adjusted in accordance with the teaching input so as to reduce the discrepancy during a subsequent trial.

Claims

1. A method of operating a robot comprising:

determining a first control signal associated with an environmental context of the robot;

causing, based at least in part on the determined first control signal, the robot to perform a task characterized by a target trajectory; and

responsive to observing a discrepancy between an actual trajectory and the target trajectory, adjusting the actual trajectory based at least in part on a physical contact by an operator with the robot, wherein adjusting of the actual trajectory comprises determining a second control signal configured to cooperate with the first control signal to cause the robot to transition the actual trajectory towards the target trajectory in a subsequent performance of the task in the environmental context.

2. The method of claim 1, further comprising performing a learning process that associates the first control signal with the environmental context.

3. The method of claim 2, wherein determining the second control signal further comprises modifying the learning process.

4. The method of claim 2, wherein the learning process comprises a supervised learning process configured to adjust based at least in part on a teaching signal comprising a prior motor control output and a motor command correction.

5. The method of claim 2, wherein:

the learning process is configured based on a teaching signal; and

the modifying of the learning process is configured based on the teaching signal being determined based on an evaluation of the adjusting of the actual trajectory.

6. The method of claim 1, wherein the physical contact comprises at least one of releasing, moving, manipulating, interacting with, and touching the robot.

7. The method of claim 1, further comprising:

determining as an output of a learning process the first and second control signals based at least in part on the environmental context of the robot; and

causing, based on the determined first and second control signal, the robot to perform the task characterized by the target trajectory.

8. The method of claim 1, further comprising determining the environmental context of the robot based at least in part on a sensory input.

9. A robot apparatus, comprising:

one or more actuators configured to maneuver the robot apparatus;

a sensor module configured to convey information related to an environment of the robot apparatus; and

an adaptive controller operable in accordance with a learning process configured to;

guide the robot apparatus using the one or more actuators to a target state in accordance with the information;

determine a discrepancy between a target trajectory that corresponds to the target state and a current trajectory that corresponds to a current state, the determination based at least in part on a physical contact by a user; and

update the learning process based on the determined discrepancy, wherein the updated learning process comprises a determination of a correction signal to guide the robot apparatus using the one or more actuators to the target state based on a subsequent conveyance of information by the sensor module.

10. The apparatus of claim 9, wherein the learning process is configured in accordance with a teaching signal.

11. The apparatus of claim 10, wherein the guiding of the robot apparatus using the one or more actuators to the target state is configured based on a control signal determined by the learning process in accordance with the conveyed information, and the teaching signal is configured based on the correction signal.

12. The apparatus of claim 11, wherein the teaching signal is inferred based at least in part on a comparison between the current state and the target state.

13. The apparatus of claim 9, wherein the physical contact comprises at least one of releasing, moving, manipulating, interacting with, and touching the robot apparatus.

14. The apparatus of claim 9, wherein the target state is a first pose of the robot apparatus and the current state is a second pose of the robot apparatus.

15. The apparatus of claim 9, wherein the robot apparatus is an autonomous vehicle.

16. A non-transitory computer readable medium comprising a plurality of instruction which, when executed by one or more processors, effectuate control of a robotic apparatus by:

based on a context, determine a first control signal configured to transition the robotic apparatus to a first state;

determine a discrepancy between a current trajectory associated with a current state, and a first trajectory associated with the first state, where the discrepancy between the trajectories comprises a measurable difference; and

determine a second control signal based on the discrepancy, the second control signal configured to transition the robotic apparatus to the current state.

17. The non-transitory computer readable medium of claim 16, wherein the determination of the first control signal and the determination of the second control signal are configured in accordance with a learning process.

18. The non-transitory computer readable medium of claim 17, wherein:

a change in the context is configured to cause an adaptation of the learning process, the adaptation being configured to produce another version of a control signal; and

the context is configured to convey information related to one or more of a sensory input, a robot state, and the teaching signal.

19. The non-transitory computer readable medium of claim 16, wherein:

individual ones of the current state and the first state are characterized by a state parameter; and

the determination of the discrepancy is configured based on an evaluation of a distance measure between the state parameter of the current state and the state parameter of the first state.

20. The non-transitory computer readable medium of claim 16, wherein the discrepancy is based at least in part on a physical contact by a user.