Apparatus and methods for control of robot actions based on corrective user inputs

• US 10,322,507 B2
• Filed: 10/16/2017
• Issued: 06/18/2019
• Est. Priority Date: 02/03/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Meier, Philip, Passot, Jean-Baptiste, Ibarz Gabardos, Borja, Laurent, Patryk, Sinyavskiy, Oleg, O'Connor, Peter, Izhikevich, Eugene
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/785,181
• Publication Number:
  US 20180099409A1
• Time in Patent Office: 610 Days
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1656 characterised by programming, planning systems for manipulators
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device

Abstract

Robots have the capacity to perform a broad range of useful tasks, such as factory automation, cleaning, delivery, assistive care, environmental monitoring and entertainment. Enabling a robot to perform a new task in a new environment typically requires a large amount of new software to be written, often by a team of experts. It would be valuable if future technology could empower people, who may have limited or no understanding of software coding, to train robots to perform custom tasks. Some implementations of the present invention provide methods and systems that respond to users'"'"' corrective commands to generate and refine a policy for determining appropriate actions based on sensor-data input. Upon completion of learning, the system can generate control commands by deriving them from the sensory data. Using the learned control policy, the robot can behave autonomously.

Claims

1. A system for forming a relationship between data and movement of a device along a path, comprising:

a memory having computer readable instructions stored thereon; and

at least one processor configured to execute the computer readable instructions to;

receive one or more sensor data outputted by one or more sensors and one or more inputs from a user when the device is following a first path, the one or more inputs corresponding to a corrective action such that the device is following a second path different from the first path, the second path being a target path;

access a first set of one or more parameters stored in a control policy upon receiving either the one or more sensor data or the one or more inputs;

modify the first set of the one or more parameters stored in the control policy upon having the one or more inputs identify a second set of the one or more parameters, the second set being different from the first set;

update the one or more parameters in the first set to correspond to the one or more parameters in the second set; and

output a signal to the device based on the update of the one or more parameters in the first set such that the device follows the second path.

2. The system of claim 1, wherein,

the at least one processor is further configured to operate the device in a first mode and a second mode, the first mode corresponds to a learning mode, the second mode corresponds to a non-learning mode;

and

the modification of the first set of the one or more parameters is smaller in the second mode compared to the modification of the first set of the one or more parameters in the first mode.

3. The system of claim 1, wherein:

the at least one processor is further configured to operate the device in a first mode and a second mode, the first mode corresponds to a learning mode, the second mode corresponds to a non-learning mode; and

the one or more parameters in the first set are updated in the second mode if the first path matches the second path.

4. The system of claim 1, wherein the at least one processor is further configured to generate a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the proposed action includes the first set of the one or more parameters.

5. The system of claim 4, wherein the at least one processor is further configured to associate a weight above a threshold to the target action, the weight above the threshold corresponds to a desired modification of the first set of the one or more parameters.

6. The system of claim 5, wherein the at least processor is further configured to associate a weight below a threshold to the target action, the weight below the threshold corresponds to either insignificant or no modification of the first set of the one or more parameters.

7. The system of claim 1, wherein the at least one processor is further configured to generate a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the target action is the proposed action if a value of the one or more inputs is null.

8. A non-transitory computer readable medium having computer readable instructions stored thereon, that when executed by at least processor cause the at least one processor to:

receive one or more sensor data outputted by one or more sensors and one or more inputs from a user when the device is following a first path, the one or more inputs corresponding to a corrective action such that the device is following a second path different from the first path, the second path being a target path;

access a first set of one or more parameters stored in a control policy upon receiving either the one or more sensor data or the one or more inputs;

modify the first set of the one or more parameters stored in the control policy upon having the one or more inputs identify a second set of the one or more parameters, the second set being different from the first set;

update the one or more parameters in the first set to correspond to the one or more parameters in the second set; and

output a signal to the device based on the update of the one or more parameters in the first set such that the device is following the second path.

9. The non-transitory computer readable medium of claim 8, wherein:

the at least one processor is further configured to operate the device in a first mode and a second mode, the first mode corresponds to a learning mode, the second mode corresponds to a non-learning mode; and

the modification of the first set of the one or more parameters is smaller in the second mode compared to the modification of the first set of the one or more parameters in the first mode.

10. The non-transitory computer readable medium of claim 8, wherein,

the at least one processor is further configured to operate the device in a first mode and a second mode, the first mode corresponds to a learning mode, the second mode corresponds to a non-learning mode;

and

the one or more parameters in the first set are updated in the second mode if the first path matches the second path.

11. The non-transitory computer readable medium of claim 8, wherein the at least one processor is further configured to generate a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the proposed action includes the first set of the one or more parameters.

12. The non-transitory computer readable medium of claim 11, wherein the at least one processor is further configured to associate a weight above a threshold to the target action, the weight above the threshold corresponds to a desired modification of the first set of the one or more parameters.

13. The non-transitory computer readable medium of claim 12, wherein the at least processor is further configured to associate a weight below a threshold to the target action, the weight below the threshold corresponds to either insignificant or no modification of the first set of the one or more parameters.

14. The non-transitory computer readable medium of claim 8, wherein the at least one processor is further configured to generate a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the target action is the proposed action if a value of the one or more inputs is null.

15. A method for forming a relationship between data and movement of a device along a path, comprising:

receiving one or more sensor data outputted by one or more sensors and one or more inputs from a user when the device is following a first path, the one or more inputs corresponding to a corrective action such that the device is following a second path different from the first path, the second path being a target path;

accessing a first set of one or more parameters stored in a control policy upon receiving either the one or more sensor data or the one or more inputs;

modifying the first set of the one or more parameters stored in the control policy upon having the one or more inputs identify a second set of the one or more parameters, the second set being different from the first set;

updating the one or more parameters in the first set to correspond to the one or more parameters in the second set; and

outputting a signal to the device based on the update of the one or more parameters in the first set such that the device is following the second path.

16. The method of claim 15, further comprising:

operating the device in a first mode and a second mode, the first mode corresponding to a learning mode, the second mode corresponding to a non-learning mode;

wherein the modifying of the first set of the one or more parameters is smaller in the second mode compared to the modification of the first set of the one or more parameters in the first mode.

17. The method of claim 15, further comprising:

operating the device in a first mode and a second mode, the first mode corresponding to a learning mode, the second mode corresponding to a non-learning mode;

wherein the one or more parameters in the first set are updated in the second mode if the first path matches the second path.

18. The method of claim 15, further comprising:

generating a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the proposed action includes the first set of the one or more parameters.

19. The method of claim 18, further comprising:

associating a weight above a threshold to the target action, the weight above the threshold corresponds to a desired modification of the first set of the one or more parameters.

20. The method of claim 15, further comprising:

generating a target action for the device such that the target action is a sum of a proposed action and the one or more inputs, the target action is the proposed action if a value of the one or more inputs is null.