System and method for seamless task-directed autonomy for robots

• US 8,271,132 B2
• Filed: 03/13/2008
• Issued: 09/18/2012
• Est. Priority Date: 03/13/2008
• Status: Active Grant

• Current Assignee: Humatics Corporation
• Original Assignee: Battelle Energy Alliance LLC (Battelle Memorial Institute)
• Inventors: Nielsen, Curtis, Bruemmer, David, Few, Douglas, Walton, Miles
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): Olsen, Lin B
• Application Number:
  US12/048,110
• Publication Number:
  US 20090234499A1
• Time in Patent Office: 1,650 Days
• Field of Search: None
• US Class Current: 700/250
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/33051 BBC behavior based control, stand alone module, cognitive, independent agent
• CPC Class Codes: G05B 2219/35472 Mode selection
• CPC Class Codes: G05B 2219/39212 Select between autonomous or teleoperation control
• CPC Class Codes: G05B 2219/40442 Voxel map, 3-D grid map
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

Systems, methods, and user interfaces are used for controlling a robot. An environment map and a robot designator are presented to a user. The user may place, move, and modify task designators on the environment map. The task designators indicate a position in the environment map and indicate a task for the robot to achieve. A control intermediary links task designators with robot instructions issued to the robot. The control intermediary analyzes a relative position between the task designators and the robot. The control intermediary uses the analysis to determine a task-oriented autonomy level for the robot and communicates target achievement information to the robot. The target achievement information may include instructions for directly guiding the robot if the task-oriented autonomy level indicates low robot initiative and may include instructions for directing the robot to determine a robot plan for achieving the task if the task-oriented autonomy level indicates high robot initiative.

Claims

1. A graphical user interface for controlling a robot, comprising:

an environment map window configured for displaying a map of an environment proximate the robot;

a robot designator configured for showing a robot position and a robot pose in the environment map window;

at least one task designator configured for positioning by a user in the environment map window and indicating a task for the robot to achieve; and

a control intermediary configured for linking user defined tasks with robot instructions by;

analyzing a position of the at least one task designator relative to a position of at least one robot component;

determining a task-oriented autonomy level for the robot responsive to the analysis; and

communicating target achievement information to the robot, wherein the target achievement information comprises;

instructions from the control intermediary for guiding the robot to achieve the task if the task-oriented autonomy level comprises low robot initiative; and

instructions from the control intermediary directing the robot to determine a robot plan for achieving the task if the task-oriented autonomy level comprises high robot initiative.

2. The graphical user interface of claim 1, wherein the map is selected from the group consisting of a two-dimensional map and a three-dimensional map.

3. The graphical user interface of claim 1, wherein the at least one task designator includes position information selected from the group consisting of two-dimensional information and three-dimensional information.

4. The graphical user interface of claim 1, wherein the at least one task designator includes task attribute information.

5. The graphical user interface of claim 1, wherein the at least one task designator is selected from the group consisting of a navigation target, an imaging target, an artillery target, a sensor target, and a manipulator target.

6. The graphical user interface of claim 1, wherein analyzing a position further comprises analyzing a change in the at least one task designator, wherein the change is selected from the group consisting of:

a time interval between movements of the at least one task designator;

a distance between the robot and the at least one task designator;

an event horizon relative to the at least one task designator;

a change from one task designator to another task designator; and

combinations thereof.

7. The graphical user interface of claim 1, wherein instructions from the control intermediary for guiding the robot include instructions for robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

8. A method for controlling a robot, comprising:

providing a user interface for controlling the robot;

positioning at least one task-oriented target in a map on the user interface representing an environment of the robot;

determining a task-oriented autonomy level of the robot correlated to a change in the at least one task-oriented target;

if the change in the at least one task-oriented target is smaller than a change threshold, instructing the robot to achieve the at least one task-oriented target by intervention instructions from the user interface; and

if the change in the at least one task-oriented target is larger than the change threshold, instructing the robot to achieve the at least one task-oriented target using robot initiative to determine a robot plan for achieving the at least one task-oriented target and implementing the robot plan.

9. The method of claim 8, further comprising selecting the at least one task-oriented target from the group consisting of a navigation target, an imaging target, an artillery target, a sensor target, and a manipulator target.

10. The method of claim 8, wherein the change in the at least one task-oriented target is selected from the group consisting of:

a time interval between movements of the at least one task-oriented target;

a distance between the robot and the at least one task-oriented target;

an event horizon relative to the at least one task-oriented target;

a change from one task-oriented target to another task-oriented target; and

combinations thereof.

11. The method of claim 8, wherein the user interface adjusts the task-oriented autonomy level between a level selected from the group consisting of:

a teleoperation mode configured to maximize a user interface intervention and minimize the robot initiative;

a safe mode configured to include less of the user interface intervention and more of the robot initiative relative to the teleoperation mode;

a shared mode configured to include less of the user interface intervention and more of the robot initiative relative to the safe mode;

a collaborative tasking mode configured to include less of the user interface intervention and more of the robot initiative relative to the shared mode; and

an autonomous mode configured to minimize the user interface intervention and maximize the robot initiative.

12. The method of claim 8, wherein the intervention instructions include instructions for robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

13. The method of claim 8, wherein the robot plan includes robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

14. The method of claim 8, wherein positioning the at least one task-oriented target in the map is performed by a user and the intervention instructions are determined by the user interface responsive to the change in the at least one task-oriented target.

15. A method for controlling a robot, comprising:

receiving instructions for achieving at least one task-oriented target from a user interface, the instructions comprising at least one of intervention instructions, robot initiative instructions, and instructions for setting a task-oriented autonomy level;

if the instructions are the robot initiative instructions, then developing a robot plan to achieve the at least one task-oriented target and performing the robot plan; and

if the instructions are the intervention instructions, then performing the intervention instructions for achieving the at least one task-oriented target and, if present, overriding the robot plan to achieve the at least one task-oriented target.

16. The method of claim 15, further comprising reporting the robot plan to the user interface.

17. The method of claim 15, wherein the intervention instructions include instructions for robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

18. The method of claim 15, wherein the robot plan includes robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

19. The method of claim 15, further comprising returning to the performing the robot plan after performing the intervention instructions if the user interface modifies the task-oriented autonomy level to provide additional robot initiative.

20. The method of claim 15, further comprising selecting the at least one task-oriented target from the group consisting of a navigation target, an imaging target, an artillery target, a sensor target, and a manipulator target.

21. The method of claim 15, wherein the user interface adjusts the task-oriented autonomy level between a level selected from the group consisting of:

a teleoperation mode configured to maximize a user interface intervention and minimize the robot initiative;

a safe mode configured to include less of the user interface intervention and more of the robot initiative relative to the teleoperation mode;

a shared mode configured to include less of the user interface intervention and more of the robot initiative relative to the safe mode;

a collaborative tasking mode configured to include less of the user interface intervention and more of the robot initiative relative to the shared mode; and

an autonomous mode configured to minimize the user interface intervention and maximize the robot initiative.

22. A robot platform, comprising:

at least one perceptor configured for perceiving environmental variables of interest;

at least one locomotor configured for providing mobility to the robot platform; and

a system controller configured for executing a task-oriented autonomy system, comprising;

receiving instructions from a user interface using a communication interface, the instructions for achieving at least one task-oriented target and comprising at least one of intervention instructions, robot initiative instructions, and instructions for setting a task-oriented autonomy level;

if the instructions are the robot initiative instructions, developing a robot plan to achieve the at least one task-oriented target and performing the robot plan; and

if the instructions are the intervention instructions, then performing the intervention instructions for achieving the at least one task-oriented target from the user interface and, if present, overriding the robot plan to achieve the at least one task-oriented target.

23. The robot platform of claim 22, further configured for reporting the robot plan through the communication interface to the user interface.

24. The robot platform of claim 22, wherein the system controller is further configured for returning to performing the robot plan after performing the intervention instructions if the user interface modifies the task-oriented autonomy level to provide additional robot initiative.

25. The robot platform of claim 22, wherein the system controller is further configured for returning to performing the robot plan after performing the intervention instructions if the robot determines that additional robot initiative is appropriate and the user interface enables a change in autonomy initiated by the robot.

26. The robot platform of claim 22, wherein the at least one task-oriented target is selected from the group consisting of a navigation target, an imaging target, an artillery target, a sensor target, and a manipulator target.

27. The robot platform of claim 22, wherein the robot plan includes robot conduct and robot behavior selected from the group consisting of focus behaviors, manipulation behaviors, go-to points, waypoints, path-planning, search region, patrol region, retro-traverse, and laser tracking.

28. A robot control system, comprising:

a user computer comprising a memory and at least one processor configured for executing a user interface, the user interface configured for;

positioning at least one task-oriented target in a map on the user interface representing an environment of the robot;

determining a task-oriented autonomy level of the robot correlated to a change in the at least one task-oriented target;

if the change in the at least one task-oriented target is smaller than a change threshold, instructing the robot to achieve the at least one task-oriented target by intervention instructions from the user interface; and

if the change in the at least one task-oriented target is larger than the change threshold, instructing the robot to achieve the at least one task-oriented target using robot initiative to determine a robot plan for achieving the at least one task-oriented target and implementing the robot plan; and

a robot platform comprising a locomotor and a system controller, the system controller configured for;

developing the robot plan to achieve the at least one task-oriented target and performing the robot plan if robot initiative instructions are received from the user computer; and

performing the intervention instructions for achieving the at least one task-oriented target and, if appropriate, overriding the robot plan to achieve the at least one task-oriented target if the intervention instructions are received from the user computer.

29. The system of claim 28, wherein the at least one task-oriented target is selected from the group consisting of a navigation target, an imaging target, an artillery target, a sensor target, and a manipulator target.

30. The system of claim 28, wherein the change in the at least one task-oriented target is selected from the group consisting of:

a time interval between movements of the at least one task-oriented target;

a distance between the robot and the at least one task-oriented target;

an event horizon relative to the at least one task-oriented target;

a change from one task-oriented target to another task-oriented target; and

combinations thereof.

31. The system of claim 28, wherein positioning the at least one task-oriented target in the map is performed by a user and the intervention instructions are derived by the user interface responsive to the change in the at least one task-oriented target.