Human Augmentation of Robotic Work

• US 20140163730A1
• Filed: 12/05/2013
• Published: 06/12/2014
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: International Electronic Machines Corp.
• Original Assignee: International Electronic Machines Corp.
• Inventors: Mian, Zahid F.
• Granted Patent: US 9,085,080 B2
• Time in Patent Office: Days
• US Class Current: 700/248
• CPC Class Codes: B25J 9/16 Programme controls programme controls in general G05B19/00, e.g. numerical pogramme controls G05B19/18; recording or playback systems G05B19/42
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/39212 Select between autonomous or teleoperation control
• CPC Class Codes: G05B 2219/40153 Teleassistance, operator assists, controls autonomous robot
• CPC Class Codes: G05B 2219/40164 Fault recovery from task execution errors
• CPC Class Codes: G05B 2219/40399 Selection of master-slave operation mode

Abstract

A solution for performing a set of tasks using one or more robotic devices is provided. The robotic device can be configured to perform each task using one or more effector devices, one or more sensor devices, and a hybrid control architecture including a plurality of dynamically changeable levels of autonomy. The levels of autonomy can include: full autonomy of the robotic device, teleoperation of the robotic device by a human user, and at least one level of shared control between the computer system and the human user.

Claims

1. A system comprising:

a robotic device including;

a set of effector devices;

a set of sensor devices; and

a first computer system including at least one computing device, wherein the first computer system implements a machine cognition engine configured to perform a task using the set of effector devices, the set of sensor devices, and a hybrid control architecture including a plurality of dynamically changeable levels of autonomy, wherein the plurality of levels of autonomy include;

full autonomy of the robotic device, teleoperation of the robotic device by a human user, and at least one level of shared control between the computer system and the human user.

2. The system of claim 1, wherein the task includes a plurality of actions, and wherein the first computer system first attempts to perform each of the plurality of actions using the full autonomy level of autonomy.

3. The system of claim 2, wherein, in response to encountering a problem completing an action in the plurality of actions using the full autonomy level of autonomy, the first computer system determines a criticality of the action and performs one of:

adjusts the level of autonomy in response to the action being a critical action or bypasses the action in response to the action being a non-critical action.

4. The system of claim 1, further comprising a second robotic device, wherein the robotic devices work in conjunction with one another to complete a set of tasks.

5. The system of claim 1, further comprising a second computer system in communication with the first computer system, wherein the second computer system includes a database with task data received from a plurality of robotic devices configured to perform the task, and wherein the first computer system is configured to obtain task data from the second computer system to improve an ability of the robotic device to autonomously perform a task.

6. The system of claim 1, wherein the robotic device is a mobile device, and wherein the machine cognition engine is configured to navigate the robotic device in an operating environment to perform the task.

7. The system of claim 6, wherein the operating environment is a rail yard, and wherein the task is a brake bleeding task on a series of rail vehicles in a consist.

8. A method of performing a task using a robotic device, the method comprising:

the robotic device initially attempting to complete an action of the task using a full autonomy level of autonomy;

in response to the robotic device encountering a problem completing the action using the full autonomy level of autonomy, the robotic device;

determining a criticality of the action;

adjusting a level of autonomy in response to the action being a critical action, wherein the adjusted level of autonomy includes at least some human assistance for the robotic device; and

bypassing the action in response to the action being a non-critical action.

9. The method of claim 8, wherein the task includes a plurality of actions, and wherein the robotic device first attempts to perform each of the plurality of actions using the full autonomy level of autonomy.

10. The method of claim 8, the method further comprising the robotic device receiving human assistance to complete the critical action and the robotic device automatically adjusting the level of autonomy to the full autonomy level of autonomy in response evaluating the critical action as being complete.

11. The method of claim 10, the robotic device further providing data corresponding to the human assistance to a central database accessible by a plurality of robotic devices.

12. The method of claim 8, wherein the human assistance includes teleoperation of at least one effector device on the robotic device.

13. The method of claim 12, wherein the teleoperation includes:

the robotic device receiving an action request from the human user;

the robotic device modeling a result of the requested action;

the robotic device performing the requested action in response to the modeled result indicating the requested action is safe; and

the robotic device intervening in the requested action in response to the modeled result indicating the requested action is not safe.

14. The method of claim 8, wherein the robotic device is a mobile device, and wherein the task includes navigating the robotic device in an operating environment.

15. The method of claim 14, wherein the operating environment is a rail yard, and wherein the task is a brake bleeding task on a series of rail vehicles in a consist.

16. A mobile robotic device comprising:

a set of effector devices;

a set of sensor devices; and

a computer system including at least one computing device, wherein the computer system implements a machine cognition engine configured to perform a set of railroad maintenance tasks using the set of effector devices, the set of sensor devices, and a hybrid control architecture including a plurality of dynamically changeable levels of autonomy, wherein the plurality of levels of autonomy include;

full autonomy of the robotic device, teleoperation of the robotic device by a human user, and at least one level of shared control between the computer system and the human user.

17. The robotic device of claim 16, wherein the set of effector devices includes:

an arm including a plurality of degrees of freedom;

a grasping manipulator located adjacent to an end of the arm;

a first imaging device located on a main body of the robotic device; and

a second imaging device located adjacent to the grasping manipulator, wherein the computer system processes image data for the second imaging device to operate the arm and the grasping manipulator.

18. The robotic device of claim 17, wherein the set of railroad maintenance tasks includes a brake bleeding task on a rail vehicle using the arm and the grasping manipulator.

19. The robotic device of claim 16, wherein, in response to encountering a problem completing an action for a task using the full autonomy level of autonomy, the computer system determines a criticality of the action and performs one of:

adjusts the level of autonomy in response to the action being a critical action or bypasses the action in response to the action being a non-critical action.

20. The robotic device of claim 16, wherein the computer system is further configured to provide task data corresponding to completion of an action for the task using human assistance for storage at a central database and obtain external task data corresponding to completion of an action for a task by another robotic device using human assistance from the central database and process the external task data to improve an ability of the robotic device to autonomously perform the action.