Human and robotic distributed operating system (HaRD-OS)

• US 9,457,468 B1
• Filed: 05/21/2015
• Issued: 10/04/2016
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: InVia Robotics Inc.
• Original Assignee: InVia Robotics, LLC
• Inventors: Elazary, Lior, Parks, Daniel Frank II, Voorhies, Randolph Charles
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US14/719,196
• Time in Patent Office: 502 Days
• Field of Search: 700/257, 700/245, 700/253, 700/258, 318/567, 318/568.11, 340/501, 340/506, 340/525, 340/531, 340/539.11, 340/539.14
• US Class Current: 1/1
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

Some embodiments provide a human and robot distributed operating system (HaRD-OS). The HaRD-OS efficiently and dynamically connects different human operators and algorithms to multiple remotely deployed robots based on the task(s) that the robots are to complete. Some embodiments facilitate an action/perception loop between the operators, algorithms and robots by routing tasks between human operators or algorithms based on various routing polices including operator familiarity, aptitude, access rights, end user preference, time zones, costs, efficiency, latency, privacy concerns, etc. In the event of a fault, some embodiments route to the best operator or algorithm that is able to handle the particular fault with the required privileges. Some embodiments secure task guarantees to be completed at particular times, priorities or costs.

Claims

1. A computer-implemented method for remotely managing robot operation, the computer-implemented method comprising:

receiving a request comprising (i) a first identifier identifying a particular robot from a plurality of distributed robots operating in a plurality of locations and (ii) a second identifier identifying a task for the particular robot to perform, wherein the particular robot comprises a plurality of actuators for performing different mechanical manipulations and a plurality of sensors providing sensory feedback about the particular robot and the plurality of actuators;

retrieving an access control list;

assigning control of the particular robot to a particular operator from a plurality of operators identified based on the access control list, wherein the access control list authorizes a set of operators including the particular operator in performing said task, wherein said assigning comprises authorizing execution by the particular robot of commands comprising a particular operator identifier of the particular operator during completion of the task;

restricting the particular robot functionality based on a first set of the plurality of actuators and a first set of the plurality of sensors said access control list authorizes the particular operator access to in completion of the task, wherein said restricting comprises enabling the particular operator with control of the first set of actuators and disabling the particular operator from controlling a different second set of the plurality of actuators during completion of the task and further comprises enabling the particular operator with control of the first set of sensors and disabling the particular operator from controlling a different second set of the plurality of sensors during completion of the task; and

filtering commands directed to the particular robot during execution of said task based on the access control list and said restricting, wherein said filtering comprises (i) blocking the particular robot from executing a first command comprising at least one of the first and second identifiers and an operator identifier other than the particular operator identifier, (ii) blocking the particular robot from executing a second command comprising the particular operator identifier and an operation involving an actuator from the second set of actuators or a sensor from the second set of sensors not authorized for use in completion of the task, and (iii) issuing the particular robot, a third command comprising the particular operator identifier and an operation involving an actuator from the first set of actuators or a sensor from the first set of sensors.

2. The computer-implemented method of claim 1, wherein restricting the particular robot functionality further comprises limiting an operational range of an actuator in the first set of actuators to an amount less than a full operational range of the actuator, and wherein filtering command execution further comprises blocking the particular robot from executing a fourth command comprising the particular operator identifier and an operation moving said actuator outside said operational range.

3. The computer-implemented method of claim 1, wherein restricting the particular robot functionality further comprises filtering the sensory feedback from a sensor in the first set of sensors when said sensory feedback comprises one of a plurality of restricted objects defined in said access control list.

4. The computer-implemented method of claim 1, wherein restricting the particular robot functionality further comprises limiting movements of the robot to a first location where the task is to be completed, and wherein filtering command execution further comprises preventing the robot from moving to a different second location until the task is completed.

5. The computer-implemented method of claim 1, wherein assigning control of the particular robot to a particular operator comprises assigning control of the particular robot to one of a first human operator or a first algorithm, enabling the first set of actuators and the first set of sensors based on permissions the access control list grants to the first human operator or the first algorithm, and disabling the second set of actuators and the second set of sensors based on restrictions the access control list places on the first human operator or the first algorithm.

6. The computer-implemented method of claim 5, and wherein restricting the particular robot functionality further comprises disabling a first subset of the first set of actuators and the first set of sensors unrelated to the completion of the task while enabling a different second subset of the first set of actuators and the first set of sensors related to the completion of the task.

7. The computer-implemented method of claim 5 further comprising reassigning control of the particular robot from the first human operator or the first algorithm to one of a different second human operator or second algorithm identified with a second operator identifier that is different than the particular operator identifier, wherein reassigning control of the particular robot comprises enabling a different third set of the plurality of actuators and a third set of the plurality of sensors while disabling a fourth set of the plurality of actuators and a fourth set of the plurality of sensors based on different permissions the access control list grants to the second human operator or the second algorithm, and wherein said filtering further comprises blocking the particular robot from executing a fourth command comprising at least one of the first and second identifiers and an operator identifier other than the second operator identifier.

8. The computer-implemented method of claim 1, wherein the task is a first task, and wherein the access control list defines enabling the first set of actuators and the first set of sensors while disabling the second set of actuators and the second set of sensors in completing the first task, and further defines enabling a different third set of actuators and a different third set of sensors while disabling a different fourth set of actuators and a different fourth set of sensors in completing a different second task.

9. The computer-implemented method of claim 8 further comprising reconfiguring the particular robot for performance of the second task, wherein reconfiguring comprises enabling the different third set of actuators and the different third set of sensors while disabling the different fourth set of actuators and the different fourth set of sensors for performance of the second task.

10. A computer-implemented method comprising:

receiving a plurality of requests comprising a first task directed to a first robot at a first location and a second task directed to a second robot at a different second location;

retrieving an access control list authorizing a plurality of remote human operators in performing different sets of tasks with the first and second robots, with at least one particular human operator authorized in the access control list to perform the first task and the second task;

configuring the particular human operator remote control of the first robot and the second robot, wherein said configuring comprises mapping a first task identifier identifying the first task and a second task identifier identifying the second task to an operator identifier of the particular human operator; and

providing the particular human operator contemporaneous and remote control of the first robot and the second robot, wherein providing said remote control comprises routing a first set of commands signed with a private key of the particular human operator and comprising the first task identifier and the operator identifier of the particular human operator to the first robot for execution and routing a second set of commands signed with the private key of the particular human operator and comprising the second task identifier and the operator identifier of the particular human operator to the second robot for execution.

11. The computer-implemented method of claim 10 further comprising blocking a third set of commands signed with a different private key than the private key of the particular human operator from execution by the first robot.

12. The computer-implemented method of claim 11, wherein said mapping further comprises providing a public key to the first robot, and wherein routing the first set of commands further comprises authorizing the first robot for execution of the first set of commands by using the public key in verifying the first set of commands is signed with the particular human operator private key.

13. The computer-implemented method of claim 10 further comprising (i) monitoring the first robot in completion of the first task and monitoring the second robot in completion of the second task and (ii) reassigning control of the first robot from the particular human operator to a different human operator or algorithm upon completion of the first task by remapping the first task identifier to a different second operator identifier.

14. The computer-implemented method of claim 10, wherein providing said remote control further comprises blocking from the first robot, commands omitting one of the first task identifier and the operator identifier during execution of the first task and blocking from the second robot, commands omitting one of the second task identifier and the operator identifier during execution of the second task.

15. A computer-implemented method for optimally scheduling a plurality of robot operators to remotely control any of a plurality of robots distributed to a plurality of locations, the computer-implemented method comprising:

receiving a task for execution by a particular robot of the plurality of robots;

issuing to each operator of the plurality of robot operators, a cost and availability request for controlling the particular robot in performance of the task;

receiving from each operator of the plurality of robot operators in response to said issuing, at least cost and availability of each of the plurality of robot operators for controlling the particular robot in performance of the task;

selecting an optimal robot operator from the plurality of robot operators based on said cost and availability of each of the plurality of robot operators; and

assigning remote control of the particular robot to the optimal robot operator during performance of the task.

16. The computer-implemented method of claim 15, wherein the plurality of robot operators comprises a plurality of human operators for controlling a robot in tasks partly involving human control of the robot, and wherein a cost of each human operator of the plurality of human operators is based on a skill level or expertise level of the human operator in performing said task.

17. The computer-implemented method of claim 16, wherein the plurality of robot operators further comprises a plurality of algorithms providing fully autonomous control of a robot in performance of different tasks, and wherein a cost of each algorithm of the plurality of algorithms is based on efficiency with which the algorithm completes a task.

18. The computer-implemented method of claim 15, wherein assigning remote control of the particular robot comprises routing commands from the optimal robot operator to the particular robot for execution while blocking commands from other operators of the plurality of robot operators from execution by the particular robot.

19. The computer-implemented method of claim 15 further comprising detecting an error during performance of the task by the particular robot under control of the optimal robot operator.

20. The computer-implemented method of claim 19 further comprising reassigning control of the particular robot from the optimal robot operator to a different robot operator from the plurality of robot operator for resolution of the error.