Human and robotic distributed operating system (HaRD-OS)

• US 9,050,723 B1
• Filed: 07/11/2014
• Issued: 06/09/2015
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: InVia Robotics Inc.
• Original Assignee: InVia Robotics, LLC
• Inventors: Elazary, Lior, Parks, Daniel Frank II, Voorhies, Randolph Charles
• Primary Examiner(s): Tran, Dalena
• Application Number:
  US14/328,916
• Time in Patent Office: 333 Days
• Field of Search: 700/257, 700/245, 700/258, 700/253, 318/567, 318/568.11, 340/501, 340/506, 340/525, 340/531, 340/539.11, 340/539.14, 726/1, 726/2, 726/4, 726/26
• US Class Current: 1/1
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system

Abstract

Some embodiments provide a human and robot distributed operating system (HaRD-OS). The HaRD-OS efficiently and dynamically connects different human operators and algorithms to multiple remotely deployed robots based on the task(s) that the robots are to complete. Some embodiments facilitate an action/perception loop between the operators, algorithms and robots by routing tasks between human operators or algorithms based on various routing polices including operator familiarity, aptitude, access rights, end user preference, time zones, costs, efficiency, latency, privacy concerns, etc. In the event of a fault, some embodiments route to the best operator or algorithm that is able to handle the particular fault with the required privileges. Some embodiments secure task guarantees to be completed at particular times, priorities or costs.

Claims

1. A computer-implemented method for routing and scheduling tasks to facilitate control of any of a plurality of robots, the computer-implemented method comprising:

configuring an access control list (i) identifying a plurality of human operators that are each qualified to perform a first set of a plurality of tasks partly involving human control of a robot and (ii) identifying a plurality of algorithms for instructing a robot in performing a second set of the plurality of tasks fully autonomously without human control of the robot;

receiving a request identifying a robot and a task for the robot to perform;

determining based on a set of criteria, at least one (i) human operator from the plurality of human operators or (ii) algorithm from the plurality of algorithms for optimally controlling the robot in performance of the task;

providing a particular human operator from the plurality of human operators control of the robot when the task is one of the first set of tasks partly involving human control and the particular human operator is identified within the access control list as being qualified to optimally perform the task in conformance of the set of criteria, wherein the robot comprises a set of controls and sensors, and wherein providing the particular human operator control of the robot comprises authorizing the particular human operator access to a first subset of the set of controls and sensors of the robot and preventing the particular human operator access to controls and sensors of the robot not within the first subset of the controls and sensors; and

providing a particular algorithm from the plurality of algorithms control of the robot when the task is one of the second set of tasks the robot can complete fully autonomously and the algorithm comprises instructions for optimally completing the task fully autonomously in conformance of the set of criteria, wherein providing the particular algorithm control of the robot comprises authorizing the particular algorithm access to a different second subset of the set of controls and sensors of the robot and preventing the particular algorithm access to controls and sensors of the robot not within the second subset of the controls and sensors.

2. The computer-implemented method of claim 1, wherein the set of criteria comprises any of a cost and efficiency metric, wherein the cost metric identifies different costs associated with using any of the plurality of human operators and the plurality of algorithms to control the robot in performance of the task, and wherein the efficiency metric identifies different efficiencies associated with using any of the plurality of human operators and the plurality of algorithms to control the robot in performance of the task.

3. The computer-implemented method of claim 1 further comprising assigning a first identifier to the task and a different second identifier to uniquely identify each human operator of the plurality of human operators and each algorithm of the plurality of algorithms.

4. The computer-implemented method of claim 3, wherein providing the particular human operator control of the robot comprises authenticating control of the robot by the particular human operator during completion of the task by (i) routing to the robot for execution, instructions intended for the robot comprising the first identifier and the second identifier assigned to the particular human operator and (ii) blocking instructions intended for the robot not comprising both the first identifier and the second identifier assigned to the particular human operator, and wherein providing the particular algorithm control of the robot comprises authenticating control of the robot by the particular algorithm during completion of the task by (i) routing to the robot for execution, instructions intended for the robot comprising the first identifier and the second identifier assigned to the particular algorithm and (ii) blocking instructions intended for the robot not comprising both the first identifier and the second identifier assigned to the particular algorithm.

5. The computer-implemented method of claim 1, wherein the task is a first task, the method further comprising receiving a second request identifying the robot and a different second task for the robot to perform and reassigning control of the robot to a different human operator or a different algorithm than was used in performing the first task.

6. The computer-implemented method of claim 1, wherein said request originates from one of a user assisted by the robot, one of the plurality of human operators, and one of the plurality of algorithms.

7. The computer-implemented method of claim 1 further comprising detecting a fault in completing said task.

8. The computer-implemented method of claim 7 further comprising selecting one of (i) a second different operator from the plurality of human operators and (ii) a second different algorithm from the plurality of algorithms based on the fault and transferring control of the robot from either the particular operator or the particular algorithm to either of the selected second different operator or second different algorithm for resolving the fault.

9. A distributed system comprising:

a plurality of robots deployed to different user locations, each robot of the plurality of robots comprising a network interface for remotely receiving a plurality of instructions associated with any set of a plurality of tasks the robot performs, the plurality of instructions controlling operation of the robot; and

at least one system node comprising;

a processor;

a storage storing an access control list identifying (i) a plurality of human operators that are each qualified to perform some subset of the plurality of tasks partly involving human control of a robot and (ii) identifying a plurality of algorithms for instructing a robot in performing a subset of the plurality of tasks fully autonomously without human control of the robot;

a network interface receiving at least a first task and a second task a user requests a particular robot of the plurality of robots to perform; and

wherein the at least one system node, by operation of the processor, communicably couples to the particular robot using the network interface and schedules the particular robot in performance of the first task and the second task by (i) providing control of the robot in performance of the first task to any of a first algorithm and a first human operator, (ii) authenticating instructions issued to the robot in performance of the first task based on a first identifier identifying any of the first algorithm and the first human operator and a second identifier identifying the first task, (iii) monitoring the robot in performance of the first task, and (iv) providing control of the robot in performance of the second task to any of a different second algorithm and a different second human operator than used in controlling the robot in performance of the first task, wherein said authenticating comprises routing to the robot any instruction signed with the first identifier and the second identifier and blocking any instruction not signed with the first identifier and the second identifier.

10. The distributed system of claim 9, wherein each robot of the plurality of robots further comprises a plurality of actuators and sensors, and wherein the at least one system node, by operation of the processor, further detects a fault during performance of the first task by collecting a plurality of data values from the plurality of actuators and sensors of the robot when monitoring the robot and by identifying at least one data value of the plurality of data values deviating from a plurality of acceptable data values for the plurality of actuators and sensors in performance of the first task.

11. The distributed system of claim 10, wherein the at least one system node, by operation of the processor, further responds to the fault by transferring control of the robot from any of the first algorithm and the first human operator to any of a different third human operator from the plurality of human operators and a different third algorithm from the plurality of algorithms qualified to resolve the fault.

12. The distributed system of claim 9, wherein the at least one system node, by operation of the processor, further authorizes any of the first human operator and the first algorithm for performing the first task by issuing signed permissions configuring the robot to execute instructions from any of the first human operator and the first algorithm for a specified time period.

13. The distributed system of claim 9, wherein the at least one system node, by operation of the processor, obtains availability for completing the first task and costs and efficiencies associated with the first task from a different second system node.

14. A computer-implemented method comprising:

configuring an access control list (i) identifying a plurality of human operators that are each qualified to perform a first set of a plurality of tasks partly involving human control of a robot and (ii) identifying a plurality of algorithms for instructing a robot in performing a second set of the plurality of tasks fully autonomously without human control of the robot;

receiving a request identifying a robot and a task for the robot to perform;

assigning a first identifier to the task and a different second identifier to uniquely identify each human operator of the plurality of human operators and each algorithm of the plurality of algorithms;

determining based on a set of criteria, at least one (i) human operator from the plurality of human operators or (ii) algorithm from the plurality of algorithms for optimally controlling the robot in performance of the task;

providing a particular human operator from the plurality of human operators control of the robot when the task is one of the first set of tasks partly involving human control and the particular human operator is identified within the access control list as being qualified to optimally perform the task in conformance of the set of criteria, wherein providing the particular human operator control of the robot comprises authenticating control of the robot by the particular human operator during completion of the task by (i) routing to the robot for execution, instructions for the robot comprising the first identifier and the particular human operator assigned second identifier and (ii) blocking instructions for the robot not comprising both the first identifier and the particular human operator assigned second identifier; and

providing a particular algorithm from the plurality of algorithms control of the robot when the task is one of the second set of tasks the robot can complete fully autonomously and the algorithm comprises instructions for optimally completing the task fully autonomously in conformance of the set of criteria, wherein providing the particular algorithm control of the robot comprises authenticating control of the robot by the particular algorithm during completion of the task by (i) routing to the robot for execution, instructions for the robot comprising the first identifier and the particular algorithm assigned second identifier and (ii) blocking instructions for the robot not comprising both the first identifier and the particular algorithm assigned second identifier.

15. The computer-implemented method of claim 14, wherein providing the particular algorithm control of the robot comprises initializing the robot with the particular algorithm and commencing fully autonomous operation of the robot under control of said algorithm without human control.

16. The computer-implemented method of claim 14 further comprising monitoring performance of the task by the robot under control of any of the particular human operator and the particular algorithm.

17. The computer-implemented method of claim 16 further comprising terminating said control of the robot upon detecting completion of the task or a fault as a result of said monitoring.

18. The computer-implemented method of claim 17 further comprising transferring control of the robot to any of a different second human operator from the plurality of human operators or different second algorithm from the plurality of algorithms in order to resolve a fault encountered during said monitoring.

19. The computer-implemented method of claim 16 further comprising generating a fault during said monitoring, wherein generating the fault comprises collecting (i) actuator values from a plurality of actuators of the robot and (ii) sensor data from a plurality of sensors of the robot and identifying at least one of either of the actuator values and the sensor data deviating from a set of acceptable actuator values and a set of acceptable sensor data for said task.

20. The computer-implemented method of claim 19 further comprising selecting,

based on the at least one actuator value or sensor data deviating from the set of acceptable actuator values and the set of acceptable sensor data for said task, at least one (i) different second human operator from a plurality of available human operators or (ii) different second algorithm from the plurality of algorithms qualified to resolve the fault and initializing the robot in resolving the fault by transferring control of the robot to the selected different second human operator or different second algorithm.