Methods and systems for multirobotic management

• US 9,513,624 B1
• Filed: 04/06/2015
• Issued: 12/06/2016
• Est. Priority Date: 05/06/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Hickman, Ryan, Gharpure, Chaitanya
• Primary Examiner(s): Holwerda, Stephen
• Application Number:
  US14/679,457
• Time in Patent Office: 610 Days
• Field of Search: 700/245, 700/246, 700/247, 700/248, 700/250, 700/253, 700/258, 700/259, 318/568.1, 318/568.11, 318/568.12, 318/568.16, 318/568.2
• US Class Current: 1/1
• CPC Class Codes: A01N 25/10 Macromolecular compounds
• CPC Class Codes: A23B 5/06 Coating eggs with a protective layer; Compositions or apparatus therefor
• CPC Class Codes: B25J 3/00 Manipulators of master-slave type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
• CPC Class Codes: G05B 19/418 Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

Multirobotic management can involve communications between a command or leader robot and one or more client or follower robots through a cloud computing system. In an example implementation, a leader robot can receive first sensory data captured by a first follower robot and second sensory data captured by a second follower robot, determine a command function based on at least one of the first sensory data and the second sensory data, and communicate with at least one of the first follower robot and the second follower robot based on the command function.

Claims

1. A method comprising:

receiving, by a command robot, sensory data captured by a first client robot, wherein the command robot is in communication with the first client robot and a second client robot, wherein the first client robot has a first sensing capability and the second client robot has a second sensing capability that is different than the first sensing capability, wherein the sensory data is captured by way of the first sensing capability of the first client robot, wherein the first client robot has a first functionality and the second client robot has a second functionality that is different than the first functionality, wherein the sensory data is relevant to the second functionality of the second client robot;

providing, by the command robot, at least a portion of the sensory data captured by the first client robot as a first input to the second client robot, wherein the portion of the sensory data is relevant to the second functionality of the second client robot;

receiving feedback from the second client robot based on the first input;

commanding, by the command robot, the second client robot to perform a task by way of the second functionality based on the feedback and the portion of the sensory data captured by the first client robot;

receiving respective sensory data from the second client robot based on the commanded task, wherein the respective sensory data is captured by the second client robot by way of the second sensing capability; and

in response to the receiving the respective sensory data, commanding the second client robot to provide a second input to the first client robot indicating performance of the commanded task.

2. The method of claim 1, further comprising:

receiving further sensory data from the first client robot; and

commanding the first client robot to perform an additional task based on the further sensory data.

3. The method of claim 1, wherein each of the first client robot and the second client robot include at least one sensor comprising one or more of an image capture device, an acoustic sensor device, and an electronic data acquisition device.

4. The method of claim 1, wherein the first sensing capability is related to capturing environment sensory data associated with a condition of an environment of the first client robot, and wherein the second sensing capability of the second client robot is related to sensing state of a device configured to adjust the condition of the environment of the first client robot.

5. The method of claim 1, wherein the command robot is in communication with the first client robot and the second client robot by way of a cloud computing system, wherein the cloud computing system includes at least one server remote from the command robot, the first client robot, and the second client robot, and wherein the at least one server is configured to store at least a portion of the sensory data and facilitate communication between the command robot and the first and second client robots.

6. The method of claim 1, wherein at least one of the first client robot and the second client robot comprises one or more mechanical actuators and configured to receive a removable module that includes wireless communication capability, wherein the command robot is configured to communicate with the at least one of the first client robot and the second client robot by way of the wireless communication capability.

7. A non-transitory computer readable memory having stored thereon instructions that, when executed by a command robot, cause the command robot to perform operations comprising:

receiving sensory data captured by a first client robot, wherein the command robot is in communication with the first client robot and a second client robot, wherein the first client robot has a first sensing capability and the second client robot has a second sensing capability that is different than the first sensing capability, wherein the sensory data is captured by way of the first sensing capability of the first client robot, wherein the first client robot has a first functionality and the second client robot has a second functionality that is different than the first functionality, wherein the sensory data is relevant to the second functionality of the second client robot;

providing at least a portion of the sensory data captured by the first client robot as a first input to the second client robot, wherein the portion of the sensory data is relevant to the second functionality of the second client robot;

receiving feedback from the second client robot based on the first input;

commanding the second client robot to perform a task by way of the second functionality based on the feedback and the portion of the sensory data captured by the first client robot;

receiving respective sensory data from the second client robot based on the commanded task, wherein the respective sensory data is captured by the second client robot by way of the second sensing capability; and

in response to the receiving the respective sensory data, commanding the second client robot to provide a second input to the first client robot indicating performance of the commanded task.

8. The non-transitory computer readable memory of claim 7, wherein the operations further comprise:

receiving further sensory data from the first client robot; and

commanding the first client robot to perform an additional task based on the further sensory data.

9. The non-transitory computer readable memory of claim 7, wherein each of the first client robot and the second client robot include at least one sensor comprising one or more of an image capture device, an acoustic sensor device, and an electronic data acquisition device.

10. The non-transitory computer readable memory of claim 7, wherein the first sensing capability is related to capturing environment sensory data associated with a condition of an environment of the first client robot, and wherein the second sensing capability of the second client robot is related to sensing state of a device configured to adjust the condition of the environment of the first client robot.

11. The non-transitory computer readable memory of claim 7, wherein the command robot is in communication with the first client robot and the second client robot by way of a cloud computing system, wherein the cloud computing system includes at least one server remote from the command robot, the first client robot, and the second client robot, and wherein the at least one server is configured to store at least a portion of the sensory data and facilitate communication between the command robot and the first and second client robots.

12. The non-transitory computer readable memory of claim 7, wherein at least one of the first client robot and the second client robot comprises one or more mechanical actuators and configured to receive a removable module that includes wireless communication capability, wherein the command robot is configured to communicate with the at least one of the first client robot and the second client robot by way of the wireless communication capability.

13. A system comprising:

a first client robot having a first sensing capability and a first functionality;

a second client robot having a second sensing capability that is different than the first sensing capability and a second functionality that is different than the first functionality; and

a command robot in communication with the first client robot and the second client robot, wherein the command robot comprises at least one processor, a memory, and program instructions stored in the memory that when executed by the at least one processor cause the command robot to perform operations comprising;

receiving sensory data captured by the first client robot by way of the first sensing capability, wherein the sensory data is relevant to the second functionality of the second client robot;

providing at least a portion of the sensory data captured by the first client robot as a first input to the second client robot, wherein the portion of the sensory data is relevant to the second functionality of the second client robot;

receiving feedback from the second client robot based on the first input;

commanding the second client robot to perform a task by way of the second functionality based on the feedback and the portion of the sensory data captured by the first client robot;

receiving respective sensory data from the second client robot based on the commanded task, wherein the respective sensory data is captured by the second client robot by way of the second sensing capability; and

in response to the receiving the respective sensory data, commanding the second client robot to provide a second input to the first client robot indicating performance of the commanded task.

14. The system of claim 13, wherein the operations further comprise:

receiving further sensory data from the first client robot; and

commanding the first client robot to perform an additional task based on the further sensory data.

15. The system of claim 13, wherein each of the first client robot and the second client robot include at least one sensor comprising one or more of an image capture device, an acoustic sensor device, and an electronic data acquisition device.

16. The system of claim 13, wherein the first sensing capability is related to capturing environment sensory data associated with a condition of an environment of the first client robot, and wherein the second sensing capability of the second client robot is related to sensing state of a device configured to adjust the condition of the environment of the first client robot.

17. The system of claim 13, further comprising:

a cloud computing system, wherein the command robot is in communication with the first client robot and the second client robot by way of the cloud computing system, wherein the cloud computing system includes at least one server remote from the command robot, the first client robot, and the second client robot, and wherein the at least one server is configured to store at least a portion of the sensory data and facilitate communication between the command robot and the first and second client robots.