Dynamically maintaining a map of a fleet of robotic devices in an environment to facilitate robotic action

• US 10,022,867 B2
• Filed: 09/24/2015
• Issued: 07/17/2018
• Est. Priority Date: 11/11/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Saboo, Rohit Ramesh, Kuffner, James Joseph, Mason, Julian
• Primary Examiner(s): Collins, Michael
• Application Number:
  US14/863,620
• Publication Number:
  US 20160129592A1
• Time in Patent Office: 1,027 Days
• Field of Search: 700245-255
• CPC Class Codes: B25J 15/0625 provided with a valve
• CPC Class Codes: B25J 19/021 Optical sensing devices
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0084 comprising a plurality of manipulators
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: G01C 21/3807 characterised by the type of data
• CPC Class Codes: G01C 21/3833 characterised by the source of data
• CPC Class Codes: G05B 19/41895 using automatic guided vehicles [AGV] control of position or course of AGV's G05D1/00
• CPC Class Codes: G05B 2219/31006 Monitoring of vehicle
• CPC Class Codes: G05B 2219/31007 Floor plan, map stored in on-board computer of vehicle
• CPC Class Codes: G05B 2219/45056 Handling cases, boxes
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0297 by controlling means in a control room
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
• CPC Class Codes: Y02P 90/60 Electric or hybrid propulsion means for production processes

Abstract

Methods and systems for dynamically maintaining a map of robotic devices in an environment are provided herein. A map of robotic devices may be determined, where the map includes predicted future locations of at least some of the robotic devices. One or more robotic devices may then be caused to perform a task. During a performance of the task by the one or more robotic devices, task progress data may be received from the robotic devices, indicative of which of the task phases have been performed. Based on the data, the map may be updated to include a modification to the predicted future locations of at least some of the robotic devices. One or more robotic devices may then be caused to perform at least one other task in accordance with the updated map.

Claims

1. A method comprising:

determining a digital map representative of a physical environment and of a first robotic device and a second robotic device within the physical environment, wherein the digital map includes an indication of a predicted future location of the first robotic device and a predicted time at which the first robotic device will arrive at the predicted future location;

causing the first robotic device to perform a task comprising a plurality of task phases, the plurality including a coordinated task phase to be performed with the second robotic device, the coordinated task phase comprising a transfer of an object between the first and second robotic devices;

during a performance of at least one of the plurality of task phases by the first robotic device, receiving, from the first robotic device, task progress data indicative of which of the plurality of task phases, other than the coordinated task phase, have been performed by the first robotic device;

based on the received task progress data, updating the digital map to include an indication of a change in one or more of the predicted future location and the predicted time; and

based on the updated digital map indicating the change, causing the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location and the changed predicted time.

2. The method of claim 1,

wherein the physical environment includes the object,

wherein the digital map further includes predicted future locations of the object, and

wherein the updated digital map includes at least one modification to the predicted future locations of the object.

3. The method of claim 1, wherein the predicted future location of the first robotic device includes a predicted future trajectory of the first robotic device, and

wherein the change in the predicted location includes a change in the predicted future trajectory.

4. The method of claim 1, wherein the predicted future location of the first robotic device includes a predicted future set of joint variables of an appendage of the first robotic device, and

wherein the change in the predicted future location of the first robotic device includes a change in the predicted future set of joint variables of the appendage.

5. The method of claim 1, further comprising:

based on the determined digital map, determining an anticipated completion time of a performance of the task by the first robotic device; and

based on the updated digital map, updating the anticipated completion time of the performance of the task.

6. The method of claim 5, wherein the performance of the task involves the object being delivered by the first robotic device to a target location after the transfer of the object is complete.

7. The method of claim 1, wherein the task is a first task, wherein the plurality of task phases is a first plurality of task phases, and wherein the second robotic device is configured to perform a task comprising a second plurality of task phases, the second plurality including the coordinated task phase, the method further comprising:

based on the determined digital map, determining an anticipated completion time of performance of at least one of the plurality of task phases by the first robotic device; and

based on the updated digital map, updating an anticipated completion time of performance by the second robotic device of at least one task phase of the second plurality, other than the coordinated task phase.

8. The method of claim 1, performed by a cloud computing system that is in communication with the first and second robotic devices, the method further comprising:

maintaining a queue of updates to the digital map, wherein each update to the digital map in the queue includes data specifying a time at which to publish the update to a given robotic device of the first and second robotic devices; and

publishing updates maintained in the queue in accordance with the specified times.

9. The method of claim 1, wherein the first robotic device and the second robotic device are two different types of robotic devices.

10. The method of claim 1, further comprising:

based on the updated digital map indicating the change, causing the second robotic device to perform, before the first robotic device and the second robotic device meet to transfer the object, a task involving preparation by the second robotic device to transfer the object.

11. A system comprising:

a first robotic device within a physical environment;

a second robotic device within the physical environment;

at least one processor; and

data storage comprising instructions executable by the at least one processor to cause the system to perform operations comprising;

determining a digital map representative of the physical environment and of the first and second robotic devices within the physical environment, wherein the digital map includes an indication of a predicted future location of the first robotic device and a predicted time at which the first robotic device will arrive at the predicted future location;

causing the first robotic device to perform a task comprising a plurality of task phases, the plurality including a coordinated task phase to be performed with the second robotic device, the coordinated task phase comprising a transfer of an object between the first and second robotic devices;

during a performance of at least one of the plurality of task phases by the first robotic device, receiving, from the first robotic device, task progress data indicative of which of the plurality of task phases, other than the coordinated task phase, have been performed by the first robotic device;

based on the received task progress data, updating the digital map to include an indication of a change in one or more of the predicted future location and the predicted time; and

based on the updated digital map indicating the change, causing the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location and the changed predicted time.

12. The system of claim 11, wherein the second robotic device is a robotic arm having a fixed base at given location within the physical environment and the first robotic device is a dynamic robotic device configured to travel to multiple locations within the physical environment, and

wherein the change in one or more of the predicted future location and the predicted time includes a change in one or more of the predicted future location and the predicted time of the robotic arm.

13. The system of claim 11, wherein causing the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location and the changed predicted time based on the updated digital map indicating the change comprises, transmitting, to the second robotic device, at least a portion of the updated digital map along with instructions for the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location.

14. The system of claim 11, the operations further comprising:

receiving, from the second robotic device, a query for an updated map, wherein causing the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location and the changed predicted time based on the updated digital map indicating the change comprises, in response to receiving the query, transmitting, to the second robotic device, at least a portion of the updated map along with instructions for the second robotic device to perform the coordinated task phase with the first robotic device at one or more of the changed predicted future location.

15. The system of claim 11, wherein receiving the task progress data includes receiving, from the first robotic device, a message indicating a completion of a performance of a given task phase of the plurality of task phases.

16. The system of claim 11, the operations further comprising:

making a determination that a performance of the task has been completed;

in response to the determination, updating the digital map to indicate a completion of the performance of the task;

based on the updated map, selecting a subsequent task to perform; and

causing the first robotic device to perform the subsequent task.

17. The system of claim 11, the operations further comprising:

determining a prioritization of the first robotic device and another robotic device that is different from the second robotic device, the prioritization defining relative priority levels among the first robotic device and the other robotic device, wherein the prioritization is based on a determination of the first robotic device being involved in a performance of the task and the other robotic device not being involved in the performance of the task; and

transmitting the updated digital map to the first robotic device and the other robotic device in accordance with the determined prioritization.

18. The system of claim 11, further comprising a plurality of other robotic devices within the physical environment, wherein a performance of the coordinated task phase involves a performance of the coordinated task phase within an area of interest of the physical environment, the operations further comprising:

determining a prioritization of the plurality of other robotic devices, the prioritization defining relative priority levels among the other robotic devices of the plurality, wherein the prioritization is based on a determination of one or more of;

respective distances from each of the plurality of other robotic devices to the area of interest, respective anticipated times each of the plurality of other robotic devices will take to reach the area of interest, and whether respective anticipated trajectories of each of the plurality of other robotic devices include a least a portion of the area of interest; and

transmitting the updated digital map to the plurality of other robotic devices in accordance with the determined prioritization.

19. The system of claim 11, wherein the first robotic device is a robotic arm having a fixed base at given location within the physical environment and the second robotic device is a dynamic robotic device configured to travel to multiple locations within the physical environment, and

wherein the change in one or more of the predicted future location and the predicted time includes a change in one or more of the predicted future location and the predicted time of the robotic arm.