Techniques for coordinating independent objects with occlusions

• US 10,192,195 B1
• Filed: 10/25/2016
• Issued: 01/29/2019
• Est. Priority Date: 10/25/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Brazeau, Jeremiah David
• Primary Examiner(s): Cumbess, Yolanda R
• Application Number:
  US15/334,062
• Time in Patent Office: 826 Days
• Field of Search: None
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1666 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 1/1373 for fulfilling orders in warehouses
• CPC Class Codes: G05B 2219/39083 Robot interference, between two robot arms
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

Systems and methods are provided herein for coordinating motion between components of an inventory system. A first set of instructions associated with a first task to be performed by a first robotic device may be received. A second set of instructions associated with a second task to be performed by a second robotic device may be received. The first and second robotic devices may be configured to utilize corresponding operational areas that may overlap to define an area of overlap. Light information representative of the spatial condition of at least one of the robotic devices may be projected onto a projection surface. The light information may be utilized to determine that at least one of the first and second robotic devices is utilizing the area of overlap. A remedial action may be performed to coordinate motion of the first and second robotic devices within the area of overlap.

Claims

1. An inventory system, comprising:

one or more data networks;

a management system comprising a first set of memories storing computer-readable instructions and a first set of processors; and

a first robotic arm and a second robotic arm, the first robotic arm and the second robotic arm individually comprising a set of light emitters and a second set of memories storing second computer-readable instructions that, upon execution by a second set of processors, cause the first robotic arm to at least;

receive, from the management system over the one or more data networks, a set of instructions associated with a task, the task comprising moving an item from a first location to a second location within a storage facility, the set of instructions identifying a projection area within the storage facility;

determine a point of reference associated with the projection area;

obtain one or more light data points from the projection area;

compute motion information of the second robotic arm based at least in part on the one or more light data points; and

perform one or more remedial actions based at least in part on the computed motion information of the second robotic arm, wherein analyzing the one or more light data points enables the first robotic arm to avoid at least one contention within an operational overlap area associated with the first robotic arm and the second robotic arm.

2. The inventory system of claim 1, wherein individual light data points correspond to a location of a particular portion of the second robotic arm.

3. The inventory system of claim 1, wherein the motion information of the second robotic arm comprises at least one of a location of the second robotic arm, an orientation corresponding to a portion of the second robotic arm, a motion measurement of the second robotic arm, or a distance measurement of the second robotic arm with respect to the point of reference associated with the projection area.

4. The inventory system of claim 1, wherein the first robotic arm and the second robotic arm are adjacent to one another within a conveyor belt system.

5. A computer-implemented method, comprising:

receiving, by a first robotic device, a first set of instructions associated with a first task to be performed by the first robotic device, the first robotic device being configured to utilize a first operational area, the first robotic device having a set of light emitters;

receiving, by a second robotic device, a second set of instructions associated with a second task to be performed by the second robotic device, the second robotic device being configured to utilize a second operational area, wherein the first and second operational areas overlap one another to define an area of overlap;

projecting, by the first robotic device utilizing the set of light emitters, onto a projection surface a plurality of light data points representative of a spatial condition of the first robotic device;

obtaining by the second robotic device, one or more of the plurality of light data points from the projection surface;

computing, by the second robotic device, motion information of the first robotic device based at least in part on the one or more of the plurality of light data points;

determining, by the second robotic device, based at least in part on the motion information, that the first robotic device is utilizing the area of overlap; and

performing, by the second robotic device, a remedial action to coordinate motion of the first and second robotic devices within the area of overlap, the remedial action being performed based at least in part on determining that the first robotic device is utilizing the area of overlap.

6. The computer-implemented method of claim 5, wherein the remedial action comprises at least one of modifying the second set of instructions, halting motion of the second robotic device, reducing speed of the second robotic device, causing the first robotic device to halt motion or to change position, requesting a different task, or executing a different task.

7. The computer-implemented method of claim 5, further comprising:

receiving task information associated with the second task corresponding to the second robotic device; and

determining potential spatial conditions of the second robotic device based at least in part on the task information, wherein the remedial action is based at least in part on the potential spatial conditions of the second robotic device.

8. The computer-implemented method of claim 5, further comprising determining, based at least in part on the motion information, that a planned motion of the first robotic device will conflict with the planned motion of the second robotic device, the planned motion of the second robotic device being based at least in part on the second set of instructions.

9. The computer-implemented method of claim 5, wherein the set of lights emitters are affixed to the first robotic device.

10. The computer-implemented method of claim 5, wherein an attribute of the plurality of light data points differs in at least one of a color, a size, or a shape, and wherein the remedial action is based at least in part on a specific value for the attribute.

11. The computer-implemented method of claim 5, wherein computing the motion information of the first robotic device comprises:

obtaining, by the second robotic device, a specification associated with the first robotic device, the specification identifying an expected light pattern that specifies attributes of a set of light data points that correspond to particular portions of the first robotic device; and

identifying particular light data points of the plurality of light data points that correspond to the particular portions of the first robotic device based at least in part on the specification.

12. The computer-implemented method of claim 5, wherein computing the motion information of the first robotic device further comprises:

obtaining a specification associated with the first robotic device, the specification identifying properties of a motion vector for identifying planned spatial conditions of the first robotic device;

identifying one or more motion vectors from the one or more of the plurality of light data points; and

computing a planned spatial condition of the first robotic device based at least in part on the specification and the one or more motion vectors.

13. The computer-implemented method of claim 5, further comprising:

transmitting, by the second robotic device to a management system, a conflict resolution request based at least in part on determining that the first robotic device is utilizing the area of overlap;

in response to the conflict resolution request, receiving, from the management system, a new set of instructions associated with execution of the second task; and

executing the second task according to the new set of instructions.

14. The computer-implemented method of claim 5, further comprising:

determining a first priority associated with the first task; and

determining a second priority associated with the second task wherein the remedial action is based at least in part on the first priority and the second priority.

15. A computer-readable storage medium comprising computer-readable instructions that, upon execution by one or more processors of a first robotic device, cause the first robotic device to perform operations comprising:

receiving, by the first robotic device, a set of instructions associated with a task to be performed within a storage facility, the task being associated with moving an object by the first robotic device and a second robotic device, wherein the object obscures a view of the second robotic device by the first robotic device, and wherein moving the object requires the first robotic device and the second robotic device to simultaneously maintain physical contact with the object for a period of time;

identifying a surface within the storage facility by utilizing at least one sensor of the first robotic device;

obtaining light information being projected on the surface by the second robotic device performing the task;

computing motion information of the second robotic device based at least in part on the light information; and

executing the set of instructions associated with the task based at least in part on the computed motion information of the second robotic device.

16. The computer-readable storage medium of claim 15, wherein the operations further comprise:

providing, by the first robotic device, additional light information on the surface, the additional light information corresponding to additional motion information of the first robotic device; and

computing a distance measurement of the first robotic device with respect to the surface based at least in part on the additional light information, wherein the motion information of the second robotic device is based at least in part on the distance measurement of the first robotic device.

17. The computer-readable storage medium of claim 15, wherein the operations further comprise:

transmitting, to the second robotic device, a first specification associated with the first robotic device, the first specification indicating a first expected light pattern of the first robotic device; and

receiving, from the second robotic device, a second specification associated with the second robotic device, the second specification indicating a second expected light pattern of the second robotic device, wherein the motion information of the second robotic device is computed based at least in part on the second specification associated with the second robotic device.

18. The computer-readable storage medium of claim 15, wherein the operations further comprise:

calculating an expected path of the object;

determining that the object deviated from the expected path and in a manner contrary to the motion information of the second robotic device; and

performing a remedial action to return the object to the expected path.

19. The computer-readable storage medium of claim 15, wherein the operations further comprise:

providing the motion information to a remote processing system; and

receiving, from the remote processing system, a set of protocols for performing remedial actions based at least in part on the motion information provided to the remote processing system.

20. The computer-readable storage medium of claim 15, wherein the operations further comprise:

receiving, from a remote processing system, a second specification associated with the second robotic device, the second specification indicating a second expected light pattern of the second robotic device, wherein the motion information of the second robotic device is computed based at least in part on the second specification associated with the second robotic device.