Systems and methods to facilitate human/robot interaction

• US 9,588,519 B2
• Filed: 03/17/2015
• Issued: 03/07/2017
• Est. Priority Date: 03/17/2015
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Stubbs, Andrew, Verminski, Matthew David, Caldara, Stephen, Shydo, Robert Michael Jr., Tierney, Gregory Edward, Mansfield, Peter K.
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US14/660,259
• Publication Number:
  US 20160274586A1
• Time in Patent Office: 721 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0234 using optical markers or beacons optical beacons per se G01S1/70
• CPC Class Codes: G05D 1/0261 using magnetic plots
• CPC Class Codes: G06K 19/07762 the adhering arrangement making the record carrier wearable, e.g. having the form of a ring, watch, glove or bracelet record carriers for insertion in the human body for medical purposes A61B90/00; record carriers adapted for attachment to animals A01K11/00
• CPC Class Codes: G06K 7/10366 the interrogation device being adapted for miscellaneous applications
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device
• CPC Class Codes: Y10S 901/47 Optical

Abstract

Short range transmissions are used to identify potential interactions between warehouse workers and warehouse robots in automated warehouses. The robot can be equipped with one or more short range transmission tags, such as radio frequency identification (RFID) tags, while the warehouse worker can be equipped with a short range transmission reader, such as an RFID reader. The robot can detect a warehouse worker that is within range when the RFID tags on the robot are written to by the RFID reader. The warehouse robots and warehouse workers can also be equipped with one or more cameras to identify fiducials in the automated warehouse and to report their positions. A central control or interaction server can ensure that warehouse robots and warehouse workers are routed appropriately to avoid incidents.

Claims

1. A method to maintain a threshold distance between a worker and a robot in a workspace, the method comprising:

receiving, at a server from an imaging device associated with the robot, first location information for the robot in the workspace at a first time, wherein the server is external to the worker and the robot;

receiving, at the server from an imaging device associated with the worker, second location information for the worker in the workspace at the first time;

designating, with the server, a work zone encompassing at least the worker and comprising at least an inner barrier and an outer barrier;

calculating, with the server, a distance between the robot and the worker based at least in part on the first location information for the robot and the second location information for the worker; and

transmitting, from the server to the robot, an instruction to take evasive action based at least in part on the robot crossing the outer barrier;

wherein the first location information for the robot comprises first fiducial data from a first fiducial, the first fiducial proximate to the robot at the first time;

wherein the second location information for the worker comprises second fiducial data from a second fiducial, the second fiducial proximate to the worker at the first time.

2. The method of claim 1, wherein the first fiducial data from the first fiducial comprises a bar code received from the imaging device associated with the robot;

and

wherein the bar code comprises at least a location of the first fiducial.

3. The method of claim 1, wherein the evasive action comprises at least one of rerouting, slowing down, or stopping the robot.

4. The method of claim 1, further comprising:

transmitting, from the server to the robot, another instruction to take a different evasive action based at least in part on the robot crossing the inner barrier.

5. The method of claim 1, wherein the first fiducial data from the first fiducial comprises an image of the first fiducial received from the imaging device associated with the robot.

6. A system comprising:

a robot to perform one or more tasks in a workspace, the robot comprising;

one or more imaging devices disposed on the robot to provide imagery of portions of the workspace proximate to the robot;

a plurality of fiducials, each comprising fiducial data, attached to a floor in the workspace; and

a cart to carry objects throughout the workspace, the cart associated with a worker and comprising;

one or more imaging devices disposed on the cart to provide imagery of portions of the workspace proximate to the cart;

a server in communication with the cart and the robot, the server receiving location information from the cart and the robot and establishing a barrier around the cart, wherein the server is external to the worker and the robot;

wherein the server instructs the robot to take evasive action based at least in part on the robot crossing the barrier around the cart.

7. The system of claim 6, wherein each of the plurality of fiducials comprises a radio frequency identification (RFID) tag, the RFID tag comprising at least one of location data or a fiducial identification number for each of the plurality of fiducials.

8. The system of claim 6, wherein each of the plurality of fiducials comprises a bar code, the bar code comprising at least one of location data or a fiducial identification number for each of the plurality of fiducials.

9. The system of claim 6, wherein the cart further comprises:

a tether linking the worker to the cart;

wherein a signal is sent to the server based at least in part on a distance detected by the tether between the worker and the cart exceeding a predetermined distance.

10. The system of claim 9, the tether further comprising:

a harness having a first end and a second end, the first end detachably coupleable to the worker and the second end detachably coupleable to a switch on the cart;

wherein the harness detaches from the switch based at least in part on the distance between the worker and the cart exceeding the predetermined distance; and

wherein the signal is sent to the server based at least in part on the harness detaching from the switch.

11. The system of claim 9, the tether further comprising:

a short range transmission tag coupled to a wearable device associated with the worker; and

a short range transmission reader coupled to the cart and configured to read the short range transmission tag;

wherein the signal is sent to the server based at least in part on the distance between the short range transmission tag and the short range transmission reader exceeding a range at which the short range transmission reader can read the short range transmission tag.

12. The system of claim 11, wherein:

the short range transmission tag comprises a radio frequency identification (RFID) tag; and

the short range transmission reader comprises an RFID reader.

13. The system of claim 6, wherein the server instructs the robot to take escalating evasive action as a distance between the robot and the cart decreases.

14. The system of claim 6, wherein the server instructs the robot to at least one of:

reroute based at least in part on a distance between the robot and the cart reaching a first predetermined distance;

slow down based at least in part on the distance between the robot and the cart reaching a second predetermined distance;

or

stop based at least in part on the distance between the robot and the cart reaching a third predetermined distance.

15. A method comprising:

receiving, at a server from one or more robot imaging devices associated with a robot, first location information for the robot in a workspace;

receiving, at the server from one or more cart imaging devices associated with a cart, second location information for the cart associated with a worker, wherein the server is external to the worker, the cart and the robot, the cart comprising;

a carrying tray for carrying one or more objects;

the one or more cart imaging devices to provide imagery proximate to the cart to the server;

a wireless transceiver to provide wireless communication between the cart and the server; and

a processor in communication with the one or more cart imaging devices and the wireless transceiver to send one or more signals to the server via the wireless transceiver;

designating, with the server, a barrier encompassing at least the cart; and

transmitting, from the server to the robot, an instruction to take evasive action based at least in part on the robot crossing the barrier.

16. The method of claim 15, wherein the first location information for the robot comprises imagery data from the one or more robot imaging devices including fiducial data for one or more fiducials proximate to the robot.

17. The method of claim 15, wherein the second location information for the cart comprises imagery from the one or more cart imaging devices including fiducial data for one or more fiducials proximate to the cart.

18. The method of claim 15, wherein the cart further comprises:

a tether to detect a distance between the worker and the cart exceeding a predetermined distance;

the method further comprising;

receiving a signal, at the server from the processor, based at least in part on the distance between the worker and the cart exceeding the predetermined distance; and

sending, from the server to the robot, another instruction to stop based at least in part on the signal.

19. The method of claim 18, further comprising:

receiving, at the server from the processor, another signal based at least in part on the distance between the worker and the cart falling below the predetermined distance; and

sending, from the server to the robot, a resume instruction to resume normal activity based at least in part on the another signal.

20. The method of claim 18, wherein the tether comprises:

one or more radio frequency identification (RFID) tags disposed on one of the worker or the cart; and

an RFID reader disposed on the other of the worker or the cart;

wherein the server receives the signal from the processor based at least in part on the RFID reader no longer reading at least one of the one or more RFID tags.