Telepresence based inventory pick and place operations through robotic arms affixed to each row of a shelf

• US 9,327,397 B1
• Filed: 05/30/2015
• Issued: 05/03/2016
• Est. Priority Date: 04/09/2015
• Status: Expired due to Fees

• Current Assignee: Codeshelf Inc.
• Original Assignee: Codeshelf Inc.
• Inventors: Williams, Jeff, Bhaskaran, Ravi, Martin, Charlie
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US14/726,458
• Time in Patent Office: 339 Days
• Field of Search: 700/245, 700/246, 700/247, 700/248, 700/253, 700/255, 700/257, 700/258, 700/260, 700/261, 700/264, 318/567, 318/568.11, 318/568.14, 318/568.16, 318/568.17, 318/568.2, 318/568.21, 318/568.25, 414/1, 414/2, 414/5, 414/22.01, 414/227, 414/265, 414/266
• US Class Current: 1/1
• CPC Class Codes: B25J 13/006 by means of a wireless system for controlling one or several manipulators
• CPC Class Codes: B25J 13/025 comprising haptic means
• CPC Class Codes: B25J 3/00 Manipulators of master-slave type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
• CPC Class Codes: B25J 3/04 involving servo mechanisms servo-actuated heads B25J15/02
• CPC Class Codes: B25J 9/0087 Dual arms double SCARA arms B25J9/043
• CPC Class Codes: B25J 9/1687 Assembly, peg and hole, palletising, straight line, weaving pattern movement
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B65G 1/137 with arrangements or automatic control means for selecting which articles are to be removed
• CPC Class Codes: B65G 2209/04 Indication location means
• CPC Class Codes: G05B 2219/39106 Conveyor, pick up article, object from conveyor, bring to test unit, place it
• CPC Class Codes: G05B 2219/40014 Gripping workpiece to place it in another place
• CPC Class Codes: G05B 2219/40078 Sort objects, workpieces
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: G05B 2219/40192 Control modes, velocity for coarse, position for fine, hand for gripper
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic

Abstract

Disclosed are a system and/or a method of telepresence based inventory pick and place operations through actuator controlled robotic arms affixed to each row of a shelf. A method includes mounting a robotic arm at an end of a row of a shelf of inventory on a set of rails affixed to the row of a shelf. The robotic arm is permitted to move horizontally along the row of the shelf. The robotic arm is repositioned along the three axes using a set of actuators. A haptic motion of a human user is mirrored that is remotely using a positioning device (e.g., human may feel the feedback of the remote arm as it touches the objects). An item is placed on a counting platform in front of the robotic arm. The items are placed automatically in the designated location down through a transport means when a pick operation is completed.

Claims

1. A method comprising:

mounting a robotic arm at an end of a row of a shelf of inventory on a set of rails affixed to the row of a shelf;

permitting the robotic arm to move horizontally along the three axes along the row of the shelf;

repositioning the robotic arm along any one of the three axes using a set of actuators, wherein each of the set of actuators are any of a backdrivable, an electrosensing, an electric, a magnetic, a hydraulic, and pneumatic actuators;

contemporaneously mirroring a haptic motion of a human user that is remotely using a positioning device that is communicatively coupled with the robotic arm through an Internet network;

automatic repositioning of the robotic arm along any one of the three axes using the set of actuators responsive to the haptic motion of the human user remotely using the positioning device;

placing an item of inventory on a counting platform in front of the robotic arm using an end effector of the robotic arm based on an action of a human that views the item of inventory on the shelf through a camera affixed to the robotic arm which is communicatively coupled with a computing device associated with the human user operating the positioning device;

moving the item of inventory to a designated location adjacent to the robotic arm using the end effector; and

automatically placing the items in the designated location down through a transport means adjacent to one end of the shelf when a pick operation is completed,

wherein the designated location is at least one of a tote and a storage bin, and

wherein the transport means is at least one of a tube and a lift platform to bring the tote from the designated location to a desired location of the shelf.

2. The method of claim 1,

wherein the item of inventory is placed on a platform having an angled surface such that the storage bin in which the item of inventory is placed is angled upward and does not fall off the shelf when placed on the counting platform, and

wherein the item of inventory is validated based on a weight of an item on the counting platform.

3. The method of claim 1,

wherein a telepresence application is used to provide the positioning device control over the robotic arm remotely through the Internet network.

4. The method of claim 3,

wherein the shelf includes a plurality of rows of the shelf and a plurality of robotic arms, wherein each of the rows of the shelf includes one or more robotic arms that horizontally traverses a particular row in which it is affixed.

5. The method of claim 4,

wherein each of the robotic arms coordinates with a warehouse management server to automatically direct the human user of the telepresence application to each location on the shelf associated with the items needed to be fulfilled in an e-commerce order.

6. The method of claim 4,

wherein each item of an e-commerce order is automatically deposited through respective ones of the plurality of robotic arms into a packing box on a conveyer belt of a distribution center when the items are deposited down through the tube adjacent to the one end of the shelf.

7. The method of claim 6,

wherein the telepresence application detects that the human has completed a movement of the items onto the counting platform and directs the human user to a next one of a plurality of the robotic arms positioned in front of the storage bin where a selection is needed.

8. The method of claim 7,

wherein a neural network automatically monitors a behavior of the human user and continuously learns how to improve the pick and place of a particular type of item from the shelf such that, over time, a control program learns how to control the robotic arm to automatically select the items onto the counting platform without a human intervention or a human haptic control.

9. The method of claim 1,

wherein the end effector of the robotic arm is at least one of a supple rubber end point, a gripping arm, a sticky polymer end, an impactive end effector, an ingressive end effector, an astrictive end effector, and a contigutive end effector.

10. The method of claim 1,

wherein a robot has two robotic arms with actuator control to permit the human user to grab and lift the items.

11. The method of claim 1,

wherein the robotic arm can safely operate around the human operators in a distribution center by slowly coming to a stop and not having to stop abruptly when an adjacent object is detected which is likely to obstruct a motion of the robotic arm through compliance to prevent users and operators being injured.

12. A method of a telepresence application comprising:

determining that a remote positioning device associated with a human user is communicatively coupled with the telepresence application;

validating the human user as an authorized user of the telepresence application using a processor and a memory of a warehouse management server on which the telepresence application operates;

automatically allocating a robotic arm in a distribution center that is assigned to the human user based on an e-commerce order;

contemporaneously repositioning the robotic arm along any one of the three axes using a set of actuators based on a mirrored haptic motion of the human user that is remotely using a positioning device that is communicatively coupled with the robotic arm through an Internet network.

13. The method of the telepresence application of claim 12 further comprising:

providing a view of an item of inventory directly in front of the robotic arm to the human user who is remotely controlling the robotic arm through the positioning device,

wherein the robotic arm to place the item of inventory on a counting platform in front of the robotic arm using an end effector of the robotic arm based on an action of a human that views the item of inventory on a shelf through a camera affixed to the robotic arm which is communicatively coupled with a computing device associated with the human user operating the positioning device based on a place command from the telepresence application;

wherein the robotic arm to validate the item of inventory based on a weight of the item on the counting platform based on a validate command from the telepresence application;

wherein the robotic arm to move the item of inventory in a location adjacent to the robotic arm using the end effector based on a move command from the telepresence application; and

moving the items at the location down through a tube adjacent to one end of the shelf when a pick operation is completed based on a complete command from the telepresence application.

14. The method of claim 12,

wherein an item of inventory is placed on a platform having an angled surface such that a storage bin in which the item of inventory is placed is angled upward and does not fall off a shelf when placed on a counting platform.

15. The method of claim 12,

wherein the telepresence application is used to provide the positioning device control over the robotic arm remotely through the Internet network.

16. The method of claim 14:

wherein the shelf includes a plurality of rows of the shelf and a plurality of robotic arms, wherein each of the rows of the shelf includes a different robotic arm, and

wherein each of the rows of the shelf includes one or more different robotic arms that horizontally traverses a particular row in which it is affixed.

17. The method of claim 15,

wherein each of the robotic arms to coordinate with the warehouse management server to automatically direct the human user of the telepresence application to each location on a shelf associated with the items needed to be fulfilled in the e-commerce order.

18. The method of claim 16,

wherein each item of the e-commerce order is automatically deposited through respective ones the plurality of robotic arms into a packing box on a conveyer belt of the distribution center when the items are deposited down through a tube adjacent to one end of the shelf.

19. The method of claim 17,

wherein the telepresence application detects that a human has completed a movement of the items onto a counting platform and directs the human user to a next one of a plurality of the robotic arms positioned in front of a storage bin where a selection is needed.

20. The method of claim 18,

wherein a neural network to automatically monitor a behavior of the human user and to continuously learn how to better pick and place a particular type of item from the shelf such that, over time, the robotic arm is able to automatically pick the item and place the item at a location without requiring mirroring of a haptic motion of the human user that is remotely using a positioning arm.

21. The method of claim 12:

wherein an end effector of the robotic arm is at least one of a supple rubber end point, a gripping arm, a sticky polymer end, an impactive end effector, an ingressive end effector, an astrictive end effector, and a contigutive end effector, and

wherein the robotic arm to safely operate around the human operators in the distribution center by slowly coming to a stop and not having to stop abruptly when an adjacent object is detected which is likely to obstruct a motion of the robotic arm.

22. A warehouse system, comprising:

a set of robotic arms that are each affixed to the individual rows of shelving in a distribution center to reposition horizontally along the individual rows of shelving along any one of the three axes using a set of actuators;

an Internet network;

a warehouse management server having a telepresence application coupled with the set of robotic arms through the Internet network to provide a human user remote control over each of a set of the robotic arms through the Internet network;

a positioning device coupled with each of the set of robotic arms and the warehouse management server having the telepresence application through the Internet network that is haptically controlled by a human user in a manner such that a haptic motion of the human user of the positioning device is imitated through a mirrored repositioning of the set of robotic arms.

23. The warehouse system of claim 22,

wherein an item of inventory is placed on a platform having an angled surface such that a storage bin in which the item of inventory is placed is angled upward and does not fall off a shelf when placed on a counting platform.

24. The warehouse system of claim 22,

wherein the telepresence application is used to provide the positioning device control over a robotic arm remotely through the Internet network.

25. The warehouse system of claim 24:

wherein a shelf includes a plurality of rows of the shelf and a plurality of robotic arms, wherein each of the rows of the shelf includes a different robotic arm, and

wherein each of the rows of the shelf includes one or more different robotic arm that horizontally traverses a particular row in which it is affixed.

26. The warehouse system of claim 25,

wherein each of the robotic arms to coordinate with the warehouse management server to automatically direct the human user of the telepresence application to each location on the shelf associated with the items needed to be fulfilled in an e-commerce order.

27. The warehouse system of claim 26,

wherein each item of the e-commerce order is automatically deposited through respective ones of the plurality of robotic arms into a packing box on a conveyer belt of the distribution center when the items are deposited down through a tube adjacent to one end of the shelf.

28. The warehouse system of claim 27,

wherein the telepresence application detects that a human has completed a movement of the items onto a counting platform and directs the human user to a next one of a plurality of the robotic arms positioned in front of a storage bin where a selection is needed.

29. The warehouse system of claim 28,

wherein a neural network to automatically monitor a behavior of the human user and to continuously learn how to better pick and place a particular type of item from the shelf such that, over time, the robotic arm is able to automatically pick the item and place the item at the location without requiring mirroring of the haptic motion of the human user that is remotely using a positioning arm.

30. The warehouse system of claim 22,

wherein an end effector of a robotic arm is at least one of a supple rubber end point, a gripping arm, a sticky polymer end, an impactive end effector, an ingressive end effector, an astrictive end effector, and a contigutive end effector, and

wherein the robotic arm to safely operate around the human operators in the distribution center by slowly coming to a stop and not having to stop abruptly when an adjacent object is detected which is likely to obstruct a motion of the robotic arm.