System and method for piece-picking or put-away with a mobile manipulation robot

• US 9,785,911 B2
• Filed: 07/25/2014
• Issued: 10/10/2017
• Est. Priority Date: 07/25/2013
• Status: Active Grant

• Current Assignee: Iam Robotics, Llc
• Original Assignee: Iam Robotics, Llc
• Inventors: Galluzzo, Thomas, Valois, Jean-Sebastien, Altman, Vladimir
• Primary Examiner(s): Logan, Kyle
• Application Number:
  US14/340,896
• Publication Number:
  US 20150032252A1
• Time in Patent Office: 1,173 Days
• Field of Search: 700218
• CPC Class Codes: B25J 11/00 Manipulators not otherwise provided for
• CPC Class Codes: B25J 15/0028 with movable, e.g. pivoting gripping jaw surfaces
• CPC Class Codes: B25J 15/0085 with means for applying an electrostatic force on the object to be gripped
• CPC Class Codes: B25J 15/06 with vacuum or magnetic holding means
• CPC Class Codes: B25J 15/0608 with magnetic holding means
• CPC Class Codes: B25J 15/0616 with vacuum
• CPC Class Codes: B25J 19/021 Optical sensing devices
• CPC Class Codes: B25J 5/007 mounted on wheels
• CPC Class Codes: B25J 9/0093 co-operating with conveyor means
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: B60P 1/5423 attached to the loading platform or similar
• CPC Class Codes: B65G 1/1373 for fulfilling orders in warehouses
• CPC Class Codes: B65G 1/1375 the orders being assembled on a commissioning stacker-crane or truck
• CPC Class Codes: B65G 2209/04 Indication location means
• CPC Class Codes: G05B 2219/37008 Calibration of measuring system, probe, sensor
• CPC Class Codes: G05B 2219/39441 Voice command, camera detects object, grasp, move
• CPC Class Codes: G05B 2219/40298 Manipulator on vehicle, wheels, mobile
• CPC Class Codes: G05B 2219/40543 Identification and location, position of components, objects
• CPC Class Codes: G05B 2219/40564 Recognize shape, contour of object, extract position and orientation
• CPC Class Codes: G05B 2219/45063 : Pick and place manipulator
• CPC Class Codes: G05D 1/0011 : associated with a remote control arrangement
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0248 : in combination with a laser lasers per se H01S
• CPC Class Codes: G06Q 10/087 : Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: Y10S 901/01 : Mobile robot
• CPC Class Codes: Y10S 901/02 : Arm motion controller
• CPC Class Codes: Y10S 901/31 : Gripping jaw
• CPC Class Codes: Y10S 901/40 : Vacuum or mangetic
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

A method and system for piece-picking or piece put-away within a logistics facility. The system includes a central server and at least one mobile manipulation robot. The central server is configured to communicate with the robots to send and receive piece-picking data which includes a unique identification for each piece to be picked, a location within the logistics facility of the pieces to be picked, and a route for the robot to take within the logistics facility. The robots can then autonomously navigate and position themselves within the logistics facility by recognition of landmarks by at least one of a plurality of sensors. The sensors also provide signals related to detection, identification, and location of a piece to be picked or put-away, and processors on the robots analyze the sensor information to generate movements of a unique articulated arm and end effector on the robot to pick or put-away the piece.

Claims

1. A system for piece-picking or piece put-away within a logistics facility comprising:

a central server comprising;

a server communication interface,

one or more server processors, and

a server memory; and

at least one mobile manipulation robot comprising;

a mobile base,

at least one articulated manipulator arm having an end effector, wherein the at least one articulated manipulator arm has a first end portion pivotally carried by the mobile base and a second end portion comprising the end effector, wherein the at least one articulated manipulator arm further comprises an extension tool positioned at or near the second end portion, wherein the extension tool is configured to provide access to a piece to be picked without interference from surrounding pieces or infrastructure within the logistics facility,

at least one piece containment area,

a plurality of sensors,

a remote communication interface,

a memory configured to store robot specific information, and

one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulator arm, wherein the memory comprises computer program instructions executable by the one or more robot processors to receive data from and send data to the central server, process data received from each of the plurality of sensors, and output control signals to the mobile base and the at least one articulated manipulator arm, wherein the plurality of sensors provide signals related to detection, identification, and location of the piece to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the extension tool and the end effector of the at least one articulated manipulator arm to pick the piece with a collision free path throughout the controlled motion of the at least one articulated manipulator arm.

2. The system of claim 1, wherein the server memory comprises computer program instructions executable by the one or more server processors to receive data from a warehouse management system and dispatch the at least one mobile manipulation robot.

3. The system of claim 1, wherein the at least one mobile manipulation robot can autonomously navigate and position itself within the logistics facility by recognition of at least one landmark by at least one of the plurality of sensors.

4. The system of claim 3, wherein the at least one landmark is a vertically mounted marker placed at a specific location within the logistics facility.

5. The system of claim 1, wherein the plurality of sensors are 3D depth cameras, color cameras, grey scale cameras, laser ranging devices, sonar devices, radar devices, or combinations thereof.

6. The system of claim 1, wherein the end effector is a gripper, a suction cup, an electroadhesion end effector, a magnetic end effector, or combinations thereof.

7. The system of claim 1, wherein the end effector is a suction cup which is connected to a vacuum pump through a valve, wherein actuation of the valve is controlled by the one or more robot processors.

8. The system of claim 1, wherein the at least one mobile manipulation robot further comprises:

a user interface comprising a graphical display monitor and an input device.

9. The system of claim 1, wherein the at least one piece containment area is at least one of a platform, a pick-to-kit holder, a container holder, or any combination thereof.

10. The system of claim 1, wherein more than one piece is placed on the at least one piece containment area by the at least one articulated manipulator arm.

11. The system of claim 1, wherein the logistics facility is a warehouse, distribution center, manufacturing facility, or retail facility.

12. The system of claim 1, wherein the extension tool is configured to position the end effector within a storage rack at a sufficient depth to reach the back of the storage rack.

13. The system of claim 1, wherein the server communication interface connects with the remote communication interface to send and receive piece-picking data which includes a unique identification for each piece to be picked, a location within the logistics facility of the pieces to be picked, and a route for the at least one mobile manipulation robot to take within the logistics facility.

14. The system of claim 13, wherein the unique identification for the piece to be picked comprises a shape of the piece, a size of the piece, a weight of the piece, a color of the piece, a property of the construction material of the piece, a visual marking on the piece, a barcode on the piece, or any combination thereof.

15. The system of claim 1, further comprising:

a conveyance device configured to accept pieces from the at least one mobile manipulation robot.

16. The system of claim 15, wherein the conveyance device is a conveyor belt which transfers the accepted pieces from a transfer area to a receiving area.

17. A system for piece-picking or piece put-away within a logistics facility comprising:

a central server comprising;

a server communication interface,

one or more server processors, and

a server memory; and

at least one mobile manipulation robot comprising;

a mobile base,

at least one articulated manipulator arm having an end effector, wherein the at least one articulated manipulator arm has a first end portion pivotally carried by the mobile base and a second end portion comprising the end effector, wherein the at least one articulated manipulator arm further comprises an extension tool positioned at or near the second end portion, and wherein the first end portion of the at least one articulated manipulator arm is mounted on a vertical actuator stage configured to raise or lower the at least one articulated manipulator arm,

at least one piece containment area,

a plurality of sensors,

a remote communication interface,

a memory configured to store robot specific information, and

one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulator arm, wherein the memory comprises computer program instructions executable by the one or more robot processors to receive data from and send data to the central server, process data received from each of the plurality of sensors, and output control signals to the mobile base and the at least one articulated manipulator arm, wherein the plurality of sensors provide signals related to detection, identification, and location of a piece to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector of the at least one articulated manipulator arm to pick the piece.

18. A system for piece-picking or piece put-away within a logistics facility comprising:

a central server comprising;

a server communication interface,

one or more server processors, and

a server memory; and

at least one mobile manipulation robot comprising;

a mobile base,

at least one articulated manipulator arm having an end effector,

at least one piece containment area,

a plurality of sensors, wherein at least one sensor is positioned at a central point on the at least one articulated manipulator arm such that rotation of the at least one articulated manipulator arm directs the at least one sensor to view the at least one piece containment area,

a remote communication interface,

a memory configured to store robot specific information, and

one or more robot processors coupled to the plurality sensors, the memory, the mobile base, and the at least one articulated manipulator arm, wherein the memory comprises computer program instructions executable by the one or more robot processors to receive data from and send data to the central server, process data received from each of the plurality of sensors, and output control signals to the mobile base and the at least one articulated manipulator arm, wherein the plurality of sensors provide signals related to detection, identification, and location of a piece to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector of the at least one articulated manipulator arm to pick the piece.

19. The system of claim 18, wherein the at least one piece containment area comprises a calibration target which allows calibration of the at least one sensor positioned at a central point on the at least one articulated manipulator arm.

20. A system for piece-picking or piece put-away within a logistics facility comprising:

a central server comprising;

a server communication interface,

one or more server processors, and

a server memory; and

at least one mobile manipulation robot comprising;

a mobile base,

at least one articulated manipulator arm having an end effector,

at least one piece containment area, wherein the at least one piece containment area is configured to sense a weight for a piece placed therein,

a plurality of sensors,

a remote communication interface,

a memory configured to store robot specific information, and

one or more robot processors coupled to the plurality sensors, the memory, the mobile base, and the at least one articulated manipulator arm, wherein the memory comprises computer program instructions executable by the one or more robot processors to receive data from and send data to the central server, process data received from each of the plurality of sensors, and output control signals to the mobile base and the at least one articulated manipulator arm, wherein the plurality of sensors provide signals related to detection, identification, and location of the piece to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector of the at least one articulated manipulator arm to pick the piece.

21. A system for piece-picking or piece put-away within a logistics facility comprising:

a central server comprising;

a server communication interface,

one or more server processors, and

a server memory; and

at least one mobile manipulation robot comprising;

a mobile base,

at least one articulated manipulator arm having an end effector,

at least one piece containment area,

a plurality of sensors,

a remote communication interface,

a memory configured to store robot specific information,

one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulator arm, wherein the memory comprises computer program instructions executable by the one or more robot processors to receive data from and send data to the central server, process data received from each of the plurality of sensors, and output control signals to the mobile base and the at least one articulated manipulator arm, wherein the plurality of sensors provide signals related to detection, identification, and location of a piece to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector of the at least one articulated manipulator arm to pick the piece, and

a graphical use interface which displays user information, wherein the user information includes direction indicators which indicate the intended direction of movement of the robot, and error information.

22. A method of piece-picking within a logistics facility, the method comprising:

(a) providing a piece-picking or put-away system comprising;

a central server including a server communication interface, one or more server processors, and a server memory; and

at least one mobile manipulation robot including a mobile base, at least one articulated manipulator arm having an end effector, at least one piece containment area, a plurality of sensors, a remote communication interface, a robot memory configured to store robot specific information, and one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulation arm, wherein the at least one articulated manipulator arm has a first end portion pivotally carried by the mobile base and a second end portion comprising the end effector, wherein the at least one articulated manipulator arm further comprises an extension tool positioned at or near the second end portion, wherein the extension tool is configured to provide access to at least one item to be picked without interference from surrounding items or infrastructure within the logistics facility;

(b) receiving at the central memory via the server communication interface at least one piece-picking order including the at least one item to be picked;

(c) generating at the one or more server processors a piece-picking itinerary based on the at least one piece-picking order, wherein the itinerary includes a unique identification for each item to be picked, a location within a logistics facility of the items to be picked, and a route for the at least one mobile manipulation robot to take within the logistics facility;

(d) receiving at the robot memory the piece-picking itinerary;

(e) moving the at least one mobile manipulation robot along the route to the location within the logistics facility of the items to be picked;

(f) picking the at least one item to be picked from the location using the end effector of the at least one mobile manipulation arm, wherein the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector and extension tool of the at least one articulated manipulator arm to pick the at least one item to be picked with a collision free path throughout the controlled motion of the at least one articulated manipulator arm; and

(g) placing the at least one item to be picked in the at least one piece containment area.

23. The method of claim 22, wherein the plurality of sensors provide signals related to detection, identification, and location of the at least one item to be picked, and the one or more robot processors analyze the sensor information to generate articulated arm control signals to guide the end effector of the at least one articulated manipulator arm to pick the item.

24. The method of claim 22, wherein moving the at least one mobile manipulation robot occurs autonomously by recognition of at least one landmark by at least one of the plurality of sensors.

25. The method of claim 22, wherein the at least one piece-picking order received at the central memory is generated by a warehouse management system.

26. A method of piece-picking within a logistics facility, the method comprising:

(a) providing a piece-picking or put-away system comprising;

a central server including a server communication interface, one or more server processors, and a server memory; and

at least one mobile manipulation robot including a mobile base, at least one articulated manipulator arm having an end effector, at least one piece containment area, a plurality of sensors, a remote communication interface, a robot memory configured to store robot specific information, and one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulation arm, wherein at least one of the plurality of sensors is positioned at a central point on the at least one articulated manipulator arm such that rotation of the at least one articulated manipulator arm directs the at least one sensor to view the at least one piece containment area;

(b) receiving at the central memory via the server communication interface at least one piece-picking order including at least one item to be picked;

(c) generating at the one or more server processors a piece-picking itinerary based on the at least one piece-picking order, wherein the itinerary includes a unique identification for each item to be picked, a location within a logistics facility of the items to be picked, and a route for the at least one mobile manipulation robot to take within the logistics facility;

(d) receiving at the robot memory the piece-picking itinerary;

(e) moving the at least one mobile manipulation robot along the route to the location within the logistics facility of the items to be picked;

(f) picking the at least one item to be picked from the location using the end effector of the at least one mobile manipulation arm; and

(g) placing the at least one item to be picked in the at least one piece containment area.

27. The method of claim 26, wherein the at least one piece containment area comprises a calibration target which allows calibration of the at least one sensor located at a central point on the at least one articulated manipulator arm.

28. A method of piece-picking within a logistics facility, the method comprising:

(a) providing a piece-picking or put-away system comprising;

a central server including a server communication interface, one or more server processors, and a server memory; and

at least one mobile manipulation robot including a mobile base, at least one articulated manipulator arm having an end effector, at least one piece containment area, a plurality of sensors, a remote communication interface, a robot memory configured to store robot specific information, and one or more robot processors coupled to the plurality of sensors, the memory, the mobile base, and the at least one articulated manipulation arm;

(b) receiving at the central memory via the server communication interface at least one piece-picking order including at least one item to be picked;

(c) generating at the one or more server processors a piece-picking itinerary based on the at least one piece-picking order, wherein the itinerary includes a unique identification for each item to be picked, a location within a logistics facility of the items to be picked, and a route for the at least one mobile manipulation robot to take within the logistics facility;

(d) receiving at the robot memory the piece-picking itinerary;

(e) moving the at least one mobile manipulation robot along the route to the location within the logistics facility of the items to be picked;

(f) picking the at least one item to be picked from the location using the end effector of the at least one mobile manipulation arm;

(g) placing the at least one item to be picked in the at least one piece containment area; and

(h) sensing a weight for the at least one item placed in the at least one piece containment area.