Identification and planning system and method for fulfillment of orders

• US 10,399,778 B1
• Filed: 10/25/2018
• Issued: 09/03/2019
• Est. Priority Date: 10/25/2018
• Status: Expired due to Fees

• Current Assignee: Grey Orange Pte. Ltd.
• Original Assignee: Grey Orange Pte. Ltd.
• Inventors: Shekhawat, Anirudh Singh, Narkar, Sameer, Patil, Akash, Kumar, Avilash, Tolia, Vaibhav
• Primary Examiner(s): Cumbess, Yolanda R
• Application Number:
  US16/170,958
• Time in Patent Office: 313 Days
• Field of Search: None
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 1/1373 for fulfilling orders in warehouses
• CPC Class Codes: B65G 1/1378 the orders being assembled on fixed commissioning areas remote from the storage areas
• CPC Class Codes: B65G 2203/0216 Codes or marks on the article
• CPC Class Codes: G05B 2219/31286 Detect position of articles and equipment by receivers, identify objects by code
• CPC Class Codes: G05B 2219/40005 Vision, analyse image at one station during manipulation at next station
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

A identification and planning system includes a scanner assembly, a robotic manipulator, and control server. The control server detects a first plurality of inventory items in a first inventory storage system and determines a first set of position parameters for a first inventory item from the detected first plurality of inventory items, with respect to a position of one or more image sensors in the scanner assembly. The control server generates a plurality of pick-path plans for the first inventory item, where each pick-path plan corresponds to a transformation of the determined first set of position parameters to a second set of position parameters with respect to the robotic manipulator. The control server further selects a first pick-path plan from the plurality of pick-path plans and controls the robotic manipulator to pick the first inventory item from the first inventory storage system, based on the first pick-path plan.

Claims

1. An identification and planning system, comprising:

a scanner assembly that comprises one or more image sensors;

a robotic manipulator that comprises a robotic arm and an end effector detachably attached to the robotic arm; and

a control server comprising control circuitry, wherein the control circuitry is configured to;

detect, by the one or more image sensors, a first plurality of inventory items in a first inventory storage system;

determine a first set of position parameters for a first inventory item from the detected first plurality of inventory items, the first set of position parameters are determined with respect to a position of an image sensor of the one or more image sensors in the scanner assembly;

generate a plurality of pick-path plans for the first inventory item, wherein each pick-path plan of the plurality of pick-path plans corresponds to a transformation of the determined first set of position parameters to a second set of position parameters with respect to a spatial differentiation caused by a difference in position of the scanner assembly and the robotic manipulator;

select a first pick-path plan from the plurality of pick-path plans, wherein the first pick-path plan is selected before a pick operation is executed on the first inventory item, and the selection of the first pick-path plan is based on a success of the pick operation in a test computer-simulation, the success of the pick operation occurs when the second set of position parameters corresponding to the selected first pick-path plan allow the robotic manipulator to reach and hold the first inventory item; and

in the pick operation, control, based on the first pick-path plan, the robotic arm and the end effector to pick the first inventory item from the first inventory storage system.

2. The identification and planning system according to claim 1, further comprising a first mobile robot, wherein the first inventory storage system is mounted on the first mobile robot that is configured to move the first inventory storage system from a first location to a second location in a warehouse based on instructions received from the control circuitry.

3. The identification and planning system according to claim 1, wherein the first inventory storage system comprises a first plurality of storage bins, wherein each storage bin of the first plurality of storage bins is located at different spatial positions in the first inventory storage system, the first plurality of storage bins accommodates the first plurality of inventory items.

4. The identification and planning system according to claim 1, wherein the control circuitry is further configured to execute, using the one or more image sensors, a scanning operation of the first plurality of inventory items in the first inventory storage system.

5. The identification and planning system according to claim 4, wherein, in the scanning operation, the control circuitry is further configured to extract color information and depth information from a plurality of three dimensional (3D) images of the first plurality of inventory items, and

wherein the scanning operation further comprises the detection of the first plurality of inventory items and the determination of first set of position parameters.

6. The identification and planning system according to claim 1, further comprising a second inventory storage system that comprises a second plurality of storage bins, wherein each storage bin of the second plurality of storage bins, present at different spatial positions in the second inventory storage system, accommodates a second plurality of inventory items.

7. The identification and planning system according to claim 6, wherein the control circuitry is further configured to execute the pick operation for the first inventory item of the first inventory storage system, in parallel with a scanning operation on the second plurality of inventory items in one or more storage bins of the second inventory storage system.

8. The identification and planning system according to claim 1, wherein the control circuitry is further configured to execute a queuing operation for the first inventory storage system and a second inventory storage system prior to an execution of the pick operation.

9. The identification and planning system according to claim 8, further comprising at least one mobile robot,

wherein, in the queuing operation, the control circuitry is configured to control the at least one mobile robot to position the first inventory storage system and the second inventory storage system in a defined queue of a queuing station.

10. The identification and planning system according to claim 9,

wherein the defined queue comprises grid regions, and

wherein the scanner assembly is positioned in front of a first grid region of the grid regions and the robotic manipulator is positioned in front of a second grid region of the grid regions.

11. The identification and planning system according to claim 10,

wherein, in the queuing operation, the control circuitry is further configured to control the at least one mobile robot to move the first inventory storage system and the second inventory storage system through the grid regions of the defined queue, and

wherein the first inventory storage system and the second inventory storage system are moved for execution of the pick operation for the first inventory item in parallel with the scanning operation on the second plurality of inventory items in the second inventory storage system.

12. The identification and planning system according to claim 1, wherein the control circuitry is further configured to:

estimate, based on a scan by one or more image sensors, three dimensional (3D) information that corresponds to different surface attributes of the first inventory item in the first inventory storage system of the first plurality of inventory items; and

estimate an orientation of the first inventory item in the first inventory storage system, based on the scan.

13. The identification and planning system according to claim 12, wherein the control circuitry is further configured to apply a plurality of transform operations to the determined first set of position parameters to obtain the second set of position parameters, based on the estimated 3D surface information and the estimated orientation of the first inventory item.

14. The identification and planning system according to claim 13, wherein the control circuitry is further configured to verify, in the test computer-simulation, the success of the pick operation on the first inventory item, based on execution of the pick operation corresponding to the application of each transform operation of the plurality of transform operations on the determined first set of position parameters.

15. The identification and planning system according to claim 1, wherein the robotic manipulator further comprises a plurality of actuators in the robotic arm, wherein the plurality of actuators allow a movement of the robotic arm and the end effector within a defined number of degrees of freedom in a three dimensional (3D) space.

16. The identification and planning system according to claim 15,

wherein the success of the pick operation corresponds to an event in the test computer-simulation, in which a simulation of the robotic manipulator picks the first inventory item using a simulated movement of the plurality of actuators, and

wherein the simulated movement is based on a defined number of degrees of freedom of the robotic arm and the second plurality of position parameters in the first pick-path plan.

17. The identification and planning system according to claim 1, further comprising a memory configured to store a set of attributes of the first inventory item prior to a scanning operation and the pick operation,

wherein the control circuitry is further configured to estimate a set of grip parameters associated with the end effector of the robotic manipulator, based on the stored set of attributes of the first inventory item.

18. The identification and planning system according to claim 17, wherein the control circuitry is further configured to verify, in the test computer-simulation, the success of the pick operation of the first inventory item by execution of the pick operation with a plurality of modifications in the estimated set of grip parameters.

19. The identification and planning system according to claim 18, wherein the control circuitry is further configured to select the end effector from a set of end effectors, based on the stored set of attributes of the first inventory item.

20. The identification and planning system according to claim 1, wherein each of the detected first plurality of inventory items in the first inventory storage system is associated with a unique tag that corresponds to a stock keeping unit (SKU).

21. The identification and planning system according to claim 20, wherein the control circuitry is further configured to segregate, prior to a scanning operation and the pick operation, the first plurality of inventory items into a first set of inventory items and a second set of inventory items.

22. The identification and planning system according to claim 21,

wherein the segregation of the first plurality of inventory items is based on the unique tag for each of the first plurality of inventory items, and

wherein the first set of inventory items correspond to inventory items that are auto pickable by the robotic manipulator and the second set of inventory items correspond to inventory items manually pickable by a warehouse operator.

23. The identification and planning system according to claim 1, wherein the first set of position parameters are spatially different from the second set of position parameters, and wherein the spatial differentiation is caused by a difference in a spatial location of the scanner assembly and the robotic manipulator.

24. An identification and planning system, comprising:

a control server comprising control circuitry, wherein the control circuitry is configured to;

detect a first plurality of inventory items in a first inventory storage system;

determine a first set of position parameters for a first inventory item from the detected first plurality of inventory items, the first set of position parameters being determined with respect to a position of an image sensor of one or more image sensors included in a scanner assembly;

generate a plurality of pick-path plans for the first inventory item, wherein each pick-path plan of the plurality of pick-path plans corresponds to a transformation of the determined first set of position parameters to a second set of position parameters with respect to a spatial differentiation caused by a difference in position of the scanner assembly and a robotic manipulator;

select a first pick-path plan from the plurality of pick-path plans, wherein the first pick-path plan is selected before a pick operation is executed on the first inventory item, and the selection of the first pick-path plan is based on a success of the pick operation in a test computer-simulation, the success of the pick operation occurs when the second set of position parameters corresponding to the selected first pick-path plan allow the robotic manipulator to reach and hold the first inventory item; and

in the pick operation, control, based on the selected first pick-path plan, a robotic arm and an end effector detachably attached to the robotic arm to pick the first inventory item from the first inventory storage system.

25. A method of an identification and planning system that includes control circuitry and one or more image sensors, the method comprising:

detecting, by the one or more image sensors, a first plurality of inventory items in a first inventory storage system;

determining, by the control circuitry, a first set of position parameters for a first inventory item from the detected first plurality of inventory items, the first set of position parameters being determined with respect to a position of an image sensor of the one or more image sensors included in a scanner assembly;

generating, by the control circuitry, a plurality of pick-path plans for the first inventory item, wherein each pick-path plan of the plurality of pick-path plans corresponds to a transformation of the determined first set of position parameters to a second set of position parameters with respect to a spatial differentiation caused by a difference in position of the scanner assembly and a robotic manipulator;

selecting, by the control circuitry, a first pick-path plan from the plurality of pick-path plans, wherein the first pick-path plan is selected before a pick operation is executed on the first inventory item, and the selection of the first pick-path plan is based on a success of the pick operation in a test computer-simulation, the success of the pick operation occurs when the second set of position parameters corresponding to the selected first pick-path plan allow the robotic manipulator to reach and hold the first inventory item; and

in the pick operation, controlling, by the control circuitry and based on the selected first pick-path plan, a robotic arm and an end effector detachably attached to the robotic arm to pick the first inventory item from the first inventory storage system.