ROBOTIC SYSTEMS AND METHODS FOR IDENTIFYING AND PROCESSING A VARIETY OF OBJECTS

• US 20170121113A1
• Filed: 09/09/2016
• Published: 05/04/2017
• Est. Priority Date: 09/11/2015
• Status: Active Grant

• Current Assignee: Berkshire Grey Operating Co., Inc. (SoftBank Group Corp.)
• Original Assignee: Berkshire Grey Inc (SoftBank Group Corp.)
• Inventors: WAGNER, Thomas, AHEARN, Kevin, COHEN, Benjamin, DAWSON-HAGGERTY, Michael, GEYER, Christopher, KOLETSCHKA, Thomas, MARONEY, Kyle, MASON, Matthew T., PRICE, Gene Temple, ROMANO, Joseph, SMITH, Daniel, SRINIVASA, Siddhartha, VELAGAPUDI, Prasanna, ALLEN, Thomas
• Granted Patent: US 10,007,827 B2
• Time in Patent Office: Days
• CPC Class Codes: B07C 5/36 Sorting apparatus characterised by the means used for distribution sorting according to destination B07C3/003, B07C3/02
• CPC Class Codes: B25J 19/022 using lasers
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/0093 co-operating with conveyor means
• CPC Class Codes: B25J 9/1679 characterised by the tasks executed
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 1/1373 for fulfilling orders in warehouses
• CPC Class Codes: G05B 19/124 using tapes, cards or discs with optically sensed marks or codes G05B19/128, G05B19/14 take precedence
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: G05B 2219/14005 Alarm
• CPC Class Codes: G05B 2219/36371 Barcode reader
• CPC Class Codes: G05B 2219/45045 Maintenance, automatic storage and retrieval system
• CPC Class Codes: G05B 2219/45047 Sorting
• CPC Class Codes: G05B 2219/45063 Pick and place manipulator
• CPC Class Codes: G06K 19/06028 using bar codes
• CPC Class Codes: G06K 7/10693 for omnidirectional scanning
• CPC Class Codes: G06K 7/1404 Methods for optical code recognition
• CPC Class Codes: G06K 7/1443 locating of the code in an image
• CPC Class Codes: G06K 7/1447 extracting optical codes from image or text carrying said optical code
• CPC Class Codes: G06Q 10/087 : Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06V 20/64 : Three-dimensional objects
• CPC Class Codes: G06V 2201/06 : Recognition of objects for industrial automation
• CPC Class Codes: G06V 40/1312 : direct reading, e.g. contactless acquisition
• CPC Class Codes: Y02P 90/02 : Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A robotic system is disclosed that include an articulated arm and a first perception system for inspecting an object, as well as a plurality of additional perception systems, each of which is arranged to be directed toward a common area in which an object may be positioned by the robotic arm such that a plurality of views within the common area may be obtained by the plurality of additional perception systems.

Claims

1. A robotic system including an articulated arm, said robotic system comprising:

a first perception system for inspecting an object; and

a plurality of additional perception systems, each of which is arranged to be directed toward a common area in which an object may be positioned by the robotic arm such that a plurality of views within the common area may be obtained by the plurality of additional perception systems.

2. The robotic system as claimed in claim 1, wherein the first perception system identifies at least one object identifying marking, and determines an object shape that encloses the at least one object identifying marking.

3. The robotic system as claimed in claim 1, wherein the first perception system generates a candidate pick list of objects to be selected from an inbound area.

4. The robotic system as claimed in claim 3, wherein the candidate pick list of objects ranks at least some of the objects corresponding to a desired pick order.

5. The robotic system as claimed in claim 1, wherein the first perception system includes a camera that is positioned above a location into which are provided articles for identification and processing by the robotic system.

6. The robotic system as claimed in claim 1, wherein the first perception system includes a scanner that is positioned above a location into which are provided articles for identification and processing by the robotic system.

7. The robotic system as claimed in claim 6, wherein the scanner is a laser reflectivity scanner.

8. The robotic system as claimed in claim 6, wherein the scanner is a radio frequency ID scanner.

9. The robotic system as claimed in claim 1, wherein the plurality of additional perception systems include a plurality of cameras.

10. The robotic system as claimed in claim 9, wherein the plurality of cameras includes five cameras that are collectively arranged to simulate holding the object in at least four different poses.

11. The robotic system as claimed in claim 9, wherein the plurality of cameras are provided in a drop camera unit into which an object may be dropped and scanned by the plurality of cameras as the object is falling through the unit.

12. The robotic system as claimed in claim 1, wherein the plurality of additional perception systems include a plurality of scanners.

13. The robotic system as claimed in claim 12, wherein each of the plurality of scanners is a laser reflectivity scanner.

14. The robotic system as claimed in claim 12, wherein each of the plurality of scanners is a radio frequency ID scanner.

15. The robotic system as claimed in claim 12, wherein the plurality of scanners includes five scanners that are collectively arranged to simulate holding the object in at least four different poses.

16. The robotic system as claimed in claim 12, wherein the plurality of scanners are provided in a drop scanner unit into which an object may be dropped and scanned by the plurality of scanners as the object is falling through the unit.

17. The robotic system as claimed in claim 1, wherein the plurality of additional perception systems are positioned such that every surface of the object may be perceived regardless of the orientation of the object.

18. The robotic system as claimed in claim 1, wherein the first perception system provides perception data regarding a unique object identifier, and wherein the plurality of additional perception systems provide additional perception data regarding any of confirming the object identifier or any additional object identifiers.

19. The robotic system as claimed in claim 1, wherein the first perception system provides perception data regarding an object shape, and wherein the plurality of additional perception systems provide additional perception data regarding a unique object identifier that is confirmed as matching the object shape.

20. A method of identifying an object in a robotic system including an articulated arm, said method comprising the steps of:

inspecting the object using a first perception system and providing an inspection response signal representative of whether the item has been identified;

moving the object to a plurality of additional perception systems responsive to the inspection response signal; and

inspecting the object using the plurality of additional perception systems, each of which is arranged to be directed toward the object in a common area from a different view of a plurality of views of the object within the common area that may be obtained by the plurality of additional perception systems.

21. The method as claimed in claim 20, wherein the method of inspecting the object using a first perception system involves providing perception data regarding the object as well as a plurality of additional objects near the object.

22. The method as claimed in claim 21, wherein the method further includes the step of providing supplemental perception data following the step of moving the object to confirm that the object was selected and moved.

23. The method as claimed in claim 20, wherein the method further includes the step of moving the object proximate the first perception system by a conveyor.

24. The method as claimed in claim 20, wherein the method further includes the step of moving the object proximate the first perception system together with a plurality of other, different products.

25. The method as claimed in claim 20, wherein the step of moving the object to a plurality of additional perception systems responsive to the inspection response signal employs using the articulated arm.

26. The method as claimed in claim 20, wherein the method further includes the step of verifying by using one of the first perception system or the plurality of additional perception systems, whether an identification of an object that has been picked up by an end effector of the articulated arm is the same as an identification of an object that was selected to be picked up by the end effector.

27. The method as claimed in claim 20, wherein the plurality of additional perception systems include a plurality of cameras.

28. The method as claimed in claim 27, wherein the plurality of cameras are provided in a drop camera unit, and the method further includes the step of dropping an object into the drop camera unit and scanned by the plurality of cameras as the object is falling through the unit.

29. The method as claimed in claim 20, wherein the plurality of additional perception systems include a plurality of scanners.

30. The method as claimed in claim 29, wherein the plurality of scanners are provided in a drop scanner unit, and the method further includes the step of dropping an object into the drop scanner unit and scanned by the plurality of scanners as the object is falling through the unit.

31. The method as claimed in claim 20, wherein the plurality of additional perception systems are positioned such that every surface of the object may be perceived regardless of the orientation of the object.

32. The method as claimed in claim 20, wherein the first perception system provides perception data regarding a unique object identifier, and wherein the plurality of additional perception systems provide additional perception data regarding any of confirming the object identifier or any additional object identifiers.

33. The method as claimed in claim 20, wherein the first perception system provides perception data regarding an object shape, and wherein the plurality of additional perception systems provide additional perception data regarding a unique object identifier that is confirmed as matching the object shape.

34. A perception system for assisting in identifying an object, said perception system comprising a plurality of perception units that are each positioned to be directed toward a plurality of locations along an object path that an object may take as it travels through the perception system.

35. The perception system as claimed in claim 34, wherein the plurality of perception units include a plurality of cameras.

36. The perception system as claimed in claim 34, wherein the plurality of perception units include a plurality of scanners.

37. The perception system as claimed in claim 34, wherein the perception system includes at least one sensor system for detecting whether an object has entered the detection unit.

38. The perception system as claimed in claim 37, wherein the at least one sensor system includes a laser source and a laser sensor.

39. The perception system as claimed in claim 34, wherein the plurality of perception units include perception units at least at the bottom and central regions of the perception system.