Automated system for locating and transferring objects on a conveyor belt

• US 5,040,056 A
• Filed: 01/29/1990
• Issued: 08/13/1991
• Est. Priority Date: 01/29/1990
• Status: Expired due to Fees

• Current Assignee: Technistar Corporation
• Original Assignee: Technistar Corporation
• Inventors: Schmehl, Michael R., Sager, James L.
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US07/472,044
• Time in Patent Office: 561 Days
• Field of Search: 358/101, 358/106, 358/903, 382/8, 901/7, 901/8, 364/468, 364/478, 209/577, 209/739
• US Class Current: 348/88
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/0093 co-operating with conveyor means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65B 23/14 Forming groups of biscuits
• CPC Class Codes: B65B 5/105 by grippers B65B5/12 and B65B5/108 take precedence
• CPC Class Codes: G05B 19/4182 manipulators and conveyor only
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

An apparatus and method for picking up and manipulating randomly oriented and randomly positioned objects moving on a belt. An image processing unit using a vision system identifies and locates objects in successive overlapping vision windows up to a predetermined optimum number of objects. The locations of those objects is entered in an output queue which is transmitted to the object location queue of a first robot motion controller. The first robot picks up and deposits all the objects it can in the time available while the objects pass, and enters the locations of the objects not picked up in an output queue which is transmitted to the object location queue of a second robot motion controller. The robots can be arranged in a series of any number and the image processing units and vision systems can also be arranged in a series.

Claims

1. A method for picking up objects from a moving conveyor belt and transferring them to a destination, comprising:

receiving a pixel image of a first vision window image portion of said belt containing images of said objects using first image-receiving means;

digitizing said first vision window image by assigning a gray scale level and a coordinate location to the pixels in said first vision window image;

electronically determining and transmitting, utilizing first data transmission means, the locations of at least one of said objects represented by object images in said first vision window image to a first motion controller directing a first robot means associated with said first image-receiving means located in the direction of belt travel with respect to said first image-receiving means;

picking up at least one of said objects represented by object images in said first vision window image, the locations of which were transmitted to said first motion controller, using said first robot means;

transmitting, utilizing second data transmission means, the locations of at least one of the objects represented by object images in said first vision window image which were not picked up by said first robot means, from said first motion controller to a second motion controller directing a second robot means associated with said first image-receiving means located in the direction of belt travel with respect to said first robot means; and

picking up at least one of said objects represented by object images in said first vision window image, the locations of which were transmitted to said second motion controller, using said second robot means.

2. The method of claim 1, further comprising:

establishing a binary image of said digitized image by assigning 0 or 1 to each pixel gray scale value depending on whether said gray scale value is above or below a predetermined threshold value; and

determining the location and configuration of said objects represented by images in said first window image using said binary image.

3. The method of claim 1, further comprising establishing a pick-up window area on said belt for each said robot means associated with said first vision window, and wherein each motion controller which directs each robot means associated with said first image-receiving means repeatedly directs said robot means to pick up certain of the objects in said robot means'"'"' pick-up window.

4. The method of claim 3, wherein each object that each motion controller directs each robot means to pick up is the object in said robot means'"'"' pick-up window that is farthest in the direction of belt travel.

5. The method of claim 1, wherein the locations of objects that are smaller than a predetermined minimum area are not transmitted to said first motion controller.

6. The method of claim 1, wherein the locations of objects that are larger than a predetermined maximum area are not transmitted to said first motion controller determined by comparing said larger object images to a prototype image, and the locations of objects represented by those larger object images that do not match any of the prototype image are not transmitted to said first motion controller.

7. The method of claim 6, wherein object images larger than a predetermined maximum area are compared to a prototype image, and those portions of said larger images that match said prototype image within preset tolerances are deemed to be images of individual objects and their locations are transmitted to said first motion controller.

8. The method of claim 7, further comprising establishing a vision window image boundary at the edge of said vision window image, and wherein the objects that are at least partially in said vision window image boundary are not picked up.

9. The method of claim 1, wherein the locations of objects in excess of a predetermined optimum number are not transmitted to said first motion controller.

10. The method of claim 9, wherein said objects are located and counted in the order of their distance from one edge of said vision window image.

11. The method of claim 9, wherein said optimum number is equal to the number of objects that the robots associated with said first image-receiving means can pick up considering variables including the speed of the belt and the character of the object.

12. The method of claim 1, further comprising:

receiving a pixel image of a subsequent vision window portion of said belt containing images of said objects using said first image-receiving means, said subsequent vision window having an edge in the direction of belt travel that is a predetermined distance in the direction opposite belt travel from the edge in the direction of belt travel of the adjacent vision window in the direction of belt travel;

digitizing said subsequent vision window image by assigning a gray scale value and a coordinate location to the pixels in said subsequent vision window image;

electronically determining and transmitting, utilizing said first data transmission means, the locations of at least one of said objects represented by object images in said subsequent vision window to said first motion controller directing said first robot means;

picking up at least one of said objects represented by object images in said subsequent vision window image, the locations of which were transmitted to said first motion controller, using said first robot means;

transmitting, utilizing said second data transmission means, the locations of at least one of the objects represented by object images in said subsequent vision window image which were not picked up by said first robot means, from said first motion controller to said second motion controller;

picking up at least one of said objects represented by object images in said subsequent vision window image, the locations of which were transmitted to said second motion controller, using said second robot means.

13. The method of claim 12, wherein each vision window overlaps each adjacent vision window, and wherein the location of objects whose locations were in the adjacent vision window in the direction of belt travel and were previously transmitted to said first motion controller are not transmitted again to said first motion controller.

14. The method of claim 13, wherein the total area on the conveyor belt from which each robot means is capable of picking up objects includes a smaller area from which the robot means can pick up a plurality of objects faster than the robot means can pick up the same number of a plurality of objects from the total area and wherein the robot means picks up objects from only the smaller area.

15. The method of claim 14, wherein said smaller area is larger for robot means away from the said image-receiving means with which they are associated than for robot means close to said image-receiving means with which they are associated.

16. The method of claim 3, wherein said robot means includes a plurality of pick-up cups, and each cup is filled with an object before all the objects in the cups are moved to the destination.

17. The method of claim 13, further comprising:

receiving a pixel image of another first vision window portion of said belt using second image-receiving means in the direction of belt travel from the last robot means in the direction of belt travel associated with said first image-receiving means;

digitizing said another first vision window image by assigning a gray scale value level and coordinate location to pixels is said another first vision window image;

electronically determining and transmitting, utilizing third data transmission means, the locations of at least one of said objects to another first motion controller directing another first robot means located in the direction of belt travel with respect to said second image-receiving means;

picking up at least one of said objects represented by object images in said another first vision window image, the locations of which were transmitted to said another first motion controller, using said another first robot means;

transmitting, utilizing fourth data transmission means, the locations of at least one of the objects represented by object images in said another first vision window image which were not picked up by said another first robot means, from said another first motion controller to another second motion controller directing another second robot means associated with said second image-receiving means located in the direction of belt travel with respect to said another first robot means; and

picking up at least one of said objects represented by object images in said another first vision window image, the locations of which were transmitted to said another second motion controller, using said another second robot means.

18. A method for picking up objects from a moving conveyor belt and moving them to a destination, comprising:

receiving a series of overlapping images of vision window portions of said belt using image-receiving means, said windows being fixed on the belt and moving therewith;

digitizing each of the vision window images in said series, said digitizing including assigning a coordinate location to pixels in said image;

electronically determining and transmitting, utilizing first data transmission means, the locations of at least some of said objects to a first motion controller directing a first robot means, said determining and transmitting being done a vision window image at a time for each vision window image in said series;

picking up at least one of said objects using said first robot means at the time it moves within the reach of said first robot means;

transmitting, utilizing second data transmission means, the locations of at least one of the objects which were not picked up by said first robot means from said first motion controller to a second motion controller directing a second robot means located in the direction of belt travel with respect to said first robot means; and

picking up at least one of said objects using said second robot means at the time the object moves within the reach of said second robot means.

19. A system for picking up and transferring objects located on a moving conveyor belt, comprising:

image-receiving means to receive pixel images of a series of vision window portions of said belt;

image processing means electronically connected with said image receiving means for assigning a location to objects represented in said images;

a plurality of robot means in series adjacent to said belt, each with a directing motion controller, wherein the motion controller directing the first robot means receives the locations of objects from said image processing means and the motion controller directing each subsequent robot means receives the locations of objects not picked up by any preceding robot means from the motion controller directing the immediately preceding robot means; and

means associated with each motion controller for directing each robot which the motion controller directs, to pick up at least some of the objects not picked up by any preceding robot.

20. The system of claim 19, wherein said image processing means determines and transmits to the first motion controller the locations of no more than a predetermined number of objects in each vision window in said series, said number being based on variables including the belt speed and number of robot means in said robot means series.

21. The system of claim 20, wherein said vision windows overlap in the direction of belt travel by a predetermined distance.

22. The system of claim 21 wherein the locations of objects in each vision window is determined and transmitted to the first motion controller a vision window at a time, and the locations of objects transmitted to the first motion controller for one vision window are not transmitted to the first motion controller for an adjacent overlapping vision window.

23. The system of claim 22, wherein the objects that can be reached by each robot means are picked up by said robot means in the order of the direction of belt movement.

24. The system of claim 23, wherein the image processing means does not transmit to the first motion controller the locations of objects smaller than a predetermined minimum area.

25. The system of claim 24, wherein the image processing means compares the shape and size of images that are larger than a predetermined maximum area with a prototype image and assigns separate object locations to each object represented by said larger images if any portion of the larger image matches the prototype image within a predetermined tolerance.

26. The system of claim 24, wherein each of said robot means includes a plurality of end effector cups for picking up objects.