IMAGE PROCESSING SYSTEM, IMAGE PROCESSING APPARATUS, WORKPIECE PICKUP METHOD, AND WORKPIECE PICKUP PROGRAM

• US 20180194573A1
• Filed: 10/19/2017
• Published: 07/12/2018
• Est. Priority Date: 01/10/2017
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: IWAI, Yosuke, IIDA, Toyoo, SHIMAMURA, Junji
• Granted Patent: US 10,407,250 B2
• Time in Patent Office: Days
• CPC Class Codes: B25J 9/0093 co-operating with conveyor means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65G 15/12 with two or more endless belts
• CPC Class Codes: B65G 47/32 Applications of transfer devices such devices per se B65G47/74
• CPC Class Codes: G05B 19/4182 manipulators and conveyor only
• CPC Class Codes: G05B 19/4189 characterised by the transport system
• CPC Class Codes: G05B 2219/39102 Manipulator cooperating with conveyor
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/75 involving models

Abstract

An imaging unit has its imaging field of view containing the conveyor, and images a subject in the imaging field. The imaging unit generates a luminance image representing the subject and a range image including height information indicating a height of at least one point on the subject from the conveyor by imaging the subject. A measurement unit measures a position of each workpiece in the luminance image. An obtaining unit obtains an overlap order of the workpieces based on the height information in the range image corresponding to the position of each workpiece in the luminance image. A determination unit determines the workpiece pickup order to allow a workpiece having a highest place in the overlap order to be picked up with a higher priority than other workpieces.

Claims

1. An image processing system for determining a workpiece pickup order in which a plurality of overlapping workpieces being transported on a conveyor are picked up by a robot, the system comprising:

an imaging unit arranged to have an imaging field of view containing the conveyor, and configured to image a subject included in the imaging field of view, the imaging unit being configured to generate a luminance image representing the subject and a range image including height information indicating a height of at least one point on the subject from the conveyor by imaging the subject;

a measurement unit configured to measure a position of each of the plurality of workpieces in the luminance image;

an obtaining unit configured to obtain an overlap order in which the plurality of workpieces overlap one another based on the height information in the range image corresponding to the position of each of the plurality of workpieces in the luminance image; and

a determination unit configured to determine the workpiece pickup order for the plurality of workpieces to allow a workpiece having a highest place in the overlap order to be picked up with a higher priority than other workpieces.

2. The image processing system according to claim 1, wherein

the determination unit determines the workpiece pickup order for each workpiece to allow a workpiece having a higher place in the overlap order to be picked up earlier.

3. The image processing system according to claim 1, wherein

when the luminance image includes a plurality of workpiece groups each including a plurality of overlapping workpieces, the obtaining unit obtains the overlap order for each workpiece group, and

the determination unit determines the workpiece pickup order for a workpiece having a highest place in the overlap order in each workpiece group to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

4. The image processing system according claim 1, wherein

when the luminance image includes at least one workpiece that does not overlap another workpiece and includes a plurality of overlapping workpieces, the determination unit determines the workpiece pickup order for the at least one workpiece that does not overlap another workpiece and a workpiece having a highest place in the overlap order among the plurality of overlapping workpieces to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

5. The image processing system according to claim 1, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

6. The image processing system according to claim 1, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces,

a first photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and

a second photoreceiver configured to receive reflected light from the subject included in the imaging field of view to generate the luminance image.

10. The image processing system according to claim 2, wherein

when the luminance image includes a plurality of workpiece groups each including a plurality of overlapping workpieces, the obtaining unit obtains the overlap order for each workpiece group, and

the determination unit determines the workpiece pickup order for a workpiece having a highest place in the overlap order in each workpiece group to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

11. The image processing system according to claim 2, wherein

when the luminance image includes at least one workpiece that does not overlap another workpiece and includes a plurality of overlapping workpieces, the determination unit determines the workpiece pickup order for the at least one workpiece that does not overlap another workpiece and a workpiece having a highest place in the overlap order among the plurality of overlapping workpieces to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

12. The image processing system according to claim 3, wherein

when the luminance image includes at least one workpiece that does not overlap another workpiece and includes a plurality of overlapping workpieces, the determination unit determines the workpiece pickup order for the at least one workpiece that does not overlap another workpiece and a workpiece having a highest place in the overlap order among the plurality of overlapping workpieces to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

13. The image processing system according to claim 10, wherein

when the luminance image includes at least one workpiece that does not overlap another workpiece and includes a plurality of overlapping workpieces, the determination unit determines the workpiece pickup order for the at least one workpiece that does not overlap another workpiece and a workpiece having a highest place in the overlap order among the plurality of overlapping workpieces to allow a workpiece more downstream in a transport path on the conveyor to be picked up with a higher priority.

14. The image processing system according to claim 2, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

15. The image processing system according to claim 3, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

16. The image processing system according to claim 4, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

17. The image processing system according to claim 10, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

18. The image processing system according to claim 11, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

19. The image processing system according to claim 12, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

20. The image processing system according to claim 13, wherein

the imaging unit includes

a projector configured to project, onto the plurality of workpieces transported on the conveyor, linear light that diverges in a direction orthogonal to a transportation direction of the plurality of workpieces, and

a photoreceiver configured to successively receive reflected light of the linear light from the plurality of workpieces transported on the conveyor to generate the range image based on a position of the received reflected light upon every reception of light, and receive reflected light from the subject included in the imaging field of view to generate the luminance image.

7. An image processing apparatus for determining a workpiece pickup order in which a plurality of overlapping workpieces being transported on a conveyor are picked up by a robot, the apparatus comprising:

an imaging unit arranged to have an imaging field of view containing the conveyor, and configured to image a subject included in the imaging field of view, the imaging unit being configured to generate a luminance image representing the subject and a range image including height information indicating a height of at least one point on the subject from the conveyor by imaging the subject;

a measurement unit configured to measure a position of each of the plurality of workpieces in the luminance image;

an obtaining unit configured to obtain an overlap order in which the plurality of workpieces overlap one another based on the height information in the range image corresponding to the position of each of the plurality of workpieces in the luminance image; and

a determination unit configured to determine the workpiece pickup order for the plurality of workpieces to allow a workpiece having a highest place in the overlap order to be picked up with a higher priority than other workpieces.

8. A pickup method for determining a workpiece pickup order in which a plurality of overlapping workpieces being transported on a conveyor are picked up by a robot, the method comprising:

imaging a subject included in an imaging field of view using an imaging unit arranged to have the imaging field of view containing the conveyor, the imaging including generating a luminance image representing the subject and a range image including height information indicating a height of at least one point on the subject from the conveyor by imaging the subject;

measuring a position of each of the plurality of workpieces in the luminance image;

obtaining an overlap order in which the plurality of workpieces overlap one another based on the height information in the range image corresponding to the position of each of the plurality of workpieces in the luminance image; and

determining the workpiece pickup order for the plurality of workpieces to allow a workpiece having a highest place in the overlap order to be picked up with a higher priority than other workpieces.

9. A non-transitory computer-readable recording medium storing a pickup program for determining a workpiece pickup order in which a plurality of overlapping workpieces being transported on a conveyor are picked up by a robot, the program causing a computer to perform operations comprising:

imaging a subject included in an imaging field of view using an imaging unit arranged to have the imaging field of view containing the conveyor, the imaging including generating a luminance image representing the subject and a range image including height information indicating a height of at least one point on the subject from the conveyor by imaging the subject;

measuring a position of each of the plurality of workpieces in the luminance image;

obtaining an overlap order in which the plurality of workpieces overlap one another based on the height information in the range image corresponding to the position of each of the plurality of workpieces in the luminance image; and

determining the workpiece pickup order for the plurality of workpieces to allow a workpiece having a highest place in the overlap order to be picked up with a higher priority than other workpieces.