Image processing system, image processing device, and image processing program

• US 10,805,546 B2
• Filed: 04/26/2018
• Issued: 10/13/2020
• Est. Priority Date: 05/17/2017
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Iida, Toyoo, Taniyasu, Yuki
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Sechser, Jill D
• Application Number:
  US15/964,033
• Publication Number:
  US 20180338090A1
• Time in Patent Office: 901 Days
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

An image processing system (SYS) specifies an arrangement situation of a workpiece (W) provided by a line (L). The image processing system (SYS) specifies a normal (V) with respect to a set measurement point (Wp) of the workpiece (W) according to the specified arrangement situation of the workpiece (W), and changes a position and posture of the two-dimensional camera (310) so that the specified normal (V) matches an optical axis of the two-dimensional camera (310) (S1). The image processing system (SYS) changes a distance between the two-dimensional camera (310) and the measurement point (Wp) so that the specified normal (V) matches the optical axis of the two-dimensional camera (310), to focus the two-dimensional camera (310) on the measurement point (Wp).

Claims

1. An image processing system for performing image measurement for one or a plurality of measurement points preset in a workpiece using an appearance image of the workpiece, the image processing system comprising:

a first imaging unit configured to capture the appearance image;

a robot that changes a relative positional relationship between the first imaging unit and the workpiece;

a first image processor configured to specify an arrangement situation of the workpiece based on information obtained by imaging the workpiece and specify a reference line with respect to each measurement point set in the workpiece according to the specified arrangement situation of the workpiece, wherein the reference line is set based on a material of the workpiece and a direction in which an irradiation light is incident;

a processor configured to position the first imaging unit on the specified reference line and give a command to the robot, wherein the robot moves the first imaging unit to a distance between the first imaging unit and the workpiece in a state in which an optical axis of the first imaging unit is matched with the specified reference line;

the first image processor configured to calculate a first feature amount regarding a degree of focusing of the first imaging unit based on the appearance image captured by the first imaging unit; and

the processor configured to determine the distance between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to a change in the distance between the first imaging unit and the workpiece.

2. The image processing system according to claim 1, further comprising:

a second imaging unit that is arranged to include at least a portion of the workpiece and the robot in a range of a field of view and captures a three-dimensional image of a subject present within the range of the field of view;

a memory configured to store information defining a reference line direction associated with each of the one or plurality of measurement points; and

a second image processor configured to search for a portion matching shape information of the workpiece in the three-dimensional image; and

configured to determine a reference line with respect to each measurement point based on information on the portion searched for by the second image processor and the information defining the reference line direction stored in the memory.

3. The image processing system according to claim 2, wherein the first image processor is configured to search for a portion matching a reference image of a corresponding measurement point for the image captured by the first imaging unit in a state in which the first imaging unit is positioned on the reference line determined by the reference line determination part;

and

configured to correct the reference line determined by the reference line determination part based on the information on the portion searched for by the first image processor.

4. The image processing system according to claim 1,

wherein the processor is configured to give a command to the robot so that an angle between the specified reference line and the optical axis of the first imaging unit is changed from 0 to a predetermined value in a state in which the distance between the first imaging unit and the workpiece is maintained;

and

configured to determine a relative positional relationship between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to the change in the angle.

5. The image processing system according to claim 1,

wherein the first image processor is configured to calculate a second feature amount regarding a preset detection target portion based on the appearance image captured by the first imaging unit;

and

configured to determine presence or absence of the detection target portion at a measurement point serving as a target or near the measurement point based on the second feature amount calculated in a state in which the first imaging unit and the workpiece is positioned with a distance therebetween determined by the processor.

6. The image processing system according to claim 5, wherein the first image processor gives a command to the robot to position the first imaging unit at the next measurement point when determination of the presence or absence of the detection target portion performed by the processor is completed.

7. The image processing system according to claim 1, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

8. The image processing system according to claim 7, further comprising a reference line calculation part configured to calculate a surface shape of each measurement point based on the design information of the workpiece for each of the measurement points set by the measurement point setting reception part.

9. The image processing system according to claim 2,

wherein the processor is configured to give a command to the robot so that an angle between the specified reference line and the optical axis of the first imaging unit is changed from 0 to a predetermined value in a state in which the distance between the first imaging unit and the workpiece is maintained;

and

configured to determine a relative positional relationship between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to the change in the angle.

10. The image processing system according to claim 3,

wherein the processor is configured to give a command to the robot so that an angle between the specified reference line and the optical axis of the first imaging unit is changed from 0 to a predetermined value in a state in which the distance between the first imaging unit and the workpiece is maintained;

and

configured to determine a relative positional relationship between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to the change in the angle.

11. The image processing system according to claim 2,

wherein the first image processor is configured to calculate a second feature amount regarding a preset detection target portion based on the appearance image captured by the first imaging unit;

and

configured to determine presence or absence of the detection target portion at a measurement point serving as a target or near the measurement point based on the second feature amount calculated in a state in which the first imaging unit and the workpiece is positioned with a distance therebetween determined by the processor.

12. The image processing system according to claim 3,

wherein the first image processor is configured to calculate a second feature amount regarding a preset detection target portion based on the appearance image captured by the first imaging unit;

and

configured to determine presence or absence of the detection target portion at a measurement point serving as a target or near the measurement point based on the second feature amount calculated in a state in which the first imaging unit and the workpiece is positioned with a distance therebetween determined by the processor.

13. The image processing system according to claim 4,

wherein the first image processor is configured to calculate a second feature amount regarding a preset detection target portion based on the appearance image captured by the first imaging unit;

and

configured to determine presence or absence of the detection target portion at a measurement point serving as a target or near the measurement point based on the second feature amount calculated in a state in which the first imaging unit and the workpiece is positioned with a distance therebetween determined by the processor.

14. The image processing system according to claim 2, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

15. The image processing system according to claim 3, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

16. The image processing system according to claim 4, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

17. The image processing system according to claim 5, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

18. The image processing system according to claim 6, further comprising a measurement point setting reception part configured to display design information of the workpiece and set the one or plurality of measurement points with respect to the displayed design information.

19. An image processing device for performing image measurement for one or a plurality of measurement points preset in a workpiece using an appearance image of the workpiece, the image processing device comprising:

a first interface that receives information on an appearance image from a first imaging unit that captures the appearance image;

a second interface for communicating with a robot that changes a relative positional relationship between the workpiece and the first imaging unit;

a first image processor configured to specify an arrangement situation of the workpiece based on information obtained by imaging the workpiece and specify a reference line with respect to each measurement point set in the workpiece according to the specified arrangement situation of the workpiece, wherein the reference line is set based on a material of the workpiece and a direction in which an irradiation light is incident;

a processor configured to position the first imaging unit on the specified reference line and give a command to the robot, wherein the robot moves the first imaging unit to a distance between the first imaging unit and the workpiece in a state in which an optical axis of the first imaging unit is matched with the specified reference line;

the first image processor configured to calculate a first feature amount regarding a degree of focusing of the first imaging unit based on the appearance image captured by the first imaging unit; and

the processor configured to determine the distance between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to a change in the distance between the first imaging unit and the workpiece.

20. A non-transitory computer-readable recording medium comprising an image processing program for performing image measurement for one or a plurality of measurement points preset in a workpiece using an appearance image of the workpiece, the image processing program causing a computer to execute:

imaging the workpiece using a first imaging unit;

specifying an arrangement situation of the workpiece based on information obtained by imaging the workpiece;

specifying a reference line with respect to each measurement point set in the workpiece according to the specified arrangement situation of the workpiece, wherein the reference line is set based on a material of the workpiece and a direction in which an irradiation light is incident;

positioning the first imaging unit on the specified reference line;

giving a command to the robot that changes a relative positional relationship between the first imaging unit and the workpiece, wherein the robot moves the first imaging unit to a distance between the first imaging unit and the workpiece in a state in which an optical axis of the first imaging unit is matched with the specified reference line;

calculating a first feature amount regarding a degree of focusing of the first imaging unit based on the appearance image captured by the first imaging unit; and

determining the distance between the first imaging unit and the workpiece for performing image measurement for the measurement point based on a change in the first feature amount according to a change in the distance between the first imaging unit and the workpiece.