Method For Displaying Recognition Result Obtained By Three-Dimensional Visual Sensor And Three-Dimensional Visual Sensor

• US 20100232683A1
• Filed: 02/23/2010
• Published: 09/16/2010
• Est. Priority Date: 03/11/2009
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Ikeda, Yasuyuki, Yano, Hiroshi, Taneno, Atsushi, Fujieda, Shiro
• Granted Patent: US 8,280,151 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/75 involving models

Abstract

Display suitable to an actual three-dimensional model or a recognition-target object is performed when stereoscopic display of a three-dimensional model is performed while correlated to an image used in three-dimensional recognition processing. After a position and a rotation angle of a workpiece are recognized through recognition processing using the three-dimensional model, coordinate transformation of the three-dimensional model is performed based on the recognition result, and a post-coordinate-transformation Z-coordinate is corrected according to an angle (elevation angle f) formed between a direction of a line of sight and an imaging surface. Then perspective transformation of the post-correction three-dimensional model into a coordinate system of a camera of a processing object is performed, and a height according to a pre-correction Z-coordinate at a corresponding point of the pre-coordinate-transformation three-dimensional model is set to each point of a produced projection image. Projection processing is performed from a specified direction of a line of sight to a point group that is three-dimensionally distributed by the processing, thereby producing a stereoscopic image of the three-dimensional model.

Claims

1. A method for displaying recognition result of a three-dimensional visual sensor, the three-dimensional visual sensor including:

an imaging unit that includes at least one camera;

a registration unit in which three-dimensional information expressing a full-scale recognition-target object is registered as a three-dimensional model; and

a recognition unit that performs three-dimensional measurement to a recognition-target object whose three-dimensional model is registered in the registration unit using an image produced by imaging of the imaging unit and checks three-dimensional information restored by the measurement with the registered three-dimensional model to recognize a position and an attitude of the recognition-target object, on condition that a world coordinate system is defined such that a distance from a reference plane traversing a direction of an optical axis of the camera in the imaging unit becomes a Z-coordinate indicating a height,

wherein at least one of the cameras is set to a processing object,

a first step, a second step, a third step, and a fourth step are performed every camera of the processing object,

in the first step, a manipulation of specification of a direction of a line of sight with respect to an imaging surface of the camera being received,

in the second step, a plurality of Z-coordinates included in a three-dimensional model being corrected to perform perspective transformation of a post-correction three-dimensional model into a coordinate system of the imaging surface of the camera of the processing object for the three-dimensional model to which coordinate transformation is performed in a world coordinate system such that the position and attitude of the three-dimensional model become identical to those of the recognition-target object recognized by the recognition unit,

in the third step, perspective transformation of a point having a three-dimensional coordinate value obtained by setting a height according to a corresponding pre-correction Z-coordinate into a point having a three-dimensional coordinate value obtained by setting a point included in an projection image of the three-dimensional model produced in the coordinate system of the imaging surface of the camera of the processing object by the perspective transformation processing being performed from the direction of the line of sight received in the first step,

in the fourth step, the projection image produced by the perspective transformation in the third step being displayed on a monitor device, and

a rule of correction operation is applied to the Z-coordinate correction in the second step,

in the rule of correction operation, a correction amount to each Z-coordinate being set to zero when an orthogonal relationship is established between the direction of the line of sight received in the first step and the imaging surface of the camera of the processing object, each Z-coordinate being corrected to a constant value Z0 when the direction of the line of sight becomes parallel to the imaging surface, each post-correction Z-coordinate being brought close to the constant value Z0 as the relationship between the direction of the line of sight and the imaging surface becomes close to a parallel state from the orthogonal state, a difference between post-correction Z-coordinates Z1′ and

Z2′

of any two of Z-coordinates Z1 and Z2 (Z1>

Z2) being gradually decreased while a relationship of Z1′

>

Z2′

is maintained.

2. The method according to claim 1, wherein a step of specifying a size per one pixel is previously performed to the camera of the processing object based on a relationship between the coordinate system of the camera and the world coordinate system, the size per one pixel being allocated to the imaging surface in performing perspective transformation of a plane in which a height in a space is zero into the imaging surface of the camera of the processing object, and

in the third step, for each point of the projection image of the three-dimensional model, the pre-correction Z-coordinate at a corresponding point of the pre-coordinate-transformation three-dimensional model is converted into the number of pixels based on the size per one pixel specified in the step, and the number of pixels is set to the height after the conversion.

3. A three-dimensional visual sensor comprising:

an imaging unit that includes at least one camera;

a registration unit in which three-dimensional information expressing a full-scale recognition-target object is registered as a three-dimensional model;

a recognition unit that performs three-dimensional measurement to a recognition-target object whose three-dimensional model is registered in the registration unit using an image produced by imaging of the imaging unit and checks three-dimensional information restored by the measurement with the registered three-dimensional model to recognize a position and an attitude of the recognition-target object, on condition that a world coordinate system is defined such that a distance from a reference plane traversing a direction of an optical axis of the camera in the imaging unit becomes a Z-coordinate indicating a height;

a specifying manipulation receiving unit that sets at least one of the cameras to a processing object to receive a manipulation of specification of a direction of a line of sight with respect to an imaging surface of the camera of the processing object;

a first perspective transformation unit that corrects a plurality of Z-coordinates included in a three-dimensional model to perform perspective transformation of a post-correction three-dimensional model into a coordinate system of the imaging surface of the camera of the processing object based on a relationship between the direction of the line of sight received by the specifying manipulation receiving unit and the imaging surface of the camera of the processing object, the three-dimensional model to which coordinate transformation is performed in a world coordinate system such that the position and attitude of the three-dimensional model become identical to those of the recognition-target object recognized by the recognition unit;

a second perspective transformation unit that performs perspective transformation of a point having a three-dimensional coordinate value obtained by setting a height according to a corresponding pre-correction Z-coordinate into a point having a three-dimensional coordinate value obtained by setting a point included in an projection image of the three-dimensional model produced in the coordinate system of the imaging surface of the camera of the processing object by the perspective transformation processing performed through the first perspective transformation unit from the direction of the line of sight received in the specifying manipulation receiving unit; and

a display control unit that displays the projection image produced through the perspective transformation performed by the second perspective transformation unit on a monitor device,

wherein a rule of correction operation is applied to the Z-coordinate correction in the first perspective transformation unit,

in the rule of correction operation, a correction amount to each Z-coordinate being set to zero when an orthogonal relationship is established between the direction of the line of sight received in the first step and the imaging surface of the camera of the processing object, each Z-coordinate being corrected to a constant value Z0 when the direction of the line of sight becomes parallel to the imaging surface, each post-correction Z-coordinate being brought close to the constant value Z0 as the relationship between the direction of the line of sight and the imaging surface becomes close to a parallel state from the orthogonal state, a difference between post-correction Z-coordinates Z1′ and

Z2′

of any two of Z-coordinates Z1 and Z2 (Z1>

Z2) being gradually decreased while a relationship of Z1′

>

Z2′

is maintained.