Structural light parameter calibration device and method based on front-coating plane mirror

• US 10,690,492 B2
• Filed: 03/22/2018
• Issued: 06/23/2020
• Est. Priority Date: 06/12/2017
• Status: Active Grant

• Current Assignee: Beihang University
• Original Assignee: Beihang University
• Inventors: Wei, Zhenzhong, Zou, Wei, Chai, Binhu, Min, Yue
• Primary Examiner(s): Ton, Tri T
• Application Number:
  US16/308,644
• Publication Number:
  US 20200132451A1
• Time in Patent Office: 824 Days
• Field of Search: 356600-640
• CPC Class Codes: G01B 11/2504 Calibration devices
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 21/042 Calibration or calibration artifacts G01B3/30, G01B9/02072 take precedence

Abstract

The present invention discloses a structural light parameter calibration device and method based on a front-coating plane mirror. The calibration device includes a camera, a laser, a front-coating plane mirror, a flat glass target, and white printing paper. The white printing paper receives a laser beam and presents a real light strip image, and the camera captures the real light stripe image and a mirror light stripe image on the front-coating plane mirror. Feature points are used to determine a rotation matrix, a translation vector, and a vanishing point for the image. The present invention achieves better quality of light stripe and better extraction accuracy, provides feature points with micron-level positional accuracy and more calibration points, and features higher calibration accuracy and more stable calibration results.

Claims

1. A structural light parameter calibration device, comprising:

a camera, a laser, a front-coating plane minor, a flat glass target, and white printing paper, wherein;

the white printing paper is used to receive a beam projected by the laser and present a real light stripe image;

the laser is a line laser which is used to project the beam onto the white printing paper to form the real light stripe image;

the camera is an area array camera which is used to simultaneously capture the real light stripe image projected by the laser on the white printing paper and a mirror light stripe image in the front-coating plane minor, and calculate a rotation matrix and a translation vector between a coordinate system of the front-coating plane minor and a coordinate system of the camera;

feature points around a front surface of the front-coating plane mirror are used to calculate the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the camera;

coated film in a central area of the front surface of the front-coating plane mirror is used to mirror the real light stripe image projected on the white printing paper and feature points on the flat glass target;

the flat glass target is used to provide constraints for optimizing the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the camera, and to calculate a vanishing point of an image, and match the real light stripe image and the minor light stripe image by using the vanishing point.

2. The structural light parameter calibration device of claim 1, wherein the front-coating plane mirror uses a double-sided polished optical glass.

3. The structural light parameter calibration device of claim 1, wherein the feature points around the front-coating plane mirror are in the form of a checkerboard, a dot matrix or a grid;

a positional accuracy of the feature points is on the order of micrometers;

the central area of the front surface of the front-coating plane mirror is coated with an aluminum film or a silver film;

a surface accuracy of the front-coating plane mirror is less than λ

, λ

is a wavelength of visible light in the unit of nm, and a reflectivity of the front-coating plane mirror is ≥

92%; and

a light source with adjustable brightness is provided on a back of the front-coating plane mirror.

4. The structural light parameter calibration device of claim 1, wherein a front surface of the flat glass target is engraved with checkerboard feature points, dot matrix feature points or grid feature points, a light source with adjustable brightness is provided on a back of the flat glass target, and a positional accuracy of the feature points is on the order of micrometers.

5. The structural light parameter calibration device of claim 1, wherein the white printing paper has a flat surface without apparent visible crease, and is non-reflective and light-proof.

11. The structural light parameter calibration of claim 2, wherein the feature points around the front-coating plane mirror are in the form of a checkerboard, a dot matrix or a grid;

a positional accuracy of the feature points is on the order of micrometers;

the central area of the front surface of the front-coating plane mirror is coated with an aluminum film or a silver film;

a surface accuracy of the front-coating plane mirror is less than λ

, λ

is a wavelength of visible light in the unit of nm, and a reflectivity of the front-coating plane mirror is ≥

92%; and

a light source with adjustable brightness is provided on a back of the front-coating plane mirror.

6. A structural light parameter calibration method based on a front-coating plane mirror, comprising steps of:

a. calibrating internal parameters of a camera in a line structured light vision sensor;

placing a flat glass target and the front-coating plane mirror in a clear imaging area in front of the camera, adjusting a brightness of a light source, and capturing a real feature point image and a mirror feature image on the flat glass target; and

correcting the real feature point image and the mirror feature image;

b. establishing a coordinate system of a real camera, a coordinate system of a mirror camera, a coordinate system of the front-coating plane mirror, and a coordinate system of an inverse plane mirror;

solving a rotation matrix and a translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the real camera;

solving a rotation matrix and a translation vector between the coordinate system of the mirror camera and the coordinate system of the real camera, solving a transformation relationship between a left-handed image coordinate system and a right-hand image coordinate system, and establishing a virtual binocular measurement model;

obtaining an optimal solution of the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the real camera by using a nonlinear optimization method;

c. calculating a distance between adjacent feature points in a horizontal direction and a vertical direction of a flat glass target;

determining and selecting candidate feature points based on a threshold, and matching the candidate feature points;

obtaining an image vanishing point by using a least square method; and

d. placing white printing paper in a clear imaging area in front of the camera for multiple times, extracting a center of a real light stripe image and a center of a mirror light stripe image respectively, matching sub-pixels of the center of the real light stripe image and the center of the mirror light stripe image according to the vanishing point, calculating three-dimensional coordinates of a center point of the real light stripe image using the virtual binocular measurement model, fitting a light plane by using a least square method, and solving light plane parameters.

7. The structural light parameter calibration method of claim 6, wherein the placing a flat glass target and the front-coating plane mirror in a clear imaging area in front of the camera, adjusting a brightness of a light source, and capturing a real feature point image and a mirror feature image on the flat glass target in step a comprises steps of:

placing a flat glass target capable of freely moving in the clear imaging area in front of the camera;

forming an angle between the front-coating plane mirror and the flat glass target;

ensuring that feature points on the flat glass target and mirrored feature points thereof, as well as most of feature points of the front-coating plane mirror are located in the clear imaging area; and

adjusting brightness of light sources of the front-coating plane mirror and the flat glass target separately, such that the feature points thereon are clearly imaged, a width of the feature points and a width of edge pixels are 1-3 pixels.

8. The structural light parameter calibration method of claim 6, wherein the establishing a coordinate system of a real camera, a coordinate system of a mirror camera, a coordinate system of the front-coating plane mirror, and a coordinate system of an inverse plane mirror;

solving a rotation matrix and a translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the real camera;

solving a rotation matrix and a translation vector between the coordinate system of the mirror camera and the coordinate system of the real camera, solving a transformation relationship between a left-handed image coordinate system and a right-hand image coordinate system, and establishing a virtual binocular measurement model;

obtaining an optimal solution of the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the real camera by using a nonlinear optimization method in step b comprises steps of;

(1) establishing a right-handed coordinate system for the coordinate system of the real camera, the coordinate system of the mirror camera, the coordinate system of the front-coating plane mirror, and the coordinate system of the inverse plane mirror, wherein an origin of a coordinate system of the real camera is in an upper left corner of an image, and an origin of a coordinate system of a mirror camera image is in an upper right corner of the image;

(2) after the feature points on the front-coating plane mirror are extracted, calculating a homography matrix by using a camera imaging model, and then solving the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the real camera;

(3) respectively solving the rotation matrix and the translation vector between the coordinate system of the front-coating plane mirror and the coordinate system of the inverse plane minor, and the rotation matrix and the translation vector between the coordinate system of the inverse plane mirror and the coordinate system of the mirror camera by using a mirror principle, and solving the rotation matrix and the translation vector between the coordinate system of the mirror camera and the coordinate system of the real camera by using the above-obtained rotation matrixes and the translation vectors;

(4) converting the left-handed image coordinate system to the right-hand image coordinate system, establishing image coordinates of a mirrored light stripe captured by the real camera in the left-handed coordinate system due to mirroring reasons, while maintaining an ordinate of a principal point of the image unchanged, establishing image coordinates of a mirror stripe in the right-hand image coordinate system, and establishing the virtual binocular measurement model based on a transformation matrix and the translation vector between the coordinate system of the mirror camera and the coordinate system of the real camera solved in step (3) as well as an internal parameter matrix obtained by camera calibration; and

(5) using a minimum distance between a measured value and a true value of adjacent feature points in the horizontal direction and the vertical direction on the flat glass target as a spatial distance constraint, obtaining the optimal solution of the rotation matrix and the translation vector between the coordinate system of the front-coating plane minor and the coordinate system of the real camera, and the optimal solution of the rotation matrix and the translation vector between the coordinate system of the real camera and the coordinate system of the mirror camera by using a Levenberg-Marquardt nonlinear optimization method.

9. The structural light parameter calibration method of claim 6, wherein the calculating a distance between adjacent feature points in the horizontal direction and the vertical direction of the flat glass target;

determining and selecting candidate feature points based on a threshold, and matching the candidate feature points;

obtaining an image vanishing point by using a least square method in step c comprises steps of;

(1) extracting and matching real feature points on the flat glass target and mirror feature points;

according to the optimal solution of the rotation matrix and the translation vector between the coordinate system of the real camera and the coordinate system of the mirror camera, substituting the image coordinates of the real feature points on the flat glass target and the image coordinates of the corresponding mirror feature points into the virtual binocular measurement model established in step b respectively, to calculate three-dimensional coordinates of the real feature points on the flat glass target in coordinate system of the real camera;

calculating a spacing of two adjacent feature points in the horizontal direction and in a vertical direction on the flat glass target respectively; and

selecting a feature point whose spacing is smaller than a set spacing threshold as a real candidate feature point;

(2) connecting the real candidate feature point on the flat glass target and a mirror candidate feature point corresponding to the real candidate feature points with a line, using this line as a matching polar line, taking a distance from the image vanishing point to the polar line as an objective function, and solving an intersection of all lines by using a linear least square method, wherein the intersection is the image vanishing point.

10. The structural light parameter calibration method of claim 6, wherein the placing white printing paper in a clear imaging area in front of the camera for multiple times, extracting the center of a real light stripe image and the center of a mirror light stripe image respectively, matching sub-pixels of the center of the real light stripe image and the center of the mirror light stripe image according to the vanishing point, calculating three-dimensional coordinates of the center point of the light stripe using the virtual binocular measurement model, fitting a light plane by using a least square method, and solving light plane parameters in step d comprises steps of:

(1) securing the front-coating plane mirror in the clear imaging area in front of the camera;

adjusting a brightness of the light source to make the feature points on the front-coating plane mirror clear, wherein a width of the feature points and a width of edge pixels are 1-3;

placing the white printing paper in the clear imaging area in front of the camera for multiple times;

projecting the laser beam onto the white printing paper and forming a clear and complete mirror image in front of the front-coating plane mirror;

when the white printing paper is placed at each position, the camera simultaneously capturing a real light stripe image projected on the white printing paper and a mirror light stripe image mirrored by the front-coating plane mirror, and using the same as a calibration image;

(2) extracting a center of a light stripe of the real light stripe image and the mirror light stripe image on the calibration image respectively by using a Steger method;

connecting the center of the real light stripe image and the image vanishing point obtained in step c with a line, and using this line as a polar line of the center of the current light stripe; and

locating two points on the center of the mirrored light stripe closest to the polar line as two candidate points and connecting the two candidate points, and using an intersection of the polar line and the line connecting the two candidate points as a sub-pixel matching point of the center of the real light stripe; and

(3) substituting image coordinates of the center of the real light stripe and image coordinates of the sub-pixel matching point into the virtual binocular measurement model established in step b, to calculate three-dimensional coordinates of the center point of the light stripe, and fitting a light plane equation aX+bY+cZ+d=0 by using the least square method, wherein a, b, c, d in the equation are the solved light plane parameters.