Three-dimensional sensor system and three-dimensional data acquisition method

• US 10,309,770 B2
• Filed: 05/27/2017
• Issued: 06/04/2019
• Est. Priority Date: 09/14/2016
• Status: Active Grant

• Current Assignee: Hangzhou Scantech Co., Ltd.
• Original Assignee: Hangzhou Scantech Co., Ltd.
• Inventors: Zheng, Jun, Chen, Shangjian
• Primary Examiner(s): Chowdhury, Tarifur R
• Assistant Examiner(s): Nixon, Omar H
• Application Number:
  US15/573,487
• Publication Number:
  US 20180356213A1
• Time in Patent Office: 738 Days
• Field of Search: 356364, 356603
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A three-dimensional system includes at least one pattern projector, at least two cameras, a two-dimensional image feature extractor, a three-dimensional point-cloud generator, and a three-dimensional point-cloud verifier. The pattern projector is configured to synchronously project at least two linear patterns. The at least two cameras are configured to synchronously capture a two-dimensional image of the scanned object. The two-dimensional image feature extractor is configured to extract a two-dimensional line set of the at least two linear patterns on the surface of the scanned object on the two-dimensional image. The three-dimensional point-cloud generator is configured to generate the candidate three-dimensional point set based on the two-dimensional line set. The three-dimensional point-cloud verifier is configured to select an authentic three-dimensional point set correctly matching with the projection contour lines on the surface of the object from the candidate three-dimensional point set.

Claims

1. A three-dimensional sensor system comprising at least one pattern projector, at least three cameras, and a processor, wherein

the pattern projector is used for projecting at least two linear patterns simultaneously;

the at least three cameras are used for synchronously capturing a two-dimensional image of a scanned object;

the processor extracts a two-dimensional line set of the at least two linear patterns of a surface of the scanned object on the two-dimensional image;

the processor generates a candidate three-dimensional point set from the two-dimensional line set, comprising;

respectively extracting two-dimensional point data from a two-dimensional line set of two synchronous two-dimensional images, and

calculating the candidate three-dimensional point set using a spatial position relationship of two cameras corresponding to the two synchronous two-dimensional images and according to a trigonometry principle and an epipolar constraint principle; and

the processor selects out an authentic three-dimensional point set that that correctly matches a projection contour line of the object surface from the candidate three-dimensional point set, comprising;

performing data verification on the candidate three-dimensional point set using a two-dimensional image photographed by a third camera or more cameras, and

filtering same to obtain an authentic three-dimensional point set,

wherein the processor performing data verification on the candidate three-dimensional point set using a two-dimensional image photographed by a third camera or more cameras and filtering same to obtain an authentic three-dimensional point set comprise;

the candidate three-dimensional point set comprising a plurality of subsets, there being an intersection point set between a line connecting the subsets to an optical center of the third camera and the two-dimensional image photographed by the third camera, and

the processor selecting out, a subset corresponding to the minimum distance value, the authentic three-dimensional point set , taking a distance from the intersection point set to a two-dimensional line on the two-dimensional image photographed by the third camera as a basis, and

wherein the processor extracting a two-dimensional line set of the at least two linear patterns of a surface of the scanned object on the two-dimensional image comprises;

performing distortion correction on the two-dimensional image according to an internal reference of a camera corresponding to the two-dimensional image,

extracting a connected region of line contour in the corrected image according to a pixel gray-scale difference, and

calculating and obtaining a highlight center two-dimensional line set with a sub-pixel level according to a gray-level centroid of the connected region.

2. The three-dimensional sensor system of claim 1 wherein

the linear patterns are projected by one of the pattern projectors, or are simultaneously projected by a plurality of pattern projectors;

and

the linear patterns are straight lines or curve lines.

3. The three-dimensional sensor system of claim 1 wherein

the pattern projector comprises a linear laser and a DOE beam splitting element, the linear laser splitting a plurality of laser line segments by means of the DOE beam splitting element.

4. The three-dimensional sensor system of claim 1 wherein

the pattern projector comprises a projector, the projector directly projecting the at least two linear patterns.

5. The three-dimensional sensor system of claim 1 wherein the three-dimensional sensor system comprises a synchronizing trigger,

the synchronizing trigger being used for triggering the cameras and the pattern projector to work synchronously.

6. A three-dimensional sensor system, characterized in that the system comprises at least one pattern projector, at least two cameras, and a processor, wherein

the pattern projector projects at least two linear patterns simultaneously;

the at least two cameras synchronously captures a two-dimensional image of a scanned object;

the processor extracts a two-dimensional line set of the at least two linear patterns of a surface of the scanned object on the two-dimensional image;

the processor generates a candidate three-dimensional point set from the two-dimensional line set, wherein generating a candidate three-dimensional point set from the two-dimensional line set comprises;

extracting two-dimensional point data from the two-dimensional line set of the image captured by any one of the cameras, and

obtaining, using a spatial position relationship between a plurality of spatial light planes projected by the pattern projector and the camera, the candidate three-dimensional point set according to a trigonometry principle; and

the processor selects out an authentic three-dimensional point set that correctly matches with a projection contour line of the object surface from the candidate three-dimensional point set, comprising;

checking the candidate three-dimensional point set and an image photographed by at least one another camera, and

filtering same to obtain an authentic three-dimensional point set,

wherein checking the candidate three-dimensional point set and an image photographed by at least one another camera and filtering same to obtain an authentic three-dimensional point set comprise;

the candidate three-dimensional point set comprising several subsets,

there being an intersection point set between a line connecting the subsets to an optical center of the at least one another camera and an image photographed by the at least one another camera, and

the processor selects out a subset corresponding to the minimum distance value, i.e. an authentic three-dimensional point set, taking a distance from the intersection point set to a two-dimensional line on the image photographed by the at least one another camera as a basis, and

wherein the processor extracting a two-dimensional line set of the at least two linear patterns of a surface of the scanned object on the two-dimensional image comprises;

performing distortion correction on the two-dimensional image according to an internal reference of a camera corresponding to the two-dimensional image,

extracting a connected region of line contour in the corrected image according to a pixel gray-scale difference, and

calculating and obtaining a highlight center two-dimensional line set with a sub-pixel level according to a gray-level centroid of the connected region.

7. The three-dimensional sensor system of claim 6, characterized in that

the linear patterns are projected by one of the pattern projectors, or are simultaneously projected by a plurality of pattern projectors;

and

the linear patterns are straight lines or curve lines.

8. The three-dimensional sensor system of claim 6, characterized in that the pattern projector comprises a linear laser and a DOE beam splitting element, the linear laser splitting a plurality of laser line segments by means of the DOE beam splitting element.

9. The three-dimensional sensor system of claim 6, characterized in that the pattern projector comprises a projector, the projector directly projecting the at least two linear patterns.

10. The three-dimensional sensor system of claim 6, characterized in that the three-dimensional sensor system comprises a synchronizing trigger,

the synchronizing trigger being used for triggering the cameras and the pattern projector to work synchronously.

11. A three-dimensional data acquisition method, characterized in that the method comprises the steps of:

a pattern projector projecting at least two linear patterns;

at least three cameras synchronously capturing a two-dimensional image;

a processor extracting a two-dimensional line set of at least two linear patterns of a surface of the scanned object on the two-dimensional image;

the processor generating a candidate three-dimensional point set from the two-dimensional line set, comprising;

respectively extracting two-dimensional point data from a two-dimensional line set of two synchronous two-dimensional images, and

calculating the candidate three-dimensional point set using a spatial position relationship of two cameras corresponding to the two synchronous two-dimensional images and according to a trigonometry principle and an epipolar constraint principle; and

the processor selecting out an authentic three-dimensional point set that correctly matches with a projection contour line of the object surface from the candidate three-dimensional point set, comprising;

performing data verification on the candidate three-dimensional point set using a two-dimensional image photographed by a third camera or more cameras, and

filtering same to obtain an authentic three-dimensional point set,

wherein performing data verification on the candidate three-dimensional point set using a two-dimensional image photographed by a third camera or more cameras and filtering same to obtain an authentic three-dimensional point set comprise;

the candidate three-dimensional point set comprising several subsets,

there being an intersection point set between a line connecting the subsets to an optical center of the third camera and the two-dimensional image photographed by the third camera, and

selecting out a subset corresponding to the minimum distance value, i.e. the authentic three-dimensional point set, taking a distance from the intersection point set to a two-dimensional line on the two-dimensional image photographed by the third camera as a basis, and

wherein extracting a two-dimensional line set of at least two linear patterns of a surface of the scanned object on the two-dimensional image comprises;

performing distortion correction on the two-dimensional image according to an internal reference of a camera corresponding to the two-dimensional image, and

extracting a connected region of line contour in the corrected image according to a pixel gray-scale difference, and

calculating and obtaining a highlight center two-dimensional line set with a sub-pixel level according to a gray-level centroid of the connected region.

12. A three-dimensional data acquisition method, characterized in that the method comprises the steps of:

a pattern projector projecting at least two linear patterns;

at least two cameras synchronously capturing a two-dimensional image;

a processor extracting a two-dimensional line set of at least two linear patterns of a surface of the scanned object on the two-dimensional image;

the processor generating a candidate three-dimensional point set from the two-dimensional line set, comprising;

extracting two-dimensional point data from the two-dimensional line set of the image captured by any one of the cameras, and

obtaining, using a spatial position relationship between a plurality of spatial light planes projected by the pattern projector and the camera, the candidate three-dimensional point set according to a trigonometry principle; and

the processor selecting out an authentic three-dimensional point set that correctly matches with a projection contour line of the object surface from the candidate three-dimensional set, comprising;

checking the candidate three-dimensional point set and an image photographed by at least one another camera, and

filtering same to obtain an authentic three-dimensional point set,

wherein checking the candidate three-dimensional point set and an image photographed by at least one another camera and filtering same to obtain an authentic three-dimensional point set comprise;

the candidate three-dimensional point set comprising several subsets,

there being an intersection point set between a line connecting the subsets to an optical center of the at least one another camera and the image photographed by the at least one another camera, and

selecting out a subset corresponding to the minimum distance value, i.e. the authentic three-dimensional point set, taking a distance from the intersection point set to a two-dimensional line on the image photographed by the at least one another camera as a basis, and

wherein extracting a two-dimensional line set of at least two linear patterns of a surface of the scanned object on the two-dimensional image comprises;

performing distortion correction on the two-dimensional image according to an internal reference of a camera corresponding to the two-dimensional image, and

extracting a connected region of line contour in the corrected image according to a pixel gray-scale difference, and

calculating and obtaining a highlight center two-dimensional line set with a sub-pixel level according to a gray-level centroid of the connected region.