System and method for calibrating one or more 3D sensors mounted on a moving manipulator

• US 10,812,778 B1
• Filed: 08/09/2016
• Issued: 10/20/2020
• Est. Priority Date: 11/09/2015
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Wang, Ruibing, Wallack, Aaron S., Michael, David J., Zhu, Hongwei
• Primary Examiner(s): Rahman, Mohammad J
• Application Number:
  US15/232,787
• Time in Patent Office: 1,533 Days
• Field of Search: None
• CPC Class Codes: G01B 11/0608 Height gauges
• CPC Class Codes: G01B 11/245 using a plurality of fixed, simultaneously operating transducers G01B11/2408 - G01B11/2425, G01B11/255 take precedence
• CPC Class Codes: G01B 11/2504 Calibration devices
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

This invention provides a system and method for concurrently (i.e. non-serially) calibrating a plurality of 3D sensors to provide therefrom a single FOV in a vision system that allows for straightforward setup using a series of relatively straightforward steps that are supported by an intuitive graphical user interface (GUI). The system and method requires minimal data input about the scene or calibration object used to calibrate the sensors. 3D features of a stable object, typically employing one or more subobjects, are first measured by one of the image sensors, and then the feature measurements are used in a calibration in which each of the 3D sensors images a discrete one of the subobjects, resolves features thereon and computes a common coordinate space between the plurality of 3D sensors. Sensor(s) can be mounted on the arm of an encoderless robot or other conveyance and motion speed can be measured in setup.

Claims

1. A system for concurrently calibrating one or more 3D sensors in a vision system performing a vision system process comprising:

one or more 3D sensors comprising laser displacement sensors operatively connected to a vision processor assembly, arranged to image a scene containing a stable object while mounted on an encoderless motion conveyance that provides relative motion between the one or more 3D sensors and the stable object along a motion direction, the encoderless motion conveyance being configured for a first interval of at least one of acceleration or non-linear motion and a second interval of straight line, constant velocity motion trajectory during imaging of the scene; and

a calibration module that calibrates the one or more 3D sensors to a common coordinate space by providing measurements of 3D features on the stable object and calibrating the stable object based upon the measurements and the 3D features found from 3D image data of the stable object acquired by the one or more 3D sensors during the relative motion, the calibration module employing speed data computed based upon a scan of the stable object during the interval of straight line, constant velocity motion trajectory.

2. The system as set forth in claim 1 wherein the measurements are generated by at least one of (a) a specification of the stable object and (b) 3D features found in an image of the stable object acquired by one of the plurality of 3D sensors.

3. The system as set forth in claim 1 wherein the laser displacement sensors are mounted in an adjacent manner and are calibrated concurrently so as to tie together an image acquired by the displacement sensors.

4. The system as set forth in claim 1 further comprising a setup process that determines at least one of (a) motion parameters of the conveyance for use in generating the measurements and (b) an exposure for use in generating the measurements.

5. The system as set forth in claim 4, wherein the motion parameters are determined by a setup process that includes inputting of a travel distance of the conveyance and moving the conveyance while polling for motion to determine a time to move across the travel distance.

6. The system as set forth in claim 5 wherein the conveyance a multi-axis robotic manipulator.

7. The system as set forth in claim 1 wherein the calibration module computes calibration of the one or more 3D sensors to the common coordinate space based on a single concurrent expression.

8. The system as set forth in claim 1 wherein the stable object comprises a calibration object having one or more 3D subobjects, each of the subobjects being located on the calibration object so as to be imaged within the field of view of a discrete one of the 3D sensors during calibration.

9. The system as set forth in claim 1 wherein the calibration module generates calibration parameters that map local coordinate spaces the one or more 3D sensors to the common coordinate space.

10. The system as set forth in claim 9 wherein the calibration parameters are defined according to a gradient descent technique that estimates initial parameter values for the gradient descent technique with an initial parameter estimator.

11. The system as set forth in claim 10 wherein the initial parameter estimator considers a feature predication error to select the best possible initial parameter estimation.

12. The system as set forth in claim 9 further comprising a 3D renderer that applies the calibration parameters to grayscale pixel values generated by each of the one or more 3D sensors to render a grayscale image of an object imaged thereby.

13. The system as set forth in claim 1 wherein the calibration module is arranged to scan the stable object in each of a plurality of phases that include orienting the stable object with respect to the conveyance in each of a plurality of orientations, the phases including a measurement phase in which measurements of the stable object are obtained with respect to the one or more 3D sensors and a calibration phase in which the measurements of the stable object from the measurement phase are applied to generate the calibration parameters.

14. The system as set forth in claim 13 wherein the stable object is a calibration object having one or more subobjects that define a 3D surface having intersecting planes.

15. The system as set forth in claim 1 wherein the conveyance comprises a conveyor, a motion stage or a multi-axis robotic manipulator and the calibration object is located at a position remote from a runtime object so that calibration is performed with the displacement sensor in a calibration orientation than a runtime orientation thereof.

20. The system as set forth in claim 1 wherein the first interval occurs at least one of:

prior to imaging the scene or subsequent to imaging the scene.

16. A method for calibrating one or more 3D sensors in a vision system performing a vision system process comprising the steps of:

imaging a scene containing a stable object with one or more 3D sensors operatively comprising laser displacement sensors connected to a vision processor assembly;

providing relative motion between to the one or more 3D sensors and the stable object along a motion direction free of motion-tracking encoder signals delivered from an encoderless motion conveyance to the vision system and the one or more 3D sensors, the encoderless motion conveyance being configured for a first interval of at least one of acceleration or non-linear motion and a second interval of straight line, constant velocity motion trajectory during imaging of the scene; and

calibrating the one or more 3D sensors to a common coordinate space by (a) providing measurements of 3D features on the stable object and (b) calibrating the stable object based upon the measurements and the 3D features found from 3D image data of the stable object acquired by the one or more 3D sensors during the relative motion, wherein speed data is computed for calibrating based upon a scan of the stable object during the interval of straight line, constant velocity motion trajectory of the conveyance.

17. The method as set forth in claim 16 wherein the step of calibrating includes, during a setup procedure, determining an approximate speed of the encoderless motion conveyance in moving, with the one or more 3D sensors, from a start position to a stop position.

18. The method as set forth in claim 17 wherein the motion conveyance comprises a multi-axis robot having a moving arm and end effector to which the one or more 3D sensors is mounted, the robot including a setting that enables the straight line, constant velocity motion trajectory.

19. The method as set forth in claim 16 wherein the stable object comprises a 3D calibration object having one or more 3D subobjects intersecting planes.