System and method for improved scoring of 3D poses and spurious point removal in 3D image data

• US 10,482,621 B2
• Filed: 07/20/2017
• Issued: 11/19/2019
• Est. Priority Date: 08/01/2016
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Hoelscher, Andrew, Wallack, Aaron S., Wagman, Adam, Michael, David J., Jia, Hongjun
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): Lemieux, Ian L
• Application Number:
  US15/654,793
• Publication Number:
  US 20180130224A1
• Time in Patent Office: 852 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

This invention provides a system and method for estimating match of a 3D alignment pose of a runtime 3D point cloud relative to a trained model 3D point cloud. It includes scoring a match of a candidate pose of the runtime 3D point cloud relative to the trained model 3D point cloud, including a visibility check that comprises (a) receiving a 3D camera optical center (b) receiving the trained model 3D point cloud; (c) receiving the runtime 3D point cloud; and (d) constructing a plurality of line segments from the optical center to a plurality of 3D points in the 3D point cloud at the runtime candidate pose. A system and method for determining an accurate representation of a 3D imaged object by omitting spurious points from a composite point cloud based on the presence or absence of such points in a given number of point clouds is also provided.

Claims

1. A method in a vision system for estimating a degree of match of a 3D alignment pose of a runtime 3D point cloud with respect to a trained model 3D point cloud comprising the steps of:

scoring, with a vision system processor, a match of a candidate pose of the runtime 3D point cloud relative to the trained model 3D point cloud, including providing a visibility check that comprises

(a) receiving an optical center of a 3D camera,

(b) receiving the trained model 3D point cloud,

(c) receiving the runtime 3D point cloud, and

(d) constructing a plurality of line segments from the optical center to a plurality of 3D points in the trained model 3D point cloud or the runtime 3D point cloud at the runtime candidate pose; and

determining, based upon a location of the 3D points along respective line segments, whether to exclude or include the 3D points in the step of scoring.

2. The method as set forth in claim 1 further comprising, restricting consideration of self-occluding 3D point information free-of explicitly rendering a view of the trained model 3D point cloud at the runtime 3D point cloud candidate pose.

3. The method as set forth in claim 1 wherein the step of scoring includes at least one of (a) computing a coverage score defining a sum of included 3D points and (b) computing a clutter score defining a sum of excluded runtime 3D points.

4. The method as set forth in claim 3 wherein the step of defining the sum of included 3D points includes using at least one of a point-to-plane, point-to-line and point-to-point metric.

5. The method as set forth in claim 1 wherein at least one of (a) the step of scoring includes operating the visibility check incrementally, and (b) the visibility check is performed based upon a range image.

6. The method as set forth in claim 5 wherein the visibility check is performed by testing if the 3D points are visible based upon at least one of (a) an adjacency test and (b) a point-against-surface-mesh test.

7. The method as set forth in claim 1 wherein the 3D camera is part of a 3D camera assembly having a plurality of 3D cameras interconnected to the vision system processor.

8. The method as set forth in claim 7 wherein the runtime 3D point cloud is acquired from an object imaged by at least two of the 3D cameras in which a portion of the object is imaged within the working section of each of the at least two 3D cameras.

9. The method as set forth in claim 1 wherein the runtime 3D point cloud is acquired from an object that is self-occluded within the working section of the 3D camera.

10. A system in a vision system for estimating a degree of match of a 3D alignment pose of a runtime 3D point cloud with respect to a trained model 3D point cloud comprising:

a scoring process, operating in a vision system processor, that scores a match of a candidate pose of the runtime 3D point cloud relative to the trained model 3D point cloud, including a visibility check process that is arranged to

(a) receive an optical center of a 3D camera,

(b) receive the trained model 3D point cloud,

(c) receive the runtime 3D point cloud, and

(d) construct a plurality of line segments from the optical center to a plurality of 3D points in the trained model 3D point cloud or the runtime 3D point cloud at the runtime candidate pose; and

a determination process that, based upon a location of the 3D points along respective line segments, determines whether to exclude or include the 3D points.

11. The system as set forth in claim 10 wherein the scoring process is arranged to (a) compute a coverage score defining a sum of included 3D points and (b) compute a clutter score defining a sum of excluded runtime 3D points.

12. The system as set forth in claim 10 wherein at least one of (a) the scoring process is arranged to operate the visibility check incrementally, and (b) the visibility check process is performed based upon a range image.

13. The system as set forth in claim 12 wherein the visibility check process is performed by testing if the 3D points are visible based upon at least one of (a) an adjacency test and (b) a point-against-surface-mesh test.

14. The system as set forth in claim 13 wherein the test is performed by SIMD machine-accelerated hardware.

15. The system as set forth in claim 10 wherein the 3D camera is part of a 3D camera assembly having a plurality of 3D cameras interconnected to the vision system processor.

16. The system as set forth in claim 15 wherein the runtime 3D point cloud is acquired from an object imaged by at least two of the 3D cameras in which a portion of the object is imaged within the working section of each of the at least two 3D cameras.

17. A system for removing spurious points from a 3D image of an object comprising:

a plurality of 3D cameras arranged to acquire images of an object within a working section thereof from a plurality of respective points of view;

one or more vision system processors configured to operate;

a visibility process that,

(a) receives a measured 3D point cloud from a 3D camera in the plurality of 3D cameras,

(b) generates presumed poses for the object relative to the measured 3D point cloud, and

(c) uses information relative to location and orientation of the 3D camera with respect to the object to determine visibility of points of the 3D point cloud; and

a composite point cloud generation process that combines the 3D point cloud from the plurality of 3D cameras into a composite 3D point cloud free of spurious points by omitting points that are not corroborated by appearing in a predetermined number of point clouds in which such points are expected to be visible, based on the visibility process.

18. The system as set forth in claim 17 wherein the visibility process is based on a respective measured position of the 3D camera and a respective local surface normal for points in the measured 3D point cloud with respect to the 3D camera.

19. The system as set forth in claim 18 wherein the local surface normal is generated based upon nearby points to the point containing the surface normal.

20. The system as set forth in claim 17 wherein the predetermined number of point clouds is defined by an automatic or user-defined threshold.