Recognition and pose determination of 3D objects in 3D scenes

• US 8,830,229 B2
• Filed: 01/06/2011
• Issued: 09/09/2014
• Est. Priority Date: 05/07/2010
• Status: Active Grant

• Current Assignee: MvTec Software GmbH (MvTec LLC)
• Original Assignee: MvTec Software GmbH (MvTec LLC)
• Inventors: Drost, Bertram Heinrich, Ulrich, Markus
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): McCulley, Ryan D
• Application Number:
  US12/985,861
• Publication Number:
  US 20110273442A1
• Time in Patent Office: 1,342 Days
• Field of Search: 345/420
• US Class Current: 345/419
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

The invention provides a method for recognizing instances of a 3D object in 3D scene data and for determining the 3D poses of said instances comprising the following steps: (a) providing 3D scene data; (b) selecting at least one reference point from the 3D scene data; (c) computing, for each selected reference point, pose candidates for the 3D object under the assumption that said reference point is part of the 3D object; and (d) computing a set of filtered poses from the pose candidates.

Claims

1. A method for recognizing instances of a 3D object represented by 3D object data in a 3D scene represented by 3D scene data and for determining the 3D poses of said instances comprising the following steps:

(a) providing 3D scene data and 3D object data;

(b) selecting at least one reference point from the 3D scene data;

(c) computing, for each selected reference point, pose candidates for the 3D object under the assumption that said reference point is part of the 3D object, by;

(c1) representing a space of possible pose candidates by two parameters, where a first parameter is a 2D point on the surface of the 3D object data that corresponds to said reference scene point, and a second parameter is a rotation angle;

(c1a) representing the first parameter as index into a set of points selected from the 3D object data and sampling the second parameter;

(c2) creating a counter for each pose space sample of step (c1a);

(c3) selecting a set of scene points from the 3D scene data;

(c4) computing, for each selected scene point, matching poses such that both the selected scene point and the reference point are on the surface of the 3D object by (c4a) computing a point pair descriptor that describes the reference scene point and the current scene point and their geometric relation;

(c4b) searching for point pairs in the 3D object data similar to said scene point pair using said point pair descriptor;

(c4c) computing, for each such similar point pair, the point corresponding to the reference scene point, and the rotation angle that aligns the scene point pair and the object point pair, after aligning the reference scene point, its corresponding object point and their surface normals;

(c5) increasing, for each corresponding point and rotation angle computed in step (c4c), the counter for the corresponding pose space sample; and

(c6) detecting peak counter values in the sampled pose space and selecting the corresponding pose samples as pose candidates.

2. The method of claim 1, wherein a 3D model is computed from the 3D object data before step (b) and is used in the subsequent steps, comprising the following steps:

(a1) selecting at least two sample points from the 3D object data;

(a2) computing the point pair descriptors that describe, for each pair of selected sample points, the two points and their geometric relation;

(a3) creating a global model description that stores the point pairs indexed by the point pair descriptor.

3. The method of claim 2, wherein step (a3) comprises:

(a3a) sampling the point pair descriptors;

(a3b) creating the global model description such that it maps each sampled point pair descriptor to a list of point pairs, where each list contains all pairs of selected sample points with identical sampled point pair descriptor.

4. The method of claim 1, wherein in step (b) the reference points are selected randomly from the 3D scene data or by uniformly sampling the 3D scene data.

5. The method of claim 1, wherein step (c4b) comprises:

(c4b1) sampling the point pair descriptor;

(c4b2) using a data structure that maps the sampled point pair descriptor to a list of point pairs.

6. The method of claim 5, wherein in step (c4b2) a hash table is used as data structure.

7. The method of claim 1, further comprising a step computing a score for each computed pose candidate, where the score describes the consistency between the 3D scene data and the 3D object data under said pose.

8. The method of claim 1 further comprising the step of:

(d) filtering the pose candidates to obtain a set of final poses.

9. The method of claim 8, wherein in step (d) the computation comprises:

(d1) defining a neighborhood relation between the pose candidates;

(d2) computing the score of each pose as the weighted sum of the scores of the neighboring pose candidates;

(d3) selecting the set of filtered poses by ranking the poses by the score computed in (d2).

10. The method of claim 9, wherein the neighborhood relation is defined by thresholding the difference in the translation of the poses and the rotation of the poses or by thresholding the distance between a surface point of an object in a candidate pose and the same surface point of the object in a different candidate pose.

11. The method of claim 9, further comprising a step where the poses selected in (d3) are recomputed as the average pose over the neighboring poses.

12. The method of claim 8, further comprising a step that refines each final pose computed in step (d) by optimizing an error function that is based on the distances between the 3D scene data and the object data under said pose.

13. The method of claim 8, further comprising a step of computing a score for each final pose computed in step (d), where the score describes the consistency between the 3D scene data and the 3D object data under said pose.