Method for Registering Points and Planes of 3D Data in Multiple Coordinate Systems

• US 20140003705A1
• Filed: 06/29/2012
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Taguchi, Yuichi, Ramalingam, Srikumar, Jian, Yong-Dian, Feng, Chen
• Granted Patent: US 9,183,631 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/75 involving models

Abstract

Three-dimensional data are registered by selecting a first set of primitives from the data in a first coordinate system, wherein the first set of primitives includes at least one plane, at least one point, and a third primitive that is either a point or a plane, and selecting a second set of primitives from the data in a second coordinate system, wherein the second set of primitives includes at least one plane, at least one point, and a third primitive corresponding to the third primitive in the first set of primitives. Then, the planes are registered with each other, as are the points, to obtain registered primitives.

Claims

1. A method for registering data, wherein the data have three dimensions, comprising the steps of:

selecting a first set of primitives from the data in a first coordinate system, wherein the first set of primitives includes at least one plane, at least one point, and a third primitive that is either a point or a plane;

selecting a second set of primitives from the data in a second coordinate system, wherein the second set of primitives includes at least one plane, at least one point, and a third primitive corresponding to the third primitive in the first set of primitives; and

registering, in the first and second set of primitives, planes with each other and points with each other to obtain registered primitives.

2. The method of claim 1, wherein the registering is used in a simultaneous localization and mapping system.

3. The method of claim 1, wherein the registering is global.

4. The method of claim 1, wherein the registering is in real time.

5. The method of claim 2, further comprising:

acquiring the data as a depth map.

6. The method of claim 5, wherein the depth map is acquired by a 3D sensor.

7. The method of claim 5, further comprising:

maintaining the registered primitives in a global map.

8. The method of claim 5, wherein the registering uses a random sample consensus procedure.

9. The method of claim 5, further comprising:

representing each point by a position and a descriptor, and each plane by a unit normal vector and a distance to an origin of the corresponding coordinate system.

10. The method of claim 5, wherein each plane is selected by:

randomly selecting a set of reference points in a 3D point cloud;

determining an optimal plane for each reference point using nearby points inside a local window;

determining all inliers that form a connected component with the reference point with respect to a grid graph of the depth map;

identifying an optimal plane with a predetermined number of the inliers; and

removing the inliers from the 3D point cloud.

11. The method of claim 7, wherein the global map is optimized by bundle adjustment using the points and the planes.

12. The method of claim 1, further comprising:

identifying false matches during the registering based on;

a distance between points;

a distance between a point and a plane;

or an angle between normals of two planes in each coordinate system.

13. The method of claim 8, further comprising:

initializing the random sample consensus procedure with an available triplet of primitives in a following order;

3 planes, 2 planes and 1 point, 1 plane and 2 points, or 3 points.

14. The method of claim 5, wherein each point is selected by a 3D keypoint detector.

15. The method of claim 11, wherein the bundle adjustment enforces geometric constraints between the planes.

16. The method of claim 1, wherein an output of the method includes a registered point cloud and a plane-based representation of a scene.

17. The method of claim 6, wherein the 3D sensor is moveable during the acquiring.

18. The method of claim 5, further comprising:

acquiring a texture image corresponding to the depth map.

19. The method of claim 18, wherein each point is selected by a 2D keypoint detector applied to the texture image and back-projecting detected keypoints using the depth map.

20. The method of claim 19, wherein the texture image is acquired by a camera.