Tracking Poses of 3D Camera Using Points and Planes

• US 20140002597A1
• Filed: 06/19/2013
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Ramalingam, Srikumar, Taguchi, Yuichi, Ataer-Cansizoglu, Esra, Garaas, Tyler W.
• Granted Patent: US 9,420,265 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A method registers data using a set of primitives including points and planes. First, the method selects a first set of primitives from the data in a first coordinate system, wherein the first set of primitives includes at least three primitives and at least one plane. A transformation is predicted from the first coordinate system to a second coordinate system. The first set of primitives is transformed to the second coordinate system using the transformation. A second set of primitives is determined according to the first set of primitives transformed to the second coordinate system. Then, the second coordinate system is registered with the first coordinate system using the first set of primitives in the first coordinate system and the second set of primitives in the second coordinate system. The registration can he used to track a pose of a camera acquiring the data.

Claims

1. A method for registering data using a set of primitives, wherein the data have three dimensions (3D) and the primitives include points and planes, comprising the steps of:

selecting a first set of primitives from the data in a first coordinate system, wherein the first set of primitives includes at least three primitives and at least one plane;

predicting a transformation from the first coordinate system to a second coordinate system;

transforming the first set of primitives to the second coordinate system using the transformation;

determining a second set of primitives according to the first set of primitives transformed to the second coordinate system; and

registering the second coordinate system with the first coordinate system using the first set of primitives in the first coordinate system and the second set of primitives in the second coordinate system corresponding to each other, wherein the steps are performed in a processor.

2. The method of claim 1, wherein the first set of primitives includes at least one point and at least one plane in the first coordinate system and the second set of primitives includes at least one point and at least one plane in the second coordinate system.

3. The method of claim 1, wherein the data are acquired by a movable camera.

4. The method of claim 1, wherein the data include textures and depths.

5. The method of claim 1, wherein the registering uses RANdom SAmple Consensus (RANSAC).

6. The method of claim 1, wherein the data are in a form of a sequence of frames acquired by a camera.

7. The method of claim 6, further comprising:

selecting a set of frames as key frames from the sequence of frames; and

storing the keyframes in a map, wherein the keyframes include the points and the planes, and the points and the planes are stored in the map as landmarks.

8. The method of claim 7, further comprising:

predicting a pose of the camera for each frame; and

determing the pose of the camera for each frame according to the registering to track the camera.

9. The method of claim 1, wherein the registering is in real time.

10. The method of claim 7, further comprising:

applying bundle adjustment using the points and the planes to refine the landmarks in the map.

11. The method of claim 8, therein the pose of a k th frame is

12. The method of claim 8, wherein the predicting uses a constant velocity assumption.

13. The method of claim 6, wherein an optical flow procedure is used to locate the points in the frames.

14. The method of claim 1, wherein correspondences of the planes are prioritizes over correspondences of the points.

15. The method of claim 1, wherein the registering is used for simultaneous localization and mapping (SLAM).