Simultaneous localization and mapping initialization

• US 10,242,453 B2
• Filed: 04/30/2014
• Issued: 03/26/2019
• Est. Priority Date: 04/30/2014
• Status: Active Grant

• Current Assignee: Micro Focus IP Development Ltd. (Open Text Corporation)
• Original Assignee: Longsand Limited (Open Text Corporation)
• Inventors: Cameron, Pashmina, Saklatvala, George
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US15/120,405
• Publication Number:
  US 20170069096A1
• Time in Patent Office: 1,791 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/579 from motion

Abstract

A pair of video frames may be used for simultaneous localization and mapping (SLAM) initialization. The pair of frames may be determined according to a translation threshold. Whether the translation threshold is met may be determined by obtaining the pair of video frames and estimating the translation between the frames.

Claims

1. A method, comprising:

obtaining, by a processor of a computing device, first and second video frames;

determining, by the processor, whether a total number of corresponding features between the first and second video frames exceeds a correspondence threshold;

in response to a determination that the total number of corresponding features exceeds the correspondence threshold, performing, by the processor, a two-dimensional (2D) transformation, including a 2D translation, from the features in the first video frame to the features in the second video frame;

determining, by the processor, whether a magnitude of the 2D translation exceeds a translation threshold; and

using the first and second of video frames for initialization of a simultaneous localization and mapping (SLAM) method in response a determination that the magnitude of the 2D translation exceeds the translation threshold.

2. The method of claim 1, further comprising:

obtaining the corresponding features from the first and second video frames, each pair of the corresponding features including a feature from the first video frame and a corresponding feature from the second video frame.

3. The method of claim 2, wherein obtaining the corresponding features comprises:

extracting features in the first video frame;

extracting features in the second video frame; and

identifying the corresponding features from the extracted features of the first and second video frames.

4. The method of claim 1, wherein performing the 2D transformation further includes performing a 2D rotation from the features in the first video frame to the features in the second video frame.

5. The method of claim 1, wherein performing the two-dimensional transformation comprises:

centering the features of the first video frame by subtracting a first centroid from the features of the first video frame;

centering the features of the second video frame by subtracting a second centroid from the features of the second video frame;

calculating a correlation matrix from the centered features of the first video frame and the centered features of the second video frame;

calculating a rotation from the centered features of the first video frame to the centered features of the second video frame to obtain a rotated first centroid; and

calculating the 2D translation by vector joining the rotated first centroid and the second centroid.

6. The method of claim 1, further comprising:

if the magnitude of the 2D translation does not exceed the translation threshold;

obtaining a third video frame;

obtaining corresponding features between the first video frame and the third video frame;

in response to a determination that a total number of corresponding features between the first and third video frames exceeds the correspondence threshold, performing a second 2D transformation, including a second 2D translation, from the features of the first video frame to the features of the third video frame; and

using the first video frame and the third video frame as initial frames in the SLAM method in response to a determination that a magnitude of the second 2D translation exceeds the translation threshold.

7. A system, comprising:

a processor; and

a memory storing instructions that when executed cause the processor to;

detect a first set of rays from a first video frame and a second set of rays from a second video frame, each ray from the first set and a corresponding ray from the second set representing a common feature in the first video frame and the second video frame;

determine whether a total number of common features in the first and second video frames exceeds a correspondence threshold;

in response to a determination that the total number of common features exceeds the correspondence threshold, estimate a two-dimensional (2D) transformation, including a 2D translation, from the first set of rays to the second set of rays;

determine whether a magnitude of the 2D translation exceeds a translation threshold; and

determine that the first video frame and the second video frame are suitable as initial frames for simultaneous localization and mapping (SLAM) based on a determination that the magnitude of the 2D translation exceeds the translation threshold.

8. The system of claim 7, wherein the instructions are to cause the processor to:

perform a SLAM initialization using the first video frame and second video frame as initial frames.

9. The system of claim 7, wherein the 2D transformation further comprises a 2D rotation.

15. The system of claim 7, wherein, to estimate the 2D transformation, the instructions are to cause the processor to:

center the first set of rays in the first video frame by subtracting a first centroid from the first set of rays;

center the second set of rays in the second video frame by subtracting a second centroid from the second set of rays;

calculate a correlation matrix from the centered first set of rays and the centered second set of rays;

calculate a rotation from the centered first set of rays to the centered second set of rays to obtain a rotated first centroid; and

calculate the 2D translation by vector joining the rotated first centroid and the second centroid.

16. The system of claim 7, wherein, in response to a determination that the magnitude of the 2D translation does not exceed the translation threshold and the total number of the common features does not exceed the correspondence threshold, the instructions are to cause the processor to:

compare subsequent video frames with the first video frame until the magnitude of the 2D translation between one of the subsequent video frames and the first video frame exceeds the translation threshold and the total number of the common features exceeds the correspondence threshold, and

use the subsequent video frame and first video frame for initialization of the SLAM method.

10. A non-transitory computer readable medium storing instructions executable by a processor to:

obtain a first video frame and a second video frame;

extract a first feature set from the first video frame;

extract a second feature set from the second video frame, elements of the first and second feature sets forming corresponding pairs of features;

determine whether a total number of common features between the first and second video frames exceeds a correspondence threshold;

in response to a determination that the total number of common features exceeds the correspondence threshold, perform a two-dimensional (2D) transformation, including a 2D translation, between the first feature set and the second feature set;

in response to a determination that a magnitude of the 2D translation exceeds a translation threshold, use the first video and the second video frame for simultaneous localization and mapping (SLAM) initialization.

11. The non-transitory computer readable medium of claim 10, wherein the instructions are to cause the processor to:

if the magnitude of the 2D translation does not exceed the translation threshold, compare subsequent video frames with the first video frame until the 2D translation between one of the subsequent video frames and the first video frame exceeds the translation threshold and the total number of common features exceeds the correspondence threshold, and use the subsequent video frame and first video frame for SLAM initialization.

12. The non-transitory computer readable medium of claim 10, wherein the instructions are to cause the processor to extract the first feature set and the second feature set by:

extracting features of the first video frame;

extracting features of the second video frame; and

identifying the common features from the extracted features of the first and second video frames.

13. The non-transitory computer readable medium of claim 10, wherein to perform the 2D transformation, the instructions are further to cause the processor to perform a 2D rotation from the first feature set to the second feature set.

14. The non-transitory computer readable medium of claim 10, wherein the instructions are to cause the processor to perform the two-dimensional transformation by:

centering the first feature set by subtracting a first centroid from the first feature set;

centering the second feature set by subtracting a second centroid from the second feature set;

calculating a correlation matrix from the centered first feature set and the centered second feature set;

calculating a rotation from the centered first feature set to the centered second feature set to obtain a rotated first centroid; and

calculating the 2D translation by vector joining the rotated first centroid and the second centroid.