FAST INITIALIZATION FOR MONOCULAR VISUAL SLAM

• US 20140126769A1
• Filed: 03/14/2013
• Published: 05/08/2014
• Est. Priority Date: 11/02/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Reitmayr, Gerhard, Mulloni, Alessandro
• Granted Patent: US 9,576,183 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Apparatuses and methods for fast visual simultaneous localization and mapping are described. In one embodiment, a three-dimensional (3D) target is initialized immediately from a first reference image and prior to processing a subsequent image. In one embodiment, one or more subsequent reference images are processed, and the 3D target is tracked in six degrees of freedom. In one embodiment, the 3D target is refined based on the processed the one or more subsequent images.

Claims

1. A processor-implemented method for visual simultaneous localization and mapping, the method comprising:

initializing a three-dimensional (3D) target from a first reference image and prior to processing a subsequent image;

processing one or more subsequent images;

tracking the 3D target in six degrees of freedom; and

refining the 3D target based on the processing of the one or more subsequent images.

2. The processor-implemented method of claim 1, further comprising:

displaying a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target.

3. The processor-implemented method of claim 2, further comprising:

updating the augmented reality representation of the 3D target while tracking the 3D target in six degrees of freedom.

4. The processor-implemented method of claim 1, wherein the processing one or more subsequent images further comprises:

extracting 2D interest points from the one or more subsequent images.

5. The processor-implemented method of claim 1, wherein initializing the 3D target further comprises:

extracting a reference set of 2D interest points from the reference image;

determining target 3D points, each target 3D point corresponding to a 2D interest point in the reference set of 2D interest points; and

assigning a predetermined initial depth value to each target 3D point.

6. The processor-implemented method of claim 5, wherein each of the target 3D points is initialized along a plane with a same predetermined initial depth value.

7. The processor-implemented method of claim 5, wherein the tracking the 3D target further comprises comparing the reference location of at least one of the target 3D points to an updated 2D location extracted from the one or more subsequent images.

8. The processor-implemented method of claim 5, wherein refining the 3D target further comprises:

determining an updated depth value for one of more of the target 3D points; and

replacing the assigned predetermined initial depth value with the updated depth value.

9. The processor-implemented method of claim 8, further comprising:

determining a threshold number of the target 3D points having an updated depth value is met, wherein the threshold number is met based on updated depth values from a respective subsequent image of the one or more subsequent images; and

assigning the respective subsequent image as a second reference image.

10. The processor-implemented method of claim 9, wherein the refining of the 3D target further comprises performing a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.

12. The medium of claim 1, further comprising:

displaying a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target.

13. The medium of claim 12, further comprising:

updating the augmented reality representation of the 3D target while tracking the 3D target in six degrees of freedom.

11. A computer readable non-transitory storage medium containing executable program instructions which cause a data processing device to perform a method for visual simultaneous localization and mapping, the method comprising:

initializing a three-dimensional (3D) target from a first reference image and prior to processing a subsequent image;

processing one or more subsequent images;

tracking the 3D target in six degrees of freedom; and

refining the 3D target based on the processing of the one or more subsequent images.

14. The medium of claim 11, wherein the processing one or more subsequent images further comprises:

extracting 2D interest points from the one or more subsequent images.

15. The medium of claim 11, wherein initializing the 3D target further comprises:

extracting a reference set of 2D interest points from the reference image;

determining a target 3D points, each target 3D point corresponding to a 2D interest point in the reference set of 2D interest points; and

assigning a predetermined initial depth value to each target 3D point.

16. The medium of claim 15, wherein each of the target 3D points is initialized along a plane with a same predetermined initial depth value.

17. The medium of claim 15, wherein the tracking the 3D target further comprises comparing the reference location of at least one of the target 3D points to an updated 2D location extracted from the one or more subsequent images.

18. The medium of claim 15, wherein refining the 3D target further comprises:

determining an updated depth value for one of more of the target 3D points; and

replacing the assigned predetermined initial depth value with the updated depth value.

19. The medium of claim 18, further comprising:

determining a threshold number of the target 3D points having an updated depth value is met, wherein the threshold number is met based on updated depth values from a respective subsequent image of the one or more subsequent images; and

assigning the respective subsequent image as a second reference image.

20. The medium of claim 19, wherein the refining of the 3D target further comprises performing a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.

21. A data processing device for visual simultaneous localization and mapping comprising:

a processor; and

a storage device coupled to the processor and configurable for storing instructions, which, when executed by the processor cause the processor to;

initialize a three-dimensional (3D) target from a first reference image and prior to processing a subsequent image;

process one or more subsequent images;

track the 3D target in six degrees of freedom; and

refine the 3D target based on the processing of the one or more subsequent images.

22. The data processing device of claim 21, further comprising instructions to:

display a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target.

23. The data processing device of claim 22, further comprising instructions to:

update the augmented reality representation of the 3D target while tracking the 3D target in six degrees of freedom.

24. The data processing device of claim 21, wherein the processing one or more subsequent images further comprises instructions to:

extract 2D interest points from the one or more subsequent images.

25. The data processing device of claim 21, wherein initializing the 3D target further comprises instructions to:

extract a reference set of 2D interest points from the reference image;

determine target 3D points, each target 3D point corresponding to a 2D interest point in the reference set of 2D interest points; and

assign a predetermined initial depth value to each target 3D point.

26. The data processing device of claim 25, wherein each of the target 3D points is initialized along a plane with a same predetermined initial depth value.

27. The data processing device of claim 25, wherein the tracking the 3D target further comprises comparing the reference location of at least one of the target 3D points to an updated 2D location extracted from the one or more subsequent images.

28. The data processing device of claim 25, wherein refining the 3D target further comprises instructions to:

determine an updated depth value for one of more of the target 3D points; and

replace the assigned predetermined initial depth value with the updated depth value.

29. The data processing device of claim 28, further comprising instructions to:

determine a threshold number of the target 3D points having an updated depth value is met, wherein the threshold number is met based on updated depth values from a respective subsequent image of the one or more subsequent images; and

assign the respective subsequent image as a second reference image.

30. The data processing device of claim 29, wherein the refining of the 3D target further comprises instructions to perform a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.

31. An apparatus for visual simultaneous localization and mapping comprising:

means for initializing a three-dimensional (3D) target from a first reference image and prior to processing a subsequent image;

means for processing one or more subsequent images;

means for tracking the 3D target in six degrees of freedom; and

means for refining the 3D target based on the processing of the one or more subsequent images.

32. The apparatus of claim 31, further comprising:

means for displaying a correctly aligned, positioned, and distanced augmented reality representation of the 3D target upon initializing the 3D target.

33. The apparatus of claim 32, further comprising:

means for updating the augmented reality representation of the target while tracking the 3D target in six degrees of freedom.

34. The apparatus of claim 31, wherein the processing one or more subsequent images further comprises:

means for extracting 2D interest points from the one or more subsequent images.

35. The apparatus of claim 31, wherein initializing the 3D target further comprises:

means for extracting a reference set of 2D interest points from the reference image;

means for determining target 3D points, each target 3D point corresponding to a 2D interest point in the reference set of 2D interest points; and

means for assigning a predetermined initial depth value to each target 3D point corresponding to a 2D interest point in the reference set of 2D interest points.

36. The apparatus of claim 35, wherein each of the target 3D points is initialized along a plane with a same predetermined initial depth value.

37. The apparatus of claim 35, wherein the tracking the 3D target further comprises means for comparing the reference location of at least one of the target 3D points to an updated 2D location extracted from the one or more subsequent images.

38. The apparatus of claim 35, wherein refining the 3D target further comprises:

means for determining an updated depth value for one of more of the target 3D points; and

means for replacing the assigned predetermined initial depth value with the updated depth value.

39. The apparatus of claim 38, further comprising:

means for determining a threshold number of the target 3D points having an updated depth value is met, wherein the threshold number is met based on updated depth values from a respective subsequent image of the one or more subsequent images; and

means for assigning the respective subsequent image as a second reference image.

40. The apparatus of claim 39, wherein the refining of the 3D target further comprises means for performing a further refinement of the 3D target by triangulating the target 3D points with a plurality of 2D interest points extracted from the second reference image.