Systems and Methods for Feature-Based Tracking

• US 20140369557A1
• Filed: 04/28/2014
• Published: 12/18/2014
• Est. Priority Date: 06/14/2013
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Najafi Shoushtari, Seyed Hesameddin, Ahuja, Dheeraj, Kayombya, Guy-Richard, Tsin, Yanghai
• Application Number:
  US14/263,866
• Publication Number:
  US 20140369557A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Disclosed embodiments pertain to feature based tracking. In some embodiments, a camera pose may be obtained relative to a tracked object in a first image and a predicted camera pose relative to the tracked object may be determined for a second image subsequent to the first image based, in part, on a motion model of the tracked object. An updated SE(3) camera pose may then be obtained based, in part on the predicted camera pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of their squared intensity differences. A feature tracker may be initialized with the updated SE(3) camera pose.

Claims

1. A method comprising:

obtaining a camera pose relative to a tracked object in a first image;

determining a predicted camera pose relative to the tracked object for a second image subsequent to the first image based, in part, on a motion model of the tracked object; and

obtaining an updated Special Euclidean Group (3) (SE(3)) camera pose, based, in part on the predicted camera pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image.

2. The method of claim 1, further comprising:

initializing a feature tracker with the updated SE(3) camera pose, wherein the feature tracker determines a feature tracker camera pose based, in part, on the updated SE(3) pose.

3. The method of claim 1, wherein the equation of the dominant plane in the first image is obtained based on a 3-dimensional (3D) model of the tracked object.

4. The method of claim 1, wherein the minimization of the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image is performed using an Inverse Compositional Image Alignment technique.

5. The method of claim 1, wherein determining the predicted camera pose based, in part, on the motion model comprises:

refining a motion model determined camera pose relative to the tracked object in the second image by estimating fronto-parallel translation motion using Normalized Cross Correlation (NCC) between a second lower resolution version of the first image and a second lower resolution version of the second image, wherein the estimated fronto-parallel translation motion is used to determined the predicted camera pose.

6. The method of claim 5, wherein the second lower resolution version of the first image and the second lower resolution version of the second image are blurred prior to NCC.

7. The method of claim 6, wherein the first and second images are associated with respective first and second image pyramids and the first lower resolution version of the first image and the first lower resolution version of the second image form part of the first and second image pyramids, respectively.

8. The method of claim 2, further comprising:

determining an initial camera pose for a third image subsequent and consecutive to the second image based, in part, on the feature tracker camera pose.

9. The method of claim 2, further comprising:

rendering an Augmented Reality (AR) image based, in part, on the feature tracker camera pose.

10. A User Device (UD) comprising:

a camera, the camera to capture a first image and a second image subsequent to the first image, and

a processor coupled to the camera, the processor configured to;

obtain a camera pose relative to a tracked object in the first image,

determine a predicted camera pose relative to the tracked object for the second image based, in part, on a motion model of the tracked object, and

obtain an updated Special Euclidean Group (3) (SE(3)) camera pose, based, in part on the predicted camera pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image.

11. The UD of claim 10, wherein the processor is further configured to:

initialize a feature tracker with the updated SE(3) camera pose, wherein the feature tracker determines a feature tracker camera pose based, in part, on the updated SE(3) camera pose.

12. The UD of claim 10, wherein the processor obtains the equation of the dominant plane in the first image based on a 3-dimensional (3D) model of the tracked object.

13. The UD of claim 10, wherein the minimization of the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image is performed using an Inverse Compositional Image Alignment technique.

14. The UD of claim 13, wherein to determine the predicted camera pose based, in part, on the motion model, the processor is further configured to:

refine a motion model determined camera pose relative to the tracked object in the second image by a estimating fronto-parallel translation motion using Normalized Cross Correlation (NCC) between a second lower resolution version of the first image and a second lower resolution version of the second image, and wherein the estimated fronto-parallel translation motion is used to determine the predicted camera pose.

15. The UD of claim 14, wherein the processor is further configured to blur the second lower resolution version of the first image and the second lower resolution version of the second image prior to NCC.

16. The UD of claim 14, wherein the first and second images are associated with respective first and second image pyramids and the first and second lower resolution versions of the first and second images form part of the first and second image pyramids, respectively.

17. The UD of claim 11, wherein the processor is further configured to:

determine an initial camera pose for a third image subsequent and consecutive to the second image based, in part, on the feature tracker camera pose.

18. The UD of claim 11, further comprising:

a display coupled to the processor, wherein the processor is further configured to;

render an Augmented Reality (AR) image on the display using the feature tracker camera pose.

19. An apparatus comprising:

imaging means, the imaging means to capture a first image and a second image subsequent to the first image,

means for obtaining a imaging means pose relative to a tracked object in the first image;

means for determining a predicted imaging means pose relative to the tracked object for the second image based, in part, on a motion model of the tracked object; and

means for obtaining an updated Special Euclidean Group (3) (SE(3)) imaging means pose, based, in part on the predicted imaging means pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image.

20. The apparatus of claim 19, further comprising:

means for initializing a feature tracker with the updated SE(3) imaging means pose, wherein the feature tracker comprises;

means for determining a feature tracker imaging means pose based, in part, on the updated SE(3) imaging means pose.

21. The apparatus of claim 19, wherein the minimization of the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image is performed using an Inverse Compositional Image Alignment technique.

22. The apparatus of claim 19, wherein means for determining the predicted imaging means pose based, in part, on the motion model, further comprises:

means for refining a motion model determined imaging means pose relative to the tracked object in the second image by a estimating fronto-parallel translation motion using Normalized Cross Correlation (NCC) between a second lower resolution version of the first image and a second lower resolution version of the second image, and wherein the estimated fronto-parallel translation motion is used by means for determining the predicted imaging means pose.

23. The apparatus of claim 20, further comprising:

means for rendering an Augmented Reality (AR) image on the display using the feature tracker imaging means pose.

24. A non-transitory computer-readable medium comprising instructions, which, when executed by a processor, perform steps in a method, the steps comprising:

obtaining a camera pose relative to a tracked object in a first image;

determining a predicted camera pose relative to the tracked object for a second image subsequent to the first image based, in part, on a motion model of the tracked object; and

obtaining an updated Special Euclidean Group (3) (SE(3)) camera pose, based, in part on the predicted camera pose, by estimating a plane induced homography using an equation of a dominant plane of the tracked object, wherein the plane induced homography is used to align a first lower resolution version of the first image and a first lower resolution version of the second image by minimizing the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image.

25. The computer-readable medium of claim 24, the steps further comprising:

initializing a feature tracker with the updated SE(3) camera pose, wherein the feature tracker determines a feature tracker camera pose based, in part, on the updated SE(3) pose.

26. The computer-readable medium of claim 24, wherein the equation of the dominant plane in the first image is obtained based on a 3-dimensional (3D) model of the tracked object.

27. The computer-readable medium of claim 24, wherein the minimization of the sum of the squared intensity differences of the first lower resolution version of the first image and the first lower resolution version of the second image is performed using an Inverse Compositional Image Alignment technique.

28. The computer-readable medium of claim 24, wherein the predicted camera pose based, in part, on the motion model is obtained by:

refining a motion model determined camera pose relative to the tracked object in the second image by estimating fronto-parallel translation motion using Normalized Cross Correlation (NCC) between a second lower resolution version of the first image and a second lower resolution version of the second image, wherein the estimated fronto-parallel translation motion is used to determined the predicted camera pose.

29. The computer-readable medium of claim 25, the steps further comprising:

determining an initial camera pose for a third image subsequent and consecutive to the second image based, in part, on the feature tracker camera pose.

30. The computer-readable medium of claim 25, the steps further comprising:

rendering an Augmented Reality (AR) image based, in part, on the feature tracker camera pose.