Tracking three-dimensional objects

• US 8,855,366 B2
• Filed: 04/18/2012
• Issued: 10/07/2014
• Est. Priority Date: 11/29/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wagner, Daniel, Gervautz, Michael
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US13/450,241
• Publication Number:
  US 20130136300A1
• Time in Patent Office: 902 Days
• Field of Search: 348/169, 375/E7.11, 382/103, 701/519
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

Method and apparatus for tracking three-dimensional (3D) objects are disclosed. In one embodiment, a method of tracking a 3D object includes constructing a database to store a set of two-dimensional (2D) images of the 3D object using a tracking background, where the tracking background includes at least one known pattern, receiving a tracking image, determining whether the tracking image matches at least one image in the database in accordance with feature points of the tracking image, and providing information about the tracking image in respond to the tracking image matches the at least one image in the database. The method of constructing a database also includes capturing the set of 2D images of the 3D object with the tracking background, extracting a set of feature points from each 2D image, and storing the set of feature points in the database.

Claims

1. A method of tracking a three-dimensional (3D) object, comprising:

receiving a tracking image of the 3D object at a mobile device;

extracting feature points of the tracking image of the 3D object;

comparing the feature points of the tracking image of the 3D object to a database, wherein the database includes feature points of a set of two-dimensional (2D) images of the 3D object with a tracking background, wherein the tracking background includes at least one known pattern;

determining whether the feature points of the tracking image match features points of one or more images in the set of 2D images in the database; and

providing information about the tracking image in a display of the mobile device in response to a determination of the feature points of the tracking image match the feature points of the one or more images in the set of 2D images in the database.

2. The method of claim 1, wherein the database is constructed by:

capturing the set of 2D images of the 3D object with the tracking background, wherein the set of 2D images includes a plurality of viewing angles of the 3D object;

extracting a set of feature points from each 2D image, wherein the set of feature points include a first subset of feature points of the 3D object and a second subset of feature points of the tracking background; and

storing the set of feature points in the database.

3. The method of claim 2, further comprises:

recording corresponding pose information of the set of 2D images with respect to a common coordinate system; and

storing the set of feature points in the database.

4. The method of claim 1, wherein comparing the feature points of the tracking image of the 3D object to the database comprises:

performing an accumulated vote on number of matched feature points between the tracking image and the set of 2D images in the database; and

identifying at least one representative image from the set of 2D images in accordance with the accumulated vote on number of matched feature points.

5. The method of claim 4, wherein comparing the feature points of the tracking image of the 3D object to the database further comprises:

estimating a representative pose of the tracking image from the at least one representative image that has a highest number of matched feature points.

6. The method of claim 4, wherein comparing the feature points of the tracking image of the 3D object to the database further comprises:

creating a set of merged feature points by merging features points from two or more of the representative images; and

estimating a representative pose of the tracking image in accordance with the set of merged feature points.

7. The method of claim 1, wherein providing information about the tracking image comprises:

providing pose information of the tracking image received.

8. The method of claim 7, wherein providing information about the tracking image further comprises:

providing information to support animation applications on a mobile device according to the pose information of the tracking image.

9. The method of claim 7, wherein providing information about the tracking image further comprises:

providing information to support augmented reality applications on a mobile device according to the pose information of the tracking image.

10. A computer program product for tracking a three-dimensional (3D) object, comprising a non-transitory medium storing computer programs for execution by one or more computer systems, the computer program product comprising:

code for receiving a tracking image of the 3D object at a mobile device;

code for extracting feature points of the tracking image of the 3D object;

code for comparing the feature points of the tracking image of the 3D object to a database, wherein the database includes feature points of a set of two-dimensional (2D) images of the 3D object with a tracking background, wherein the tracking background includes at least one known pattern;

code for determining whether the feature points of the tracking image match features points of one or more images in the set of 2D images in the database; and

code for providing information about the tracking image in a display of the mobile device in response to a determination of the feature points of the tracking image match the feature points of the one or more images in the set of 2D images in the database.

11. The computer program product of claim 10, wherein the database is constructed by:

code for capturing the set of 2D images of the 3D object with the tracking background, wherein the set of 2D images includes a plurality of viewing angles of the 3D object;

code for extracting a set of feature points from each 2D image, wherein the set of feature points include a first subset of feature points of the 3D object and a second subset of feature points of the tracking background; and

code for storing the set of feature points in the database.

12. The computer program product of claim 11, further comprises:

code for recording corresponding pose information of the set of 2D images with respect to a common coordinate system; and

code for storing the set of feature points in the database.

13. The computer program product of claim 10, wherein code for comparing the feature points of the tracking image of the 3D object to the database comprises:

code for performing an accumulated vote on number of matched feature points between the tracking image and the set of 2D images in the database; and

code for identifying at least one representative image from the set of 2D images in accordance with the accumulated vote on number of matched feature points.

14. The computer program product of claim 13, wherein code for comparing the feature points of the tracking image of the 3D object to the database further comprises:

code for estimating a representative pose of the tracking image from the at least one representative image that has a highest number of matched feature points.

15. The computer program product of claim 13, wherein code for comparing the feature points of the tracking image of the 3D object to the database further comprises:

code for creating a set of merged feature points by merging features points from two or more of the representative images; and

code for estimating a representative pose of the tracking image in accordance with the set of merged feature points.

16. The computer program product of claim 10, wherein code for providing information about the tracking image comprises:

code for providing pose information of the tracking image received.

17. The computer program product of claim 16, wherein code for providing information about the tracking image further comprises:

code for providing information to support animation applications on a mobile device according to the pose information of the tracking image.

18. The computer program product of claim 16, wherein code for providing information about the tracking image further comprises:

code for providing information to support augmented reality applications on a mobile device according to the pose information of the tracking image.

19. A mobile device, comprising:

at least one processor configured to control operations of the mobile device; and

a 3D object tracking module configured to work with the at least one processor, wherein the 3D object tracking module includes;

logic configured to receive a tracking image of the 3D object at the mobile device;

logic configured to extract feature points of the tracking image of the 3D object;

logic configured to compare the feature points of the tracking image of the 3D object to a database, wherein the database includes feature points of a set of two-dimensional (2D) images of the 3D object with a tracking background, wherein the tracking background includes at least one known pattern;

logic configured to determine whether the feature points of the tracking image match features points of one or more images in the set of 2D images in the database; and

logic configured to provide information about the tracking image in a display of the mobile device in response to a determination of the feature points of the tracking image match the feature points of the one or more images in the set of 2D images in the database.

20. The mobile device of claim 19, wherein the database is constructed by:

logic configured to capture the set of 2D images of the 3D object with the tracking background, wherein the set of 2D images includes a plurality of viewing angles of the 3D object;

logic configured to extract a set of feature points from each 2D image, wherein the set of feature points include a first subset of feature points of the 3D object and a second subset of feature points of the tracking background; and

logic configured to store the set of feature points in the database.

21. The mobile device of claim 20, further comprises:

logic configured to recording corresponding pose information of the set of 2D images with respect to a common coordinate system; and

logic configured to storing the set of feature points in the database.

22. The mobile device of claim 19, wherein logic configured to compare the feature points of the tracking image of the 3D object to the database comprises:

logic configured to perform an accumulated vote on number of matched feature points between the tracking image and the set of 2D images in the database; and

logic configured to identify at least one representative image from the set of 2D images in accordance with the accumulated vote on number of matched feature points.

23. The mobile device of claim 22, wherein logic configured to compare the feature points of the tracking image of the 3D object to the database further comprises:

logic configured to estimate a representative pose of the tracking image from the at least one representative image that has a highest number of matched feature points.

24. The mobile device of claim 22, wherein logic configured to compare the feature points of the tracking image of the 3D object to the database further comprises:

logic configured to create a set of merged feature points by merging features points from two or more of the representative images; and

logic configured to estimate a representative pose of the tracking image in accordance with the set of merged feature points.

25. The mobile device of claim 19, wherein logic configured to provide information about the tracking image comprises:

logic configured to provide pose information of the tracking image received.

26. The mobile device of claim 25, wherein logic configured to provide information about the tracking image further comprises:

logic configured to provide information to support animation applications on the mobile device according to the pose information of the tracking image.

27. The mobile device of claim 25, wherein logic configured to provide information about the tracking image further comprises:

logic configured to provide information to support augmented reality applications on the mobile device according to the pose information of the tracking image.

28. An apparatus, comprising:

at least one processor configured to control operations of the apparatus; and

a 3D object tracking module configured to work with the at least one processor, wherein the 3D object tracking module includes;

means for receiving a tracking image of the 3D object at the apparatus;

means for extracting feature points of the tracking image of the 3D object;

means for comparing the feature points of the tracking image of the 3D object to a database, wherein the database includes feature points of a set of two-dimensional (2D) images of the 3D object with a tracking background, wherein the tracking background includes at least one known pattern;

means for determining whether the feature points of the tracking image match features points of one or more images in the set of 2D images in the database; and

means for providing information about the tracking image in a display of the apparatus in response to a determination of the feature points of the tracking image match the feature points of the one or more images in the set of 2D images in the database.

29. The apparatus of claim 28, wherein the database is constructed by:

means for capturing the set of 2D images of the 3D object with the tracking background, wherein the set of 2D images includes a plurality of viewing angles of the 3D object;

means for extracting a set of feature points from each 2D image, wherein the set of feature points include a first subset of feature points of the 3D object and a second subset of feature points of the tracking background; and

means for storing the set of feature points in the database.

30. The apparatus of claim 29, further comprises:

means for recording corresponding pose information of the set of 2D images with respect to a common coordinate system; and

means for storing the set of feature points in the database.

31. The apparatus of claim 28, wherein means for comparing the feature points of the tracking image of the 3D object to the database comprises:

means for performing an accumulated vote on number of matched feature points between the tracking image and the set of 2D images in the database; and

means for identifying at least one representative image from the set of 2D images in accordance with the accumulated vote on number of matched feature points.

32. The apparatus of claim 31, wherein means for comparing the feature points of the tracking image of the 3D object to the database further comprises:

means for estimating a representative pose of the tracking image from the at least one representative image that has a highest number of matched feature points.