METHOD OF DETERMINING REFERENCE FEATURES FOR USE IN AN OPTICAL OBJECT INITIALIZATION TRACKING PROCESS AND OBJECT INITIALIZATION TRACKING METHOD

• US 20110194731A1
• Filed: 12/23/2010
• Published: 08/11/2011
• Est. Priority Date: 12/23/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Metaio GmbH (Apple Inc.)
• Inventors: Ulbricht, Daniel, BenHimane, Selim
• Granted Patent: US 8,805,002 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

A method of determining reference features for use in an optical object initialization tracking process is disclosed, said method comprising the following steps: a) capturing at least one current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one current image, b) providing reference features adapted for use in an optical object initialization tracking process, c) matching a plurality of the current features with a plurality of the reference features, d) estimating at least one parameter associated with the current image based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched, e) wherein the steps a) to d) are processed iteratively multiple times, wherein in step a) of every respective iterative loop a respective new current image is captured by at least one camera and steps a) to d) are processed with respect to the respective new current image, and f) determining at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched, wherein the at least one indicator is determined depending on how often the respective reference feature has been correctly matched or incorrectly matched, respectively.

Claims

1. A method of determining reference features for use in an optical object initialization tracking process, said method comprising the following steps:

a) capturing at least one current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one current image;

b) providing reference features adapted for use in an optical object initialization tracking process;

c) matching a plurality of the current features with a plurality of the reference features;

d) estimating at least one parameter associated with the current image based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched;

e) wherein the steps a) to d) are processed iteratively multiple times, wherein in step a) of every respective iterative loop a respective new current image is captured by at least one camera and steps a) to d) are processed with respect to the respective new current image; and

f) determining at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched, wherein the at least one indicator is determined depending on how often the respective reference feature has been correctly matched or incorrectly matched, respectively.

2. The method of claim 1, wherein the indicator is a weight associated to the reference features.

3. The method of claim 2, wherein the weight of the reference features is increased when the respective reference feature was correctly matched.

4. The method of claim 2, wherein the weight of the reference features is decreased when the respective reference feature was incorrectly matched.

5. The method of claim 2, wherein if one of the reference features is not matched, its weight is not updated.

6. The method of claim 1, wherein in step d) the method includes the step of comparing at least one property of the current image with at least one property of a reference image, which provides the reference features, under consideration of the estimated at least one parameter, and verifying the estimated at least one parameter based on the comparison, and wherein the at least one indicator is updated if the estimated at least one parameter is verified.

7. The method of claim 6, wherein when comparing at least one property of the current image with at least one property of the reference image under consideration of the estimated at least one parameter, a verification score value is determined which is indicative of the comparison result, and the estimated at least one parameter is verified if the verification score value is equal or beyond a verification threshold value.

8. The method of claim 7, further comprising the step of determining at least one first indicator and second indicator associated to each of the reference features which were correctly matched and/or to each of the reference features which were incorrectly matched, the first and second indicators being indicative of different respective conditions when capturing multiple current images.

9. An object initialization tracking method making use of reference features, said method comprising the following steps:

a) capturing at least one second current image with at least one camera and extracting current features from the at least one second current image;

b) providing reference features adapted for use in an optical object initialization tracking process, and providing at least one indicator associated to each of the reference features, the indicator determined according to the method of one of the preceding claims; and

c) using the indicator for a selection or prioritization of reference features which are to be matched with current features of the at least one second current image, and using the selected or prioritized reference features in an object initialization tracking initialization process.

10. The method of claim 9, further comprising the step of updating the at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched with current features of the at least one second current image.

11. An object initialization tracking method making use of reference features, said method comprising the steps of:

a) capturing at least one first current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one first current image;

b) providing reference features adapted for use in an optical object initialization tracking process;

c) matching a plurality of the current features with a plurality of the reference features;

d) determining at least one parameter associated with the first current image in an object initialization tracking process based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched;

e) updating at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched; and

f) using the indicator for a selection or prioritization of reference features which are to be matched with current features of at least one second current image captured by at least one camera, and using the selected or prioritized reference features in an object initialization tracking process with respect to the second current image.

12. The method of claim 11, wherein the indicator is a weight associated to each of the reference features.

13. The method of claim 11, wherein the indicator is a weight associated to each of the reference features, and reference features with a weight above a first threshold value are used in the matching process and features with a weight below a second threshold value are not used in the matching process

14. The method of claim 11, wherein when estimating or determining the parameter, a higher priority is applied to reference features that have a higher weight and a lower priority is applied to reference features that have a lower weight, wherein the priority is an input parameter when estimating or determining the parameter.

15. The method of claim 11, further comprising

providing at least one first indicator and second indicator associated to each of the reference features which were correctly matched and/or to the reference features which were incorrectly matched, the first and second indicator being associated to different conditions, and

using at least one of the first and second indicators, or a combination of the first and second indicators, or switching between the first and second indicators for the selection or prioritization of reference features according to the current estimated condition.

16. The method of claim 15, wherein the at least one indicator is additionally determined according to a matching strength of the matched features.

17. The method of claim 11, wherein the at least one parameter is indicative of at least one of the following:

a pose of the camera, a condition of the real environment, a property of the real object to be tracked.

18. A computer program product adapted to be loaded into the internal memory of a digital computer system coupled with at least one camera, the computer program product being used to determine reference features for use in an optical object initialization tracking process, the computer program product comprising software code sections by means of which the following steps are performed:

a) capturing at least one current image of a real environment or synthetically generated by rendering a virtual model of a real object to be tracked with at least one camera and extracting current features from the at least one current image;

b) providing reference features adapted for use in an optical object initialization tracking process;

c) matching a plurality of the current features with a plurality of the reference features;

d) estimating at least one parameter associated with the current image based on a number of current and reference features which were matched, and determining for each of the reference features which were matched with one of the current features whether they were correctly or incorrectly matched;

e) wherein the steps a) to d) are processed iteratively multiple times, wherein in step a) of every respective iterative loop a respective new current image is captured by at least one camera and steps a) to d) are processed with respect to the respective new current image; and

f) determining at least one indicator associated to reference features which were correctly matched and/or to reference features which were incorrectly matched, wherein the at least one indicator is determined depending on how often the respective reference feature has been correctly matched or incorrectly matched, respectively.