Method for determining the pose of a camera and for recognizing an object of a real environment

• US 10,229,511 B2
• Filed: 12/21/2015
• Issued: 03/12/2019
• Est. Priority Date: 10/19/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Meier, Peter
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US14/977,288
• Publication Number:
  US 20160189384A1
• Time in Patent Office: 1,177 Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method for determining the pose of a camera relative to a real environment includes the following steps: taking at least one image of a real environment by means of a camera, the image containing at least part of a real object, performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera, determining at least one parameter of an environmental situation, and performing the tracking method in accordance with the at least one parameter. Analogously, the method can also be utilized in a method for recognizing an object of a real environment in an image taken by a camera.

Claims

1. A non-transitory computer readable medium comprising computer readable code which, when executed by one or more processors, causes the one or more processors to:

obtain an image captured by a camera, the image comprising a real environment containing at least part of a real object having features;

determine one or more parameter values corresponding to one or more environmental parameters of the real environment;

capture sensor data from a plurality of sensors, wherein the sensor data are weighted based on the one or more parameter values; and

evaluate one or more correspondences between the imaged features of the real object and modeled features of the real object, wherein the evaluation is influenced by the one or more parameter values and the sensor data.

2. The non-transitory computer readable medium of claim 1, further comprising computer code to cause the one or more processors to determine a pose of the camera based on the evaluation.

3. The non-transitory computer-readable medium of claim 2, further comprising computer code which cause the one or more processors to calculate an uncertainty of the determined pose based on at least one of the one or more parameter values and an ideal value corresponding to each of the at least one of the one or more environmental parameters.

4. The non-transitory computer readable medium of claim 1,

wherein the plurality of sensors are prioritized during the evaluation based on the one or more parameter values.

5. The non-transitory computer readable medium of claim 1, wherein the one or more parameter values are determined from sensor data from one or more sensors local to the camera.

6. The non-transitory computer readable medium of claim 1, wherein the one or more parameter values are obtained from a remote source based on a context of the camera.

7. A system comprising:

a camera;

one or more processors; and

a memory, coupled to the one or more processors and comprising computer code which, when executed, causes the one or more processors to;

obtain an image captured by the camera, the image comprising a real environment containing at least part of a real object having features;

determine one or more parameter values corresponding to one or more environmental parameters of the real environment; and

capture sensor data from a plurality of sensors, wherein the sensor data are weighted based on the one or more parameter values; and

evaluate one or more correspondences between the imaged features of the real object and modeled features of the real object, wherein the evaluation is influenced by the one or more parameter values and the sensor data.

8. The system of claim 7, the memory further comprising computer code to cause the one or more processors to determine a pose of the camera based on the evaluation.

9. The system of claim 8, the memory further comprising computer code which cause the one or more processors to calculate an uncertainty of the determined pose based on at least one of the one or more parameter values and an ideal value corresponding to each of the at least one of the one or more environmental parameters.

10. The system of claim 7,

wherein the plurality of sensors are prioritized during the evaluation based on the one or more parameter values.

11. The system of claim 7, wherein the one or more parameter values are determined from sensor data from one or more sensors local to the camera.

12. The system of claim 7, wherein the one or more parameter values are obtained from a remote source based on a context of the camera.

13. A method comprising:

obtaining an image captured by a camera, the image comprising a real environment containing at least part of a real object having features;

determining one or more parameter values corresponding to one or more environmental parameters of the real environment; and

capturing sensor data from a plurality of sensors, wherein the sensor data are weighted based on the one or more parameter values; and

evaluating one or more correspondences between the imaged features of the real object and modeled features of the real object, wherein the evaluation is influenced by the one or more parameter values and the sensor data.

14. The method of claim 13, further comprising determining a pose of the camera based on the evaluation.

15. The method of claim 14, further comprising calculating an uncertainty of the determined pose based on at least one of the one or more parameter values and an ideal value corresponding to each of the at least one of the one or more environmental parameters.

16. The method of claim 13,

wherein the plurality of sensors are prioritized during the evaluation based on the one or more parameter values.

17. The method of claim 13, wherein the one or more parameter values are determined from sensor data from one or more sensors local to the camera.

18. The method of claim 13, wherein the one or more parameter values are obtained from a remote source based on a context of the camera.