Method for determining the pose of a camera and for recognizing an object of a real environment

• US 9,218,665 B2
• Filed: 07/11/2014
• Issued: 12/22/2015
• Est. Priority Date: 10/19/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Metaio GmbH (Apple Inc.)
• Inventors: Meier, Peter
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US14/329,547
• Publication Number:
  US 20140321708A1
• Time in Patent Office: 529 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method for determining the pose of a camera relative to a real environment includes the following steps: taking at least one image of a real environment by means of a camera, the image containing at least part of a real object, performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera, determining at least one parameter of an environmental situation, and performing the tracking method in accordance with the at least one parameter. Analogously, the method can also be utilized in a method for recognizing an object of a real environment in an image taken by a camera.

Claims

1. A method for recognizing an object of a real environment in an image taken by a camera, said method comprising the following steps:

taking at least one image of a real environment by means of a camera, the image containing at least part of a real object;

performing an image recognition method providing information as to the recognition of the at least one real object in the image;

determining at least one parameter of an environmental situation; and

performing the image recognition method in accordance with the at least one parameter;

wherein a data model is used that is compared with data of the image of the real object in the image of the real environment; and

wherein the at least one parameter has influence on one or both of the preparation or use of the data model.

2. The method of claim 1, further comprising providing correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment according to a result of the performed image recognition method, performing a tracking method that evaluates information with respect to the correspondences, so as to obtain conclusions about the pose of the camera.

3. The method of claim 1, wherein the step of determining the at least one parameter of the environmental situation uses one or more of items of information selected from the group consisting of:

time of day, time of year, weather, in particular rain, clouds, sun and fog, position of moon, snow conditions, tree foliage, altitude above sea level, public events, in particular strong movements, traffic situation, position, orientation in a world coordinate system, histogram of image, temperature, or maintenance step.

4. The method of claim 1, further comprising starting a website in a web browser or an application according to a result of the performed image recognition method.

5. The method of claim 1, wherein the at least one parameter has an effect on minimum requirements with respect to successful retrieval of model data in the image, in particular determines a number of features associated with the real object, which are to be brought into conformity with corresponding features of the image of the real object in the image of the real environment.

6. The method of claim 1, wherein a parameter vector per data model or per feature within the data model is stored.

7. The method of claim 6, wherein the at least one parameter influences the kind of preparation of the data model, in particular influences the preparation of feature detectors and descriptors.

8. The method of claim 1, wherein the data model is extracted from a number of individual models, said data model containing an intersection of information present in several environmental situations.

9. The method of claim 1, wherein several partial systems respectively perform the method, wherein not each of the partial systems calculates a tracking situation itself, but finished situation information is made available to all partial systems, advantageously even a finished recognition or initialization model is made available to the partial systems.

10. The method of claim 9, wherein the partial systems provide information on the environmental situation to a server provided or to a peer-to-peer process.

11. A method for determining the pose of a camera relative to a real environment, said method comprising the following steps:

taking at least one image of a real environment by means of a camera, the image containing at least part of a real object;

determining at least one parameter of an environmental situation;

performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera; and

performing the tracking method in accordance with the at least one parameter;

wherein, by determining the camera pose, depth information with respect to at least one real object contained in the image is calculated, the depth information being used to compute an occlusion model for a real object, when virtual information superimposed on the real environment is occluded in the view by the real object.