Method of providing a descriptor for at least one feature of an image and method of matching features

• US 8,942,418 B2
• Filed: 05/28/2010
• Issued: 01/27/2015
• Est. Priority Date: 10/19/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Metaio GmbH (Apple Inc.)
• Inventors: Kurz, Daniel, Benhimane, Selim
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US13/502,952
• Publication Number:
  US 20120219188A1
• Time in Patent Office: 1,705 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method of providing a descriptor for at least one feature of an image comprises the steps of providing an image captured by a capturing device and extracting at least one feature from the image, and assigning a descriptor to the at least one feature, the descriptor depending on at least one parameter which is indicative of an orientation, wherein the at least one parameter is determined from the orientation of the capturing device measured by a tracking system. The invention also relates to a method of matching features of two or more images.

Claims

1. A method of providing a descriptor for at least one feature of an image, comprising the steps of:

providing an image captured by a capturing device and extracting at least one feature from the image; and

assigning a descriptor to the at least one feature, the descriptor depending on at least one parameter which is indicative of an orientation in the image with respect to a projection of the direction of the gravitational force in the image,

wherein the at least one parameter is determined from at least one tracking system; and

wherein the at least one parameter is determined under consideration of intrinsic parameters of the capturing device.

2. The method of claim 1, wherein the at least one parameter is determined from an orientation of the capturing device in reference to a common coordinate system.

3. The method of claim 1, wherein the tracking system comprises a sensor, in particular an inertial sensor, or an accelerometer, or a gyroscope or a compass, which is associated with the capturing device.

4. The method of claim 1, wherein the tracking system comprises a mechanical and/or electromagnetic and/or acoustic and/or optical tracking system.

5. The method of claim 1, wherein the capturing device comprises a range data capturing device, particularly a laser scanner, time-of-flight camera, or a stereo camera, which provides image pixels with an associated depth and/or 3D position.

6. The method of claim 1, wherein the at least one parameter is determined along with information about intrinsic parameters of the capturing device for a transformation in a coordinate system of the image.

7. The method of claim 1, wherein for the at least one extracted feature the at least one parameter is determined from 3D positions of pixels of the image.

8. The method of claim 1, further comprising the step of normalizing the neighborhood of the at least one feature with respect to the orientation.

9. The method of claim 8, wherein in the step of normalizing the neighborhood of the at least one feature the orientation provides an angle for rotating the image.

10. The method of claim 8, wherein in the step of normalizing the neighborhood of the at least one feature the orientation is used to warp the entire image or parts of it to one or more reference orientations to correct for perspective distortions in particular feature neighborhoods.

11. The method of claim 1, where one or more directions of the feature with respect to the orientation in a common coordinate system are computed based on pixel intensities and stored as part of the descriptor.

12. A method of matching features of two or more images, comprising:

extracting at least one first feature of a first image and at least one second feature of a second image;

providing a first descriptor for the first feature and a second descriptor for the second feature, wherein at least one of the first and second descriptors is provided according to the method of claim 1; and

comparing the first and second descriptors in a matching process for the first and second features.

13. The method of claim 12, wherein in the matching process it is determined based on a similarity measure whether the first and second features correspond with each other.

14. The method of claim 13 where the similarity measure comprises a multi-stage method that successively computes different measures to reject wrong matches as early as possible.

15. The method of claim 12, wherein the method is implemented in a process of stereo matching, particularly wide-baseline stereo matching, camera tracking, image retrieval, object recognition, visual search, pose estimation, visual surveillance, scene reconstruction, motion estimation, panorama stitching or image restoration.

16. The method of claim 1, wherein the method is implemented in a process of stereo matching, particularly wide-baseline stereo matching, camera tracking, image retrieval, object recognition, visual search, pose estimation, visual surveillance, scene reconstruction, motion estimation, panorama stitching or image restoration.

17. A non-transitory computer readable medium comprising software code sections adapted to be loaded into the internal memory of a digital computer system coupled with at least one capturing device for capturing an image or a memory medium for loading an image therefrom by means of which the steps according to claim 1 are performed when running on said computer system.

18. The method according to claim 1, wherein the orientation is aligned with the projection of the direction of the gravitational force in the image.