Systems and methods for matching scenes using mutual relations between features

• US 9,091,553 B2
• Filed: 12/22/2009
• Issued: 07/28/2015
• Est. Priority Date: 12/22/2009
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Kotaba, Ondrej, Lukas, Jan
• Primary Examiner(s): To, Tuan C.
• Assistant Examiner(s): SMITH, ISAAC G
• Application Number:
  US12/644,559
• Publication Number:
  US 20110153206A1
• Time in Patent Office: 2,044 Days
• Field of Search: 701/208, 701/527, 345/419, 345/420, 348/113, 348/208.4, 348/308, 348/218.1, 382/106, 382/124, 382/154, 382/173, 382/181, 382/209, 382/218, 382/305, 382/100, 382/113, 382/153, 382/128, 600/407, 700/83, 703/7, 705/14.58
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3804 Creation or updating of map data
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Embodiments of the present invention provide improved systems and methods for matching scenes. In one embodiment, a processor for implementing robust feature matching between images comprises: a first process for extracting a first feature set from a first image projection and extracting a second feature set from a second image projection; a memory for storing the first feature set and the second feature set; and a second process for feature matching using invariant mutual relations between features of the first feature set and the second feature set; wherein the second feature set is selected from the second image projection based on the identification of similar descriptive subsets between the second image projection and the first image projection.

Claims

1. A non-transitory computer readable media device having computer-executable instructions for a method for matching image features, the method comprising:

defining a first descriptive subset of features, including at least one minimal subset, from a feature set A extracted by at least one processor from a first image projection;

selecting at least two second descriptive subsets from a feature set B extracted by the at least one processor from a second image projection, the at least two second descriptive subsets correlated to at least two minimal subsets, which meet a congruency criteria with the first descriptive subset of features from the feature set A, wherein the at least two second descriptive subsets are selected based on a similarity of invariant mutual relations between features of one of the at least one minimal subset in the first descriptive subset and invariant mutual relations between features of the at least two minimal subsets in the at least two second descriptive subsets, wherein the first descriptive subset and the at least two second descriptive subsets define at least two pairs of similar descriptive subsets;

calculating at the at least one processor a transformation T from at least one of the at least two pairs of similar descriptive subsets to form a hypothesis H;

evaluating at the at least one processor a quality of the hypothesis H;

storing the hypothesis H into a memory device, the hypothesis H including transformation T and a plurality of paired features that support the hypothesis H; and

selecting a final hypothesis H_B having a highest quality as determined by a quality metric, the method further comprising at least one of;

determining coordinates for a new position by applying the transformation T belonging to the selected final hypothesis H_B to coordinates in a navigation frame; and

generating a map in the memory device for navigating a vehicle, the map based on matching features from the first image projection with the second image projection.

2. The non-transitory computer readable media device of claim 1, the method further comprising:

defining a plurality of descriptive subsets of features from the feature set A;

for each of the plurality of descriptive subsets from the feature set A, selecting at least two descriptive subsets from the feature set B based on a similarity of invariant mutual relations between features to form a plurality of similar descriptive subset pairs;

calculating a hypothetic transformation T for each of the plurality of similar descriptive subset pairs to form at least one hypothesis H; and

storing into memory a plurality of paired features that support the final hypothesis H_B, where this plurality of paired features each include a feature from the feature set A and a feature from the feature set B.

3. The non-transitory computer readable media device of claim 2, the method further comprising determining a distance to an object based on a best transformation T_B and plurality of paired features that support the final hypothesis H_B.

4. The non-transitory computer readable media device of claim 1, the method further comprising:

determining a number n of features to include in the first descriptive subset based on a type of projection utilized in an image sensor to generate the first image projection from a real-world scene; and

an anticipated transformation undergone between the first image projection and the second image projection.

5. The non-transitory computer readable media device of claim 1, wherein similarity is determined by estimating using one or both of variances and co-variance of invariant properties.

6. The non-transitory computer readable media device of claim 1, wherein selection of the first descriptive subset is based on one of a random, a pseudorandom, a quasi-random,. or a deterministic process.

7. The non-transitory computer readable media device of claim 1, the method further comprising calculating a solution for transformation T from a plurality of paired features that support the final hypothesis H_B using an over determined system parameter estimator.

8. The non-transitory computer readable media device of claim 1, the method further comprising:

calculating a match between features from a captured image and a map of features stored in the memory device; and

defining a position and an orientation from the final hypothesis H_B.

9. The non-transitory computer readable media device of claim 1, wherein the invariant mutual relations between features include at least one of a set of invariant mutual relations including a distance, an area, an angle, a solid angle, a volume, a generalized area or volume to higher dimensions, a central moment or a histogram.