Feature value extraction apparatus and place estimation apparatus

• US 9,396,396 B2
• Filed: 11/06/2013
• Issued: 07/19/2016
• Est. Priority Date: 11/06/2012
• Status: Active Grant

• Current Assignee: Soinn Inc.
• Original Assignee: Soinn Holdings Llc
• Inventors: Hasegawa, Osamu, Hua, Gangchen
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): PERLMAN, DAVID S
• Application Number:
  US14/440,768
• Publication Number:
  US 20150294157A1
• Time in Patent Office: 986 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/36 Indoor scenes
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: G06V 2201/11 Technique with transformation invariance effect

Abstract

A place estimation apparatus performs a place estimation process by using position-invariant feature values extracted by a feature value extraction unit. The feature value extraction unit includes local feature value extraction unit that extracts local feature values from each of input images formed from successively-shot successive images, feature value matching unit that obtains matching between successive input images based on the extracted local feature values, corresponding feature value selection unit that selects matched feature values as corresponding feature values, and position-invariant feature value extraction unit that obtains position-invariant feature values based on the corresponding feature values. The position-invariant feature value extraction unit extracts, from among the corresponding feature values, corresponding feature values whose position change is equal to or less than a predetermined threshold as the position-invariant feature values.

Claims

1. A place estimation apparatus comprising:

a feature value extraction unit that extracts a position-invariant feature value from an input image;

a matching unit that refers to a database and obtaining matching between the input image and a registered place, the database storing each registered place and its associated position-invariant feature value;

a similarity-level calculation unit that calculates a similarity level in which a registered place near a selected registered place is included in the calculation when the matching is equal to or higher than a predetermined threshold; and

a place recognition unit that recognizes that the input image is the registered place when the similarity level is equal to or higher than a predetermined threshold,

wherein

the feature value extraction unit comprises;

a local feature value extraction unit that extracts a local feature value from each of input images formed from successively-shot successive images;

a feature value matching unit that obtains matching between successive input images for the local feature value extracted by the local feature value extraction unit;

a corresponding feature value selection unit that selects a feature value, for which the matching is obtained between the successive images by the feature value matching unit, as a corresponding feature value; and

a position-invariant feature value extraction unit that obtains a position-invariant feature value based on the corresponding feature value, and

the position-invariant feature value extraction unit extracts, from among the corresponding feature values, a corresponding feature value of which a change in a relative position with respect to another corresponding feature value present in the input image is equal to or less than a predetermined threshold as the position-invariant feature value;

wherein the position-invariant feature value extraction unit repeatedly performs, for all of the corresponding feature values, processes comprising;

defining a vector in each of the input images formed from the successive images, the vector being formed by a center of gravity of the already-extracted position-invariant feature values and one of the corresponding feature values;

newly extracting, when a difference between the vectors formed in respective input images formed from the successive images is equal to or less than a predetermined threshold, the one of the corresponding feature values as the position-invariant feature value; and

updating the center of gravity by a center of gravity between the center of gravity and the new position-invariant feature value.

2. The place estimation apparatus according to claim 1, wherein the position-invariant feature value extraction unit:

selects two pairs of the corresponding feature values in a random fashion, the pairs of the corresponding feature values being present in respective input images formed from the successive images and corresponding to each other;

defines a vector in each of the input images formed from the successive images, the vector being formed by two corresponding feature values; and

extracts, when a difference between the vectors is equal to or less than a predetermined threshold, those corresponding feature values as first position-invariant feature values.

3. The place estimation apparatus according to claim 1, wherein the local feature value is a feature value of at least one of a SIFT (Scale Invariant Feature Transformation) or SURF (Speed Up Robustness Features).

4. The place estimation apparatus according to claim 1, wherein the position-invariant feature value extraction unit outputs a logical sum of a first position-invariant feature value and a second position-invariant feature value as the position-invariant feature value, the first position-invariant feature value being extracted from an image shot at a time t and an image shot at a time t−

1, the secondposition-invariant feature value being extracted from the image shot at the time t and an image shot at a time t+1, the time t−

1 being before the time t, the time t+1 being after the time t.

5. The place estimation apparatus according to claim 1, wherein the corresponding feature value selection unit inspects to determine whether there is an error in the matching obtained by the feature value matching unit based on a relative distance between a certain corresponding feature value and another corresponding feature value, and removes, if there is an error, an erroneously-matched corresponding feature value.

6. The place estimation apparatus according to claim 1, wherein the position-invariant feature value extraction unit extracts the position-invariant feature value by detecting an affine-invariant quantity related to the corresponding feature value.

7. A feature value extraction apparatus comprising:

local feature value extraction unit that extracts a local feature value from each of input images formed from successively-shot successive images;

feature value matching unit that obtains matching between successive input images for the local feature value extracted by the local feature value extraction unit;

corresponding feature value selection unit that selects a feature value, for which the matching is obtained between the successive images by the feature value matching unit, as a corresponding feature value; and

position-invariant feature value extraction unit that obtains a position-invariant feature value based on the corresponding feature value, and

the position-invariant feature value extraction unit extracts, from among the corresponding feature values, a corresponding feature value of which a change in a relative position with respect to another corresponding feature value present in the input image is equal to or less than a predetermined threshold as the position-invariant feature value;

wherein the position-invariant feature value extraction unit repeatedly performs, for all of the corresponding feature values, processes comprising;

defining a vector in each of the input images formed from the successive images, the vector being formed by a center of gravity of the already-extracted position-invariant feature values and one of the corresponding feature values;

newly extracting, when a difference between the vectors formed in respective input images formed from the successive images is equal to or less than a predetermined threshold, the one of the corresponding feature values as the position-invariant feature value; and

updating the center of gravity by a center of gravity between the center of gravity and the new position-invariant feature value.

8. The feature value extraction apparatus according to claim 7, wherein the position-invariant feature value extraction unit:

selects two pairs of the corresponding feature values in a random fashion, the pairs of the corresponding feature values being present in respective input images formed from the successive images and corresponding to each other;

defines a vector in each of the input images formed from the successive images, the vector being formed by two corresponding feature values; and

extracts, when a difference between the vectors is equal to or less than a predetermined threshold, those corresponding feature values as first position-invariant feature values.

9. The feature value extraction apparatus according to claim 7, wherein the local feature value is a feature value of at least one of a SIFT (Scale Invariant Feature Transformation) or SURF (Speed Up Robustness Features).

10. The feature value extraction apparatus according to claim 7, wherein the position-invariant feature value extraction unit outputs a logical sum of a first position-invariant feature value and a second position-invariant feature value as the position-invariant feature value, the first position-invariant feature value being extracted from an image shot at a time t and an image shot at a time t−

1, the second position-invariant feature value being extracted from the image shot at the time t and an image shot at a time t+1, the time t−

1 being before the time t, the time t+1 being after the time t.

11. The feature value extraction apparatus according to claim 7, wherein the corresponding feature value selection unit inspects to determine whether there is an error in the matching obtained by the feature value matching unit based on a relative distance between a certain corresponding feature value and another corresponding feature value, and removes, if there is an error, an erroneously-matched corresponding feature value.

12. The feature value extraction apparatus according to claim 7, wherein the position-invariant feature value extraction unit extracts the position-invariant feature value by detecting an affine-invariant quantity related to the corresponding feature value.