Local feature amount calculating device, method of calculating local feature amount, corresponding point searching apparatus, and method of searching corresponding point

• US 8,774,508 B2
• Filed: 02/27/2012
• Issued: 07/08/2014
• Est. Priority Date: 02/27/2012
• Status: Active Grant

• Current Assignee: Denso IT Laboratory, Inc. (DENSO Corporation)
• Original Assignee: Denso IT Laboratory, Inc. (DENSO Corporation)
• Inventors: Yoshida, Yuichi, Anbai, Mitsuru
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/405,723
• Publication Number:
  US 20130223737A1
• Time in Patent Office: 862 Days
• Field of Search: 382/170, 382/195
• US Class Current: 382/170
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 2201/11 Technique with transformation invariance effect

Abstract

A local feature amount calculating device includes a local feature amount calculating unit that acquires a local feature amount of a feature point for each pixel by giving a vote of a strength of the direction of a pixel, which is calculated by a direction strength calculating unit, to a vote cell to which a pixel of interest belongs out of vote cells of a rotated vote cell pattern that is acquired by rotating a reference vote cell pattern by an angle corresponding to a main axis direction along a corrected edge gradient direction that is acquired by correcting an edge gradient direction of the pixel that is calculated by a direction strength calculating unit in accordance with the main axis direction.

Claims

1. A local feature amount calculating device that calculates a local feature amount of an image, the local feature amount calculating device comprising:

a feature point near-field region extracting unit that extracts a feature point and a near-field region that is set near the feature point from the image;

a direction strength calculating unit that calculates a direction of each pixel located within the near-field region and a strength of the direction;

a main axis direction detecting unit that detects a main axis direction of the near-field region; and

a local feature amount calculating unit that acquires a local feature amount of the feature point by giving a vote of the strength of the direction of each pixel, which is calculated by the direction strength calculating unit, to a vote cell to which the pixel belongs out of vote cells of a rotated vote cell pattern that is acquired by rotating a reference vote cell pattern by an angle corresponding to the main axis direction along a corrected direction that is acquired by correcting the direction of the pixel that is calculated by the direction strength calculating unit in accordance with the main axis direction.

2. The local feature amount calculating device according to claim 1, wherein the main axis direction detecting unit detects the main axis direction of the near-field area based on directions of a plurality of the pixels located within the near-field region and the strengths of the directions that are calculated by the direction strength calculating unit.

3. The local feature amount calculating device according to claim 1, further comprising:

a direction quantizing unit that quantizes the direction of the pixel,

wherein the main axis direction detecting unit generates a direction histogram by giving a vote of the strength of the direction of the pixel along the direction of the pixel quantized by the direction quantizing unit, approximates a portion near a peak of the direction histogram as a curve, and detects a direction corresponding to a local maximum of a curve as the main axis direction.

4. The local feature amount calculating device according to claim 1, further comprising a vote cell pattern generating unit that generates the rotated vote cell pattern by rotating the reference vote cell pattern by an angle corresponding to the main axis direction.

5. The local feature amount calculating device according to claim 1, further comprising:

a direction correcting pattern generating unit that is used for correcting the direction of the pixel, which is calculated by the direction strength calculating unit, in accordance with the main axis direction,

wherein the local feature amount calculating unit acquires the corrected direction by correcting the direction of each pixel by using the direction correcting pattern.

6. The local feature amount calculating device according to claim 1, further comprising:

a direction quantizing unit that quantizes the direction of the pixel,

wherein the main axis direction detecting unit generates a direction histogram by giving a vote of the strength of the direction of the pixel that is quantized by the direction quantizing unit along the direction of the pixel that is quantized by the direction quantizing unit and detects a direction which has a local maximum value in the direction histogram as the main axis direction.

7. The local feature amount calculating device according to claim 6, further comprising:

a vote cell pattern table in which a plurality of vote cell patterns acquired by rotating the reference vote cell pattern by a plurality of angles corresponding to a plurality of the main axis directions detected by the main axis direction detecting unit is stored,

wherein the local feature amount calculating unit calculates the local feature amount by using a vote cell pattern corresponding to the main axis direction out of the vote cell patterns stored in the vote cell pattern table as the rotated vote cell pattern.

8. The local feature amount calculating device according to claim 6, further comprising:

a representative cell pattern table in which a plurality of representative vote cell patterns having unique shapes of the patterns out of a plurality of vote cell patterns acquired by rotating the reference vote cell pattern by a plurality of angles corresponding to a plurality of the main axis directions detected by the main axis detecting unit is stored; and

a cell number converting table that is used for converting a cell number of the representative vote cell pattern along the main axis direction,

wherein the local feature amount calculating unit determines a vote cell to which a vote of the strength of the direction is given by converting the cell number of the representative cell pattern corresponding to the main axis direction out of the representative vote cell patterns stored in the representative cell pattern table in accordance with the main axis direction by referring to the cell number converting table.

9. The local feature amount calculating device according to claim 6, further comprising:

a direction correcting table in which the corrected direction for a direction quantized by the direction quantizing unit is defined for each of a plurality of the main axis directions detected by the main axis direction detecting unit,

wherein the local feature amount calculating unit acquires the corrected direction corresponding to the direction quantized by the direction quantizing unit along the main axis direction by referring to the direction correcting table.

10. The local feature amount calculating device according to claim 6, further comprising:

a representative direction correcting table in which the main axis directions, in which the corrected directions for the directions quantized by the direction quantizing unit have a same period, are set as one group out of a plurality of the main axis directions detected by the main axis direction detecting unit, and the representative corrected directions for the directions quantized by the direction quantizing unit are defined for each group; and

a direction converting table that is used for converting the representative corrected direction of the representative direction correcting table along the main axis direction,

wherein the local feature amount calculating unit acquires the corrected direction by converting the representative corrected directions corresponding to the main axis direction and the directions quantized by the direction quantizing unit along the main axis direction by referring to the direction converting table out of the representative corrected directions defined in the representative direction correcting table.

11. The local feature amount calculating device according to claim 1, wherein the local feature amount calculating unit gives a vote of the strength of the direction of the pixel to a plurality of vote cells in accordance with a ratio of each vote cell occupying the pixel in a case where the pixel belongs to the plurality of vote cells for each pixel.

12. The local feature amount calculating device according to claim 1, wherein the direction is an edge gradient direction of the pixel, and the strength of the direction is an edge strength of the pixel.

13. The local feature amount calculating device according to claim 1, wherein the local feature amount calculating unit gives a vote while weighting the strength of the direction in accordance with a size of the vote cell to which the vote of the strength of the direction is given.

14. The local feature amount calculating device according to claim 1, wherein the direction strength calculating unit calculates the strength of the direction while more heavily weighting the strength as the pixel is closer to the feature point.

15. A corresponding point searching apparatus that searches for corresponding points of a plurality of images, the corresponding point searching apparatus comprising:

the local feature amount calculating device according to claim 1; and

a corresponding point searching unit that searches for the corresponding points of the plurality of images by comparing the local feature amounts of the plurality of images that are calculated by using the local feature amount calculating device with each other.

16. A computer-implemented method of calculating a local feature amount in which the local feature amount of an image is calculated, the method comprising:

extracting, using a processor, a feature point from the image;

setting, using the processor, a near-field area near the feature point;

calculating, using the processor, a direction of each pixel located within the near-field region;

calculating, using the processor, a strength of the direction of each pixel located within the near-field area;

detecting, using the processor, a main axis direction of the near-field area based on the direction of the plurality of pixels located within the near-field region and the strength of the direction;

acquiring, using the processor, a rotated vote cell pattern by rotating a reference vote cell pattern by an angle corresponding to the main axis direction;

acquiring, using the processor a corrected direction by correcting the direction of each pixel in accordance with the main axis direction; and

acquiring, using the processor a local feature amount of the feature point for each pixel located within the near-field region by giving a vote of the strength of the direction of a pixel to a vote cell to which the pixel belongs out of vote cells of a rotated vote cell pattern along the corrected direction.

18. A method of searching for corresponding points that is used for searching for corresponding points of a plurality of images, the method comprising:

calculating local feature amounts of the plurality of images by using the method according to claim 16 or 17; and

searching for the corresponding points of the plurality of images by comparing the local feature amounts of the plurality of images that are calculated in the calculating of local feature amounts with each other.

19. A non-transitory computer-readable medium having stored thereon a computer executable program that when executed causes an operation processing device to perform the method of searching for corresponding points according to claim 18.

20. A non-transitory computer-readable medium having stored thereon a computer executable program that when executed causes an operation processing device to perform the method of calculating a local feature amount according to claim 16 or 17.

17. A computer-implemented method of calculating a local feature amount in which the local feature amount of an image is calculated, the method comprising:

extracting, using a processor, a feature point from the image;

setting, using the processor, a near-field area near the feature point;

calculating, using the processor, a direction of each pixel located within the near-field region;

quantizing, using the processor, the direction of each pixel located within the near-field region;

calculating, using the processor, a strength of the direction of each pixel located within the near-field area;

generating, using the processor, a direction histogram by giving a vote of the strength of the direction of the pixel along the direction of the pixel that is quantized in the quantizing of the direction and detecting a direction in which a local maximum value of the direction histogram is included as the main axis direction of the near-field region;

determining, using the processor, a vote cell to which the pixel belongs in the vote cell pattern corresponding to the main axis direction for each pixel located within the near-field area by referring to a vote cell pattern table in which a plurality of vote cell patterns acquired by rotating a reference vote cell pattern by a plurality of angles corresponding to a plurality of main axis directions detected in the generating of a direction histogram is stored;

acquiring, using the processor, corrected directions corresponding to the main axis direction and the directions quantized in the quantizing of the direction by referring to a direction correcting table in which the corrected directions for the directions quantized in the quantizing of the direction are defined for each one of the plurality of main axis directions detected in the generating of a direction histogram and detecting of a direction for each pixel located within the near-term area; and

acquiring, using the processor, a local feature amount of the feature point for each pixel located within the near-field region by giving a vote of the strength of the direction of a pixel to a vote cell determined in the determining of a vote cell along the corrected direction.