IMAGE SEARCH APPARATUS AND METHOD THEREOF

• US 20100299355A1
• Filed: 05/12/2010
• Published: 11/25/2010
• Est. Priority Date: 05/25/2009
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsushita, Masahiro, Magai, Koichi, Shiiyama, Hirotaka, Sohma, Hidetomo
• Granted Patent: US 8,412,730 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

An image search apparatus extracts a symmetric partial image based on acquired pairs of feature points, and divides the partial image at a symmetry axis of the partial image to decide two partial areas, as a first area and a second area based on the tendency of image features in the partial image. In determining the similarity between the query image and a comparison destination image, a coordinate transformation coefficient for a coordinate transformation process is set based on pairs of feature points extracted from the partial area decided as the first area of the two images, the coordinate transformation process using the coordinate transformation coefficient is applied to the pairs of feature points extracted from areas including areas other than the first area of the two images, and coordinates of the pairs of feature points after the coordinate transformation process are compared.

Claims

1. An image search apparatus that searches an image similar to an inputted query image from registered images, the image search apparatus comprising:

an extraction unit configured to acquire a pair of feature points corresponding to each other based on a feature amount of feature points in the query image and to extract a symmetric partial image based on the acquired pair of feature points;

a decision unit configured to decide two partial areas, which are obtained by dividing the partial image at a symmetry axis of the partial image, as a first area and a second area based on the tendency of image features in the partial image;

a setting unit configured to extract, based on the feature amount, at least two sets of corresponding pairs of feature points from the feature points in partial areas decided as the first area of the query image and a registered comparison destination image and to set a coordinate transformation coefficient for transforming the feature points of one of the two images so that the coordinates of the extracted pairs of feature points match; and

a comparison unit configured to extract pairs of feature points corresponding to each other from the feature points in the two images based on the feature amount to determine the similarity between the two images, to transform coordinates of the feature points extracted from the one of the images by a coordinate transformation process using the coordinate transformation coefficient, and to compare the coordinates of the feature points after the transformation with coordinates of the feature points in the other image of the two images.

2. The apparatus according to claim 1, wherein

said comparison unit is configured to extract pairs of corresponding feature points from each of the area decided as the first area, the area decided as the second area, and an asymmetric area excluding the symmetric partial area extracted by said extraction unit, of the two images.

3. The apparatus according to claim 1, wherein

the feature points are rotation-invariant feature points, and the feature amount is a rotation-invariant local feature amount.

4. The apparatus according to claim 1, wherein

if a symmetric partial image is not extracted from the query image in said extraction unit, said setting unit is configured to extract at least two sets of pairs of corresponding feature points from the entire two images.

5. The apparatus according to claim 1, wherein

said decision unit is configured to decide the two partial areas as the first area and the second area based on directions of concentration gradient in the two partial areas.

6. The apparatus according to claim 1, wherein

said decision unit is configured to decide the two partial areas as the first area and the second area depending on whether the numbers of the feature points in the two partial areas are large or small.

7. The apparatus according to claim 1, wherein

the determination of the similarity with the query image is not applied to the comparison destination image without a partial area decided as the first area.

8. The apparatus according to claim 1, wherein

said extraction unit is configured to obtain the symmetry axis by applying a Hough transform to a midpoint group obtained from a plurality of sets of pairs of feature points acquired from the query image and to extract an area which includes a pair of feature points with the midpoint near the symmetry axis, as the symmetric partial image.

9. The apparatus according to claim 1, wherein

among a straight line group passing through two points selected from the midpoint group obtained from the plurality of sets of pairs of feature points acquired from the query image, said extraction unit is configured to set a straight line, in which an accumulated distance from the midpoint group is minimum, as the symmetry axis and to extract an area including the pair of feature points with the midpoint near the symmetry axis as the symmetric partial image.

10. An image search method of searching for an image similar to an inputted query image from registered images, the image search method comprising:

an extraction step of acquiring a pair of feature points corresponding to each other based on a feature amount of feature points from the query image and extracting a symmetric partial image based on the acquired pair of feature points;

a decision step of deciding two partial areas which are obtained by dividing the partial image at a symmetry axis of the partial image, as a first area and a second area based on the tendency of image features in the partial image;

a setting step of extracting, based on the feature amount, at least two sets of corresponding pairs of feature points from the feature points in partial areas decided as the first area of the query image and a registered comparison destination image and setting a coordinate transformation coefficient for transforming the feature points of one of the two images so that the coordinates of the extracted pairs of feature points match; and

a comparison step of extracting pairs of feature points corresponding to each other from the feature points in the two images based on the feature amount to determine the similarity between the two images, transforming coordinates of the feature points extracted from the one of the images by a coordinate transformation process using the coordinate transformation coefficient, and comparing the coordinates of the feature points after the transformation with coordinates of the feature points in the other image of the two images.

11. A computer-readable storage medium storing a computer program causing a computer to execute the image search method according to claim 10.