Image recognition device using feature points, method for recognizing images using feature points, and robot device which recognizes images using feature points
First Claim
1. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
- feature point extracting means for extracting a feature point from each of the object image and the model image;
feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions;
feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and
model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any,wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms by sequentially shifting all of the feature points in the one of the density gradient direction histograms one by one to generate a plurality of shifted histograms, and generates the candidate-associated feature point pair by determining a shortest distance between (1) an other of the density gradient direction histograms and (2) the one of the density gradient direction histograms and the shifted histograms.
1 Assignment
0 Petitions
Accused Products
Abstract
In an image recognition apparatus, feature point extraction sections and extract feature points from a model image and an object image. Feature quantity retention sections extract a feature quantity for each of the feature points and retain them along with positional information of the feature points. A feature quantity comparison section compares the feature quantities with each other to calculate the similarity or the dissimilarity and generates a candidate-associated feature point pair having a high possibility of correspondence. A model attitude estimation section repeats an operation of projecting an affine transformation parameter determined by three pairs randomly selected from the candidate-associated feature point pair group onto a parameter space. The model attitude estimation section assumes each member in a cluster having the largest number of members formed in the parameter space to be an inlier. The model attitude estimation section finds the affine transformation parameter according to the least squares estimation using the inlier and outputs a model attitude determined by this affine transformation parameter.
-
Citations
23 Claims
-
1. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms by sequentially shifting all of the feature points in the one of the density gradient direction histograms one by one to generate a plurality of shifted histograms, and generates the candidate-associated feature point pair by determining a shortest distance between (1) an other of the density gradient direction histograms and (2) the one of the density gradient direction histograms and the shifted histograms. - View Dependent Claims (2, 3, 4, 5)
-
-
6. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms and generates the candidate-associated feature point pair by assuming a shortest distance to be a distance between the density gradient direction histograms, and wherein the model attitude estimation means repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an amine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space. - View Dependent Claims (7, 8)
-
-
9. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any; and candidate-associated feature point pair selection means for performing generalized Hough transform for a candidate-associated feature point pair generated by the feature quantity comparison means, assuming a rotation angle, enlargement and reduction ratios, and horizontal and vertical linear displacements to be a parameter space, and selecting a candidate-associated feature point pair having voted for the most voted parameter from candidate-associated feature point pairs generated by the feature quantity comparison means, wherein the model attitude estimation means detects the presence or absence of the model on the object image using a candidate-associated feature point pair selected by the candidate-associated feature point pair selection means and estimates a position and an attitude of the model, if any, wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms and generates the candidate-associated feature point pair by assuming a shortest distance to be a distance between the density gradient direction histograms.
-
-
10. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms and generates the candidate-associated feature point pair by assuming a shortest distance to be a distance between the density gradient direction histograms, and wherein the feature point extraction means extracts a local maximum point or a local minimum point in second-order differential filter output images with respective resolutions as the feature point, i.e., a point free from positional changes due to resolution changes within a specified range in a multi-resolution pyramid structure acquired by repeatedly applying smoothing filtering and reduction resampling to the object image or the model image.
-
-
11. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining a feature quantity in a neighboring region at the feature point in each of the object image and the model image, the feature quantity being a density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities, each candidate-associated feature point pair including one feature point of the object image and one feature point of the model image; and model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the model attitude estimation means repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space. - View Dependent Claims (12, 13)
-
-
14. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining a feature quantity in a neighboring region at the feature point in each of the object image and the model image, the feature quantity being a density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any; and candidate-associated feature point pair selection means for performing generalized Hough transform for a candidate-associated feature point pair generated by the feature quantity comparison means, assuming a rotation angle, enlargement and reduction ratios, and horizontal and vertical linear displacements to be a parameter space, and selecting a candidate-associated feature point pair having voted for the most voted parameter from candidate-associated feature point pairs generated by the feature quantity comparison means, wherein the model attitude estimation means repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space, and wherein the model attitude estimation means detects the presence or absence of the model on the object image using a candidate-associated feature point pair selected by the candidate-associated feature point pair selection means and estimates a position and an attitude of the model, if any.
-
-
15. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
feature point extracting means for extracting a feature point from each of the object image and the model image; feature quantity retention means for extracting and retaining a feature quantity in a neighboring region at the feature point in each of the object image and the model image, the feature quantity being a density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; model attitude estimation means for detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the model attitude estimation means repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space, and wherein the feature point extraction means extracts a local maximum point or a local minimum point in second-order differential filter output images with respective resolutions as the feature point, i.e., a point free from positional changes due to resolution changes within a specified range in a multi-resolution pyramid structure acquired by repeatedly applying smoothing filtering and reduction resampling to the object image or the model image.
-
-
16. An image recognition method which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the method comprising:
-
at least one processor performing the steps of, extracting a feature point from each of the object image and the model image; extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the comparing itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms by sequentially shifting all of the feature points in the one of the density gradient direction histograms one by one to generate a plurality of shifted histograms, and generates the candidate-associated feature point pair by determining a shortest distance between (1) an other of the density gradient direction histograms and (2) the one of the density gradient direction histograms and the shifted histograms.
-
-
17. An image recognition method which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the method comprising:
-
at least one processor performing the steps of, extracting a feature point from each of the object image and the model image; extracting and retaining a feature quantity in a neighboring region at the feature point in each of the object image and the model image, the feature quantity being a density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; comparing the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities, each candidate-associated feature point pair including one feature point of the object image and one feature point of the model image; and detecting the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the detecting repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space.
-
-
18. An autonomous robot apparatus capable of comparing an input image with a model image containing a model to be detected and extracting the model from the input image, the apparatus comprising:
-
image input means for imaging an outside environment to generate the input image; feature point extracting means for extracting a feature point from each of the input image and the model image; feature quantity retention means for extracting and retaining, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the input image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the input image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and model attitude estimation means for detecting the presence or absence of the model on the input image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the feature quantity comparison means itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms by sequentially shifting all of the feature points in the one of the density gradient direction histograms one by one to generate a plurality of shifted histograms, and generates the candidate-associated feature point pair by determining a shortest distance between (1) an other of the density gradient direction histograms and (2) the one of the density gradient direction histograms and the shifted histograms.
-
-
19. An autonomous robot apparatus capable of comparing an input image with a model image containing a model to be detected and extracting the model from the input image, the apparatus comprising:
-
image input means for imaging an outside environment to generate the input image; feature point extracting means for extracting a feature point from each of the input image and the model image; feature quantity retention means for extracting and retaining a feature quantity in a neighboring region at the feature point in each of the input image and the model image, the feature quantity being a density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; feature quantity comparison means for comparing the feature quantity of each feature point of the input image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities, each candidate-associated feature point pair including one feature point of the object image and one feature point of the model image; and a model attitude estimation means for detecting the presence or absence of the model on the input image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the model attitude estimation means repeatedly projects an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and finds an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space.
-
-
20. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
a feature point extracting unit configured to extract a feature point from each of the object image and the model image; a feature quantity retention unit configured to extract and retain, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; a feature quantity comparison unit configured to compare the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and generating a candidate-associated feature point pair having similar feature quantities; and a model attitude estimation unit configured to detect the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the feature quantity comparison unit itinerantly shifts one of the density gradient direction histograms of feature points to be compared in density gradient direction to find distances between the density gradient direction histograms by sequentially shifting all of the feature points in the one of the density gradient direction histograms one by one to generate a plurality of shifted histograms, and generates the candidate-associated feature point pair by determining a shortest distance between (1) an other of the density gradient direction histograms and (2) the one of the density gradient direction histograms and the shifted histograms.
-
-
21. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
a feature point extracting unit configured to extract a feature point from each of the object image and the model image; a feature quantity retention unit configured to extract and retain, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; a feature quantity comparison unit configured to compare the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and to generate a candidate-associated feature point pair having similar feature quantities, each candidate-associated feature point pair including one feature point of the object image and one feature point of the model image, each feature quantity not including gradient magnitude information; and a model attitude estimation unit configured to detect the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the model attitude estimation unit is configured to repeatedly project an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and to find an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space. - View Dependent Claims (22)
-
-
23. An image recognition apparatus which compares an object image containing a plurality of objects with a model image containing a model to be detected and extracts the model from the object image, the apparatus comprising:
-
a feature point extracting unit configured to extract a feature point from each of the object image and the model image; a feature quantity retention unit configured to extract and retain, as a feature quantity, a density gradient direction histogram at least acquired from density gradient information in a neighboring region at the feature point in each of the object image and the model image, the density gradient direction histogram storing a number of points near the feature point having each of a plurality of gradient directions; a feature quantity comparison unit configured to compare the feature quantity of each feature point of the object image with the feature quantity of each feature point of the model image and to generate a candidate-associated feature point pair having similar feature quantities, each feature quantity not including gradient magnitude information; and a model attitude estimation unit configured to detect the presence or absence of the model on the object image using the candidate-associated feature point pair and estimating a position and an attitude of the model, if any, wherein the model attitude estimation unit is configured to repeatedly project an affine transformation parameter determined from three randomly selected candidate-associated feature point pairs onto a parameter space and to find an affine transformation parameter to determine a position and an attitude of the model based on an affine transformation parameter belonging to a cluster having the largest number of members out of clusters formed on a parameter space, and wherein the feature quantity comparison unit is configured to generate the dissimilarity for each respective candidate-associated feature point pair by itinerantly shifting by one step the plurality of gradient directions for one of the object image and the model image to compute a number of similarities to a number of the plurality of gradient directions, and to take a minimum dissimilarity to be the dissimilarity.
-
Specification