Image collation system and image collation method

• US 20070031001A1
• Filed: 10/21/2004
• Published: 02/08/2007
• Est. Priority Date: 10/21/2003
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Hamanaka, Masahiko
• Granted Patent: US 7,715,619 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

There are provided a three-dimensional data input means (10) for input the three-dimensional data of an object, a reference image storing unit (30) for storing a reference image of at least one object, a pose candidate deciding means (20) for generating a pose candidate that is a candidate for the pose of the object, a comparison image generating means (40) for generating a comparison image close to the reference image while projecting the three-dimensional data onto a two-dimensional image in accordance with the pose candidate, and an image comparing means (55) for performing comparison based on either a distance value or similarity degree between the reference image and the comparison image. This allows comparison or search to be performed with high accuracy even when the reference image of the object is captured under different conditions such as pose and illumination, even when no three-dimensional object model of the object can be obtained in advance, or even when only one or few reference images exist.

Claims

1. An image comparison system characterized by comprising:

three-dimensional data input means for input three-dimensional data of an object;

reference image storing means for storing a reference image of at least one object;

pose candidate deciding means for generating at least one pose candidate as a candidate for pose of the object;

comparison image generating means for generating at least one comparison image close to the reference image while projecting the three-dimensional data onto a two-dimensional image in accordance with the pose candidate; and

image comparing means for performing comparison on the basis of one of a distance value and a similarity degree between the reference image and the comparison image.

2. An image comparison system according to claim 1, characterized in that said image comparing means comprises:

calculating means for calculating one of the distance value and the similarity degree between the reference image and the comparison image;

selecting means for selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree; and

comparing means for performing comparison on the basis of one of a result of comparison between the minimum distance value and a threshold value and a result of comparison between the maximum similarity degree and a threshold value.

3. An image comparison system according to claim 1, characterized in that

said comparison image generating means generates a comparison image close to each reference image, and

said image comparing means comprises:

calculating means for calculating one of a distance value and a similarity degree between each reference image and the comparison image;

selecting means for selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree for each reference image; and

comparing means for outputting, as a comparison result, one of a reference image including a smallest minimum distance value which is a smallest one of minimum distance values and a reference image including a largest maximum similarity degree which is a largest one of maximum similarity degrees.

4. An image comparison system according to claim 1, characterized by further comprising:

reference correction coefficient storing means for storing a correction coefficient corresponding to the reference image; and

correcting means for correcting one of the minimum distance value and the maximum similarity degree by using the correction coefficient.

5. An image comparison system according to claim 1, characterized by further comprising reference weighting coefficient storing means for storing a weighting coefficient corresponding to the reference image,

said image comparing means comprising calculating means for calculating one of the distance value and the similarity degree between the reference image and the comparison image by using the weighting coefficient corresponding to the reference image.

6. An image comparison system according to claim 1, characterized by further comprising:

standard three-dimensional reference point storing means for storing a standard three-dimensional reference point corresponding to a standard three-dimensional object model;

standard three-dimensional weighting coefficient storing means for storing a standard three-dimensional weighting coefficient;

three-dimensional reference point extracting means for extracting a three-dimensional reference point from the input three-dimensional data; and

input weighting coefficient converting means for obtaining a coordinate correspondence of the standard three-dimensional weighting coefficient to the three-dimensional data by using the standard three-dimensional reference point and the three-dimensional reference point of the three-dimensional data, and converting the standard three-dimensional weighting coefficient into a two-dimensional weighting coefficient in accordance with the pose candidate,

said image comparing means comprising calculating means for calculating one of the distance value and the similarity degree between the reference image and the comparison image by using the converted two-dimensional weighting coefficient.

7. An image comparison system according to claim 1, characterized by further comprising:

representative three-dimensional object model storing means for storing representative ones of three-dimensional object models as representative three-dimensional object models;

group storing means for storing related information of the representative three-dimensional object models and reference images;

three-dimensional comparing means for comparing the input three-dimensional data with the representative three-dimensional object models, and selecting a representative three-dimensional object model similar to the three-dimensional data; and

reference image selecting means for selecting a reference image corresponding to the selected representative three-dimensional object model by referring to the related information,

wherein said image comparing means compares the selected reference image with the input three-dimensional data.

8. An image comparison system according to claim 1, characterized by further comprising:

representative image storing means for storing representative ones of images as representative images;

group storing means for storing related information of the representative images and reference images;

representative image selecting means for comparing the input three-dimensional data with the representative images, and selecting a representative image similar to the three-dimensional data; and

reference image selecting means for selecting a reference image corresponding to the selected representative image by referring to the related information,

wherein said image comparing means compares the selected reference image with the input three-dimensional data.

9. An image comparison system according to claim 4, characterized in that the correction coefficient is determined on the basis of at least one of a distance value and a similarity degree between a representative three-dimensional object model and the reference image.

10. An image comparison method characterized by comprising the steps of:

input three-dimensional data of an object;

generating at least one pose candidate as a candidate for pose of the object;

generating at least one comparison image close to a reference image while projecting the three-dimensional data onto a two-dimensional image in accordance with the pose candidate; and

performing comparison on the basis of one of a distance value and a similarity degree between the reference image and the comparison image.

11. An image comparison method according to claim 10, characterized in that the step of performing comparison comprises the steps of:

calculating one of the distance value and the similarity degree between the reference image and the comparison image;

selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree; and

performing comparison on the basis of one of a result of comparison between the minimum distance value and a threshold value and a result of comparison between the maximum similarity degree and a threshold value.

12. An image comparison method according to claim 10, characterized in that

the step of generating a comparison image comprises the step of generating a comparison image close to each reference image, and

the step of performing comparison comprises the steps of:

calculating one of a distance value and a similarity degree between each reference image and the comparison image;

selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree for each reference image; and

outputting, as a comparison result, one of a reference image including a smallest minimum distance value which is a smallest one of minimum distance values and a reference image including a largest maximum similarity degree which is a largest one of maximum similarity degrees.

13. An image comparison method according to claim 10, characterized by further comprising the step of correcting one of the minimum distance value and the maximum similarity degree by using a correction coefficient corresponding to the reference image.

14. An image comparison method according to claim 10, characterized in that the step of performing comparison comprises the step of calculating one of the distance value and the similarity degree between the reference image and the comparison image by using a weighting coefficient corresponding to the reference image.

15. An image comparison method according to claim 10, characterized by further comprising the steps of:

extracting a three-dimensional reference point from the input three-dimensional data; and

obtaining a coordinate correspondence of a standard three-dimensional weighting coefficient to the three-dimensional data by using a standard three-dimensional reference point corresponding to a standard three-dimensional object model and the three-dimensional reference point of the three-dimensional data, and converting the standard three-dimensional weighting coefficient into a two-dimensional weighting coefficient in accordance with the pose candidate,

the step of performing comparison comprising the step of calculating one of the distance value and the similarity degree between the reference image and the comparison image by using the converted two-dimensional weighting coefficient.

16. An image comparison method according to claim 10, characterized by further comprising the steps of:

comparing the input three-dimensional data with representative three-dimensional object models which are representative ones of three-dimensional object models, and selecting a representative three-dimensional object model similar to the three-dimensional data; and

selecting a reference image corresponding to the selected representative three-dimensional object model by referring to information indicating relations between the representative three-dimensional object models and reference images,

the step of performing comparison comprising the step of comparing the selected reference image with the input three-dimensional data.

17. An image comparison method according to claim 10, characterized by further comprising the step of:

comparing the input three-dimensional data with representative images which are representative ones of images, and selecting a representative image similar to the three-dimensional data; and

selecting a reference image corresponding to the selected representative image by referring to information indicating relations between the representative images and reference images,

the step of performing comparison comprising the step of comparing the selected reference image with the input three-dimensional data.

18. An image comparison method according to claim 13, characterized by further comprising the step of determining the correction coefficient on the basis of at least one of a distance value and a similarity degree between a representative three-dimensional object model and the reference image.

23. A program according to claim 10, wherein in the procedure of performing comparison, the program causes the computer to execute a procedure of calculating one of the distance value and the similarity degree between the reference image and the comparison image by using a weighting coefficient corresponding to the reference image.

19. A program for causing a computer to execute:

a procedure of input three-dimensional data of an object;

a procedure of generating at least one pose candidate as a candidate for pose of the object;

a procedure of generating at least one comparison image close to a reference image while projecting the three-dimensional data onto a two-dimensional image in accordance with the pose candidate; and

a procedure of performing comparison on the basis of one of a distance value and a similarity degree between the reference image and the comparison image.

20. A program according to claim 19, wherein in the procedure of performing comparison, the program causes the computer to execute:

a procedure of calculating one of the distance value and the similarity degree between the reference image and the comparison image;

a procedure of selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree; and

a procedure of performing comparison on the basis of one of a result of comparison between the minimum distance value and a threshold value and a result of comparison between the maximum similarity degree and a threshold value.

21. A program according to claim 19, wherein

in the procedure of generating a comparison image, the program causes the computer to execute a procedure of generating a comparison image close to each reference image, and

in the procedure of performing comparison, the program causes the computer to execute:

a procedure of calculating one of a distance value and a similarity degree between each reference image and the comparison image;

a procedure of selecting one of a minimum distance value which is a smallest distance value and a maximum similarity degree which is a largest similarity degree for each reference image; and

a procedure of outputting, as a comparison result, one of a reference image including a smallest minimum distance value which is a smallest one of minimum distance values and a reference image including a largest maximum similarity degree which is a largest one of maximum similarity degrees.

22. A program according to claim 19, which further causes the computer to execute a procedure of correcting one of the minimum distance value and the maximum similarity degree by using a correction coefficient corresponding to the reference image.

24. A program according to claim 19, which further causes the computer to execute:

a procedure of extracting a three-dimensional reference point from the input three-dimensional data; and

a procedure of obtaining a coordinate correspondence of a standard three-dimensional weighting coefficient to the three-dimensional data by using a standard three-dimensional reference point corresponding to a standard three-dimensional object model and the three-dimensional reference point of the three-dimensional data, and converting the standard three-dimensional weighting coefficient into a two-dimensional weighting coefficient in accordance with the pose candidate,

wherein in the procedure of performing comparison, the program causes the computer to execute a procedure of calculating one of the distance value and the similarity degree between the reference image and the comparison image by using the converted two-dimensional weighting coefficient.

25. A program according to claim 19, which further causes the computer to execute:

a procedure of comparing the input three-dimensional data with representative three-dimensional object models which are representative ones of three-dimensional object models, and selecting a representative three-dimensional object model similar to the three-dimensional data; and

a procedure of selecting a reference image corresponding to the selected representative three-dimensional object model by referring to information indicating relations between the representative three-dimensional object models and reference images,

wherein in the procedure of performing comparison, the program causes the computer to execute a procedure of comparing the selected reference image with the input three-dimensional data.

26. A program according to claim 19, which further causes the computer to execute:

a procedure of comparing the input three-dimensional data with representative images which are representative ones of images, and selecting a representative image similar to the three-dimensional data; and

a procedure of selecting a reference image corresponding to the selected representative image by referring to information indicating relations between the representative images and reference images,

wherein in the procedure of performing comparison, the program causes the computer to execute a procedure of comparing the selected reference image with the input three-dimensional data.

27. A program according to claim 22, which further causes the computer to execute a procedure of determining the correction coefficient on the basis of at least one of a distance value and a similarity degree between a representative three-dimensional object model and the reference image.