Device, method and record medium for image comparison

• US 20010033685A1
• Filed: 04/03/2001
• Published: 10/25/2001
• Est. Priority Date: 04/03/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ishiyama, Rui
• Granted Patent: US 7,227,973 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

An image comparison device for comparing a target object (an object to be recognized or identified) with registered objects (objects which are preliminarily registered to be compared with the target object) includes a registration section and a comparison section. The registration section measures the 3-D shape and surface reflectance of each registered object and registers and stores them as registered data. In the comparison section, a photographing section photographs a target object and thereby obtains an input image. A position/pose estimation section estimates the position/pose of the target object in the input image and thereby obtains position/pose parameters. An illumination correction section generates a reference image of each registered object (a 2-D image of the registered object in the same position/pose as the target object in the input image and under the same illumination condition as in the input image) by use of the position/pose parameters obtained by the position/pose estimation section and the registered data. An image comparison section calculates an evaluation value concerning the similarity between the reference image and the input image. A judgment section judges whether or not each of the registered objects is the same as or similar to the target object based on the evaluation value.

Claims

1. An image comparison device comprising:

a registration means for obtaining and registering 3-D data of all or part of one or more registered objects; and

a comparison means for obtaining 2-D data of a target object and comparing the obtained 2-D data with the data registered in the registration means for conducting judgment concerning the similarity/sameness between the target object and each of the one or more registered objects.

2. An image comparison device as claimed in

claim 1

, wherein the comparison means includes:

a photographing means for photographing the target object and thereby obtaining an input image;

a position/pose determination means for determining the position/pose of the target object in the input image obtained by the photographing means;

an illumination correction means for generating an image of each registered object in the same position/pose as the target object in the input image and under an illumination condition most similar to that of the input image as a reference image by use of the position/pose determined by the position/pose determination means and the data registered in the registration means;

an image comparison means for comparing each reference image generated by the illumination correction means with the input image obtained by the photographing means and thereby calculating an evaluation value concerning the similarity between the two images; and

a judgment means for judging whether or not each of the registered objects registered in the registration means is the same as or similar to the target object photographed by the photographing means based on the evaluation value calculated by the image comparison means.

3. An image comparison device as claimed in

claim 2

, wherein the registration means includes:

a 3-D shape measurement means for measuring the 3-D shape of each registered object;

a reflectance measurement means for measuring the reflectance at each position on the surface of the 3-D shape of the registered object; and

a data storage means for storing the 3-D shapes measured by the 3-D shape measurement means and the reflectance measured by the reflectance measurement means as the registered data.

4. An image comparison device as claimed in

claim 2

, wherein the registration means includes:

a 3-D shape measurement means for measuring the 3-D shapes of all or part of the one or more registered objects;

a color/intensity measurement means for photographing the registered object and thereby obtaining information concerning color or intensity of the registered object; and

a data storage means for storing the 3-D shapes measured by the 3-D shape measurement means and the color/intensity information obtained by the color/intensity measurement means as the registered data.

5. An image comparison device as claimed in

claim 2

, wherein the registration means includes:

a 3-D shape measurement means for measuring the 3-D shapes of all or part of the one or more registered objects;

an average shape generation means for generating an average 3-D shape as the average of one or more of the 3-D shapes measured by the 3-D shape measurement means;

a reflectance measurement means for measuring the reflectance at each position on the surface of the 3-D shape of the registered object; and

a data storage means for storing the 3-D shapes measured by the 3-D shape measurement means, the average 3-D shape generated by the average shape generation means and the reflectance measured by the reflectance measurement means as the registered data.

6. An image comparison device as claimed in

claim 2

, wherein the illumination correction means includes:

an image generation means for generating images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions as illumination variation images by use of the position/pose estimated by the position/pose determination means and the data registered in the registration means; and

an illumination condition estimation means for generating an image that is the most similar to the input image obtained by the photographing means by use of the illumination variation images generated by the image generation means and outputting the generated image to the image comparison means as the reference image.

7. An image comparison device as claimed in

claim 6

, wherein:

the illumination correction means further includes an illumination variation image space generation means for generating an illumination variation image space which is spanned by the illumination variation images generated by the image generation means, and

the illumination condition estimation means generates the image that is the most similar to the input image from the illumination variation image space generated by the illumination variation image space generation means and outputs the generated image to the image comparison means as the reference image.

8. An image comparison device as claimed in

claim 7

, wherein:

the illumination variation image space generation means generates basis vectors of a space that almost accommodates image variation due to the illumination variation by conducting principal component analysis (PCA) to the illumination variation images generated by the image generation means, and

the illumination condition estimation means obtains inner products between the input image obtained by the photographing means and each of the basis vectors generated by the illumination variation image space generation means, generates the image that is the most similar to the input image by use of the basis vectors and based on the inner products, and outputs the generated image to the image comparison means as the reference image.

9. An image comparison device as claimed in

claim 6

, wherein the illumination correction means further includes an illumination condition variation means for setting various illumination conditions and outputting the illumination conditions to the image generation means.

10. An image comparison device as claimed in

claim 2

, wherein:

the registration means includes;

a 3-D shape measurement means for measuring the 3-D shape of each registered object;

a texture image photographing means for photographing the registered object under various illumination conditions and thereby obtaining texture images of the registered object; and

a data storage means for storing the 3-D shapes measured by the 3-D shape measurement means and the texture images obtained by the texture image photographing means as the registered data, and

the illumination correction means includes;

an image generation means for generating images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions as illumination variation images by use of the position/pose determined by the position/pose determination means and the 3-D shapes and the texture images of the registered object registered in the registration means; and

an illumination condition estimation means for generating an image that is the most similar to the input image obtained by the photographing means by use of the illumination variation images generated by the image generation means and outputting the generated image to the image comparison means as the reference image.

11. An image comparison device as claimed in

claim 10

, wherein the illumination correction means further includes an illumination variation image space generation means for generating an illumination variation image space which is spanned by the illumination variation images generated by the image generation means, and

the illumination condition estimation means generates the image that is the most similar to the input image from the illumination variation image space generated by the illumination variation image space generation means and outputs the generated image to the image comparison means as the reference image.

12. An image comparison device as claimed in

claim 11

, wherein:

the illumination variation image space generation means generates basis vectors of a space that almost accommodates image variation due to the illumination variation by conducting the principal component analysis (PCA) to the illumination variation images generated by the image generation means, and

the illumination condition estimation means obtains inner products between the input image obtained by the photographing means and each of the basis vectors generated by the illumination variation image space generation means, generates the image that is the most similar to the input image by use of the basis vectors and based on the inner products, and outputs the generated image to the image comparison means as the reference image.

13. An image comparison device as claimed in

claim 2

, wherein the position/pose determination means outputs a predetermined position/pose to the illumination correction means.

14. An image comparison device as claimed in

claim 2

, wherein the position/pose determination means receives inputs concerning position/pose from outside, and outputs the received position/pose to the illumination correction means.

15. An image comparison device as claimed in

claim 2

, wherein the position/pose determination means estimates the position/pose of the target object in the input image obtained by the photographing means, and outputs the estimated position/pose to the illumination correction means.

16. An image comparison device as claimed in

claim 15

, wherein:

the registration means includes;

a 3-D shape measurement means for measuring the 3-D shape of each registered object;

a reflectance measurement means for measuring the reflectance at each position on the surface of the 3-D shape of the registered object;

a first feature point extraction means for extracting feature points of the registered object based on the 3-D shape measured by the 3-D shape measurement means and the reflectance measured by the reflectance measurement means and obtaining the positions of the extracted feature points; and

a data storage means for storing the 3-D shapes measured by the 3-D shape measurement means, the reflectance measured by the reflectance measurement means and the feature point positions obtained by the first feature point extraction means as the registered data, and

the position/pose determination means includes;

a second feature point extraction means for extracting feature points corresponding to those extracted by the first feature point extraction means from the input image obtained by the photographing means and obtaining the positions of the extracted feature points; and

a position/pose calculation means for estimating the position/pose of the target object in the input image based on the 3-D shape and the feature point positions stored in the data storage means and the input image feature point positions obtained by the second feature point extraction means and outputting the estimated position/pose to the illumination correction means.

17. An image comparison device as claimed in

claim 16

, wherein the illumination correction means includes:

an image generation means for generating images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions as illumination variation images by use of the position/pose estimated by the position/pose determination means and the 3-D shape and the reflectance stored in the data storage means; and

an illumination condition estimation means for generating an image that is the most similar to the input image obtained by the photographing means by use of the illumination variation images generated by the image generation means and outputting the generated image to the image comparison means as the reference image.

18. An image comparison device as claimed in

claim 17

, wherein:

the illumination correction means further includes an illumination variation image space generation means for generating an illumination variation image space which is spanned by the illumination variation images generated by the image generation means, and

the illumination condition estimation means generates the image that is the most similar to the input image from the illumination variation image space generated by the illumination variation image space generation means and outputs the generated image to the image comparison means as the reference image.

19. An image comparison device as claimed in

claim 18

, wherein:

the illumination variation image space generation means generates basis vectors of a space that almost accommodates image variation due to the illumination variation by conducting principal component analysis (PCA) to the illumination variation images generated by the image generation means, and

the illumination condition estimation means obtains inner products between the input image obtained by the photographing means and each of the basis vectors generated by the illumination variation image space generation means, generates the image that is the most similar to the input image by use of the basis vectors and based on the inner products, and outputs the generated image to the image comparison means as the reference image.

20. An image comparison device as claimed in

claim 19

, wherein the illumination correction means further includes an illumination condition variation means for setting various illumination conditions and outputting the illumination conditions to the image generation means.

21. An image comparison device as claimed in

claim 1

, wherein:

the registration means includes;

a 3-D shape measurement means for measuring the 3-D shape of each registered object;

a reflectance measurement means for measuring the reflectance at each position on the surface of the 3-D shape of the registered object;

an image generation means for generating images of each registered object under various illumination conditions as illumination variation images;

an illumination variation image space generation means for generating an illumination variation image space which is spanned by the illumination variation images generated by the image generation means; and

a data storage means for storing the illumination variation image space generated by the illumination variation image space generation means as the registered data, and

the comparison means includes;

a photographing means for photographing the target object and thereby obtaining an input image;

an illumination condition estimation means for generating an image of each registered object that is the most similar to the input image as a reference image from the illumination variation image space stored in the data storage means;

an image comparison means for comparing each reference image generated by the illumination condition estimation means with the input image obtained by the photographing means and thereby calculating an evaluation value concerning the similarity between the two images; and

a judgment means for judging whether or not each of the registered objects registered in the registration means is the same as or similar to the target object photographed by the photographing means based on the evaluation value calculated by the image comparison means.

22. An image comparison device as claimed in

claim 21

, wherein:

the illumination variation image space generation means generates basis vectors of a space that almost accommodates image variation due to the illumination variation by conducting principal component analysis (PCA) to the illumination variation images generated by the image generation means, and

the data storage means stores the basis vectors generated by the illumination variation image space generation means as the registered data, and

the illumination condition estimation means obtains inner products between the input image obtained by the photographing means and each of the basis vectors stored in the data storage means, generates the image that is the most similar to the input image by use of the basis vectors and based on the inner products, and outputs the generated image to the image comparison means as the reference image.

23. An image comparison device as claimed in

claim 3

, wherein

the 3-D shape measurement means obtains the 3-D shapes by reading data or drawings.

24. An image comparison device as claimed in

claim 4

, wherein

the 3-D shape measurement means obtains the 3-D shapes by reading data or drawings.

25. An image comparison device as claimed in

claim 10

, wherein

the 3-D shape measurement means obtains the 3-D shapes by reading data or drawings.

26. An image comparison device as claimed in

claim 3

, wherein

the reflectance measurement means obtains the reflectance by reading data or drawings.

27. An image comparison device as claimed in

claim 2

, wherein

the photographing means obtains the input image by scanning a film, a photograph or printed matter.

28. An image comparison device as claimed in

claim 2

, wherein the judgment means judges which registered object matches the target object.

29. An image comparison device as claimed in

claim 2

, wherein the judgment means searches for one or more registered objects that are similar to the target object.

30. An image comparison device as claimed in

claim 1

, wherein the registered objects are automobiles.

31. An image comparison device as claimed in

claim 1

, wherein the registered objects are human faces.

32. An image comparison method comprising the steps of:

a registration step in which 3-D data of all or part of one or more registered objects are obtained and registered; and

a comparison step in which 2-D data of a target object is obtained and the obtained 2-D data is compared with the data registered in the registration step for conducting judgment concerning the similarity/sameness between the target object and each of the one or more registered objects.

33. An image comparison method as claimed in

claim 32

, wherein the comparison step includes:

a photographing step in which the target object is photographed and thereby an input image is obtained;

a position/pose determination step in which the position/pose of the target object in the input image obtained in the photographing step is determined;

an illumination correction step in which an image of each registered object in the same position/pose as the target object in the input image and under an illumination condition most similar to that of the input image is obtained as a reference image by use of the position/pose determined in the position/pose determination step and the data registered in the registration step;

an image comparison step in which each reference image generated in the illumination correction step is compared with the input image obtained in the photographing step and thereby an evaluation value concerning the similarity between the two images is calculated; and

a judgment step in which it is judged whether or not each of the registered objects registered in the registration step is the same as or similar to the target object photographed in the photographing step based on the evaluation value calculated in the image comparison step.

34. An image comparison method as claimed in

claim 33

,

wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the reflectance measured in the reflectance measurement step are stored as the registered data.

35. An image comparison method as claimed in

claim 33

, wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shapes of all or part of the one or more registered objects are measured;

a color/intensity measurement step in which the registered object is photographed and thereby information concerning color or intensity of the registered object is obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the color/intensity information obtained in the color/intensity measurement step are stored as the registered data.

36. An image comparison method as claimed in

claim 33

, wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shapes of all or part of the one or more registered objects are measured;

an average shape generation step in which an average 3-D shape is generated as the average of one or more of the 3-D shapes measured in the 3-D shape measurement step;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step, the average 3-D shape generated in the average shape generation step and the reflectance measured in the reflectance measurement step are stored as the registered data.

37. An image comparison method as claimed in

claim 33

, wherein the illumination correction step includes:

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose estimated in the position/pose determination step and the data registered in the registration step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

38. An image comparison method as claimed in

claim 37

, wherein:

the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

39. An image comparison method as claimed in

claim 38

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

40. An image comparison method as claimed in

claim 37

, wherein the illumination correction step further includes an illumination condition variation step in which various illumination conditions are set and generated to be used in the image generation step.

41. An image comparison method as claimed in

claim 33

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a texture image photographing step in which the registered object is photographed under various illumination conditions and thereby texture images of the registered object are obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the texture images obtained in the texture image photographing step are stored as the registered data, and

the illumination correction step includes;

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose determined in the position/pose determination step and the 3-D shapes and the texture images of the registered object registered in the registration step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

42. An image comparison method as claimed in

claim 41

, wherein the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

43. An image comparison method as claimed in

claim 42

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting the principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

44. An image comparison method as claimed in

claim 33

, wherein in the position/pose determination step, a predetermined position/pose is adopted as the position/pose of the target object in the input image to be used in the illumination correction step.

45. An image comparison method as claimed in

claim 33

, wherein in the position/pose determination step, a position/pose inputted from outside is adopted as the position/pose of the target object in the input image to be used in the illumination correction step.

46. An image comparison method as claimed in

claim 33

, wherein in the position/pose determination step, the position/pose of the target object in the input image obtained in the photographing step is estimated to be used in the illumination correction step.

47. An image comparison method as claimed in

claim 46

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured;

a first feature point extraction step in which feature points of the registered object are extracted based on the 3-D shape measured in the 3-D shape measurement step and the reflectance measured in the reflectance measurement step and the positions of the extracted feature points are obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step, the reflectance measured in the reflectance measurement step and the feature point positions obtained in the first feature point extraction step are stored as the registered data, and

the position/pose determination step includes;

a second feature point extraction step in which feature points corresponding to those extracted in the first feature point extraction step are extracted from the input image obtained in the photographing step and the positions of the extracted feature points are obtained; and

a position/pose calculation step in which the position/pose of the target object in the input image is estimated based on the 3-D shape and the feature point positions stored in the data storage step and the input image feature point positions obtained in the second feature point extraction step.

48. An image comparison method as claimed in

claim 47

, wherein the illumination correction step includes:

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose estimated in the position/pose determination step and the 3-D shape and the reflectance stored in the data storage step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

49. An image comparison method as claimed in

claim 48

, wherein:

the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

50. An image comparison method as claimed in

claim 49

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

51. An image comparison method as claimed in

claim 50

, wherein the illumination correction step further includes an illumination condition variation step in which various illumination conditions are set and generated to be used in the image generation step.

52. An image comparison method as claimed in

claim 32

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured;

an image generation step in which images of each registered object under various illumination conditions are generated as illumination variation images;

an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated; and

a data storage step in which the illumination variation image space generated in the illumination variation image space generation step is stored as the registered data, and

the comparison step includes;

a photographing step in which the target object is photographed and thereby an input image is obtained;

an illumination condition estimation step in which an image of each registered object that is the most similar to the input image is generated as a reference image from the illumination variation image space stored in the data storage step;

an image comparison step in which each reference image generated in the illumination condition estimation step is compared with the input image obtained in the photographing step and thereby an evaluation value concerning the similarity between the two images is calculated; and

a judgment step in which it is judged whether or not each of the registered objects registered in the registration step is the same as or similar to the target object photographed in the photographing step based on the evaluation value calculated in the image comparison step.

53. An image comparison method as claimed in

claim 52

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the data storage step, the basis vectors generated in the illumination variation image space generation step are stored as the registered data, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors stored in the data storage step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

54. An image comparison method as claimed in

claim 34

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

55. An image comparison method as claimed in

claim 35

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

56. An image comparison method as claimed in

claim 41

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

57. An image comparison method as claimed in

claim 34

, wherein in the reflectance measurement step, the reflectance is obtained by reading data or drawings.

58. An image comparison method as claimed in

claim 33

, wherein in the photographing step, the input image is obtained by scanning a film, a photograph or printed matter.

59. An image comparison method as claimed in

claim 33

, wherein the judgment step is executed for judging which registered object matches the target object.

60. An image comparison method as claimed in

claim 33

, wherein the judgment step is executed for searching for one or more registered objects that are similar to the target object.

61. An image comparison method as claimed in

claim 32

, wherein the registered objects are automobiles.

62. An image comparison method as claimed in

claim 32

, wherein the registered objects are human faces.

63. A machine-readable record medium storing one or more programs for instructing one or more computers, devices, MPUs (Microprocessor Units), etc. to execute an image comparison process comprising the steps of:

a registration step in which 3-D data of all or part of one or more registered objects are obtained and registered; and

a comparison step in which 2-D data of a target object is obtained and the obtained 2-D data is compared with the data registered in the registration step for conducting judgment concerning the similarity/sameness between the target object and each of the one or more registered objects.

64. A machine-readable record medium as claimed in

claim 63

, wherein the comparison step includes:

a photographing step in which the target object is photographed and thereby an input image is obtained;

a position/pose determination step in which the position/pose of the target object in the input image obtained in the photographing step is determined;

an illumination correction step in which an image of each registered object in the same position/pose as the target object in the input image and under an illumination condition most similar to that of the input image is obtained as a reference image by use of the position/pose determined in the position/pose determination step and the data registered in the registration step;

an image comparison step in which each reference image generated in the illumination correction step is compared with the input image obtained in the photographing step and thereby an evaluation value concerning the similarity between the two images is calculated; and

a judgment step in which it is judged whether or not each of the registered objects registered in the registration step is the same as or similar to the target object photographed in the photographing step based on the evaluation value calculated in the image comparison step.

65. A machine-readable record medium as claimed in

claim 64

, wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the reflectance measured in the reflectance measurement step are stored as the registered data.

66. A machine-readable record medium as claimed in

claim 64

, wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shapes of all or part of the one or more registered objects are measured;

a color/intensity measurement step in which the registered object is photographed and thereby information concerning color or intensity of the registered object is obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the color/intensity information obtained in the color/intensity measurement step are stored as the registered data.

67. A machine-readable record medium as claimed in

claim 64

, wherein the registration step includes:

a 3-D shape measurement step in which the 3-D shapes of all or part of the one or more registered objects are measured;

an average shape generation step in which an average 3-D shape is generated as the average of one or more of the 3-D shapes measured in the 3-D shape measurement step;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step, the average 3-D shape generated in the average shape generation step and the reflectance measured in the reflectance measurement step are stored as the registered data.

68. A machine-readable record medium as claimed in

claim 64

, wherein the illumination correction step includes:

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose estimated in the position/pose determination step and the data registered in the registration step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

69. A machine-readable record medium as claimed in

claim 68

, wherein:

the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

70. A machine-readable record medium as claimed in

claim 69

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

71. A machine-readable record medium as claimed in

claim 68

, wherein the illumination correction step further includes an illumination condition variation step in which various illumination conditions are set and generated to be used in the image generation step.

72. A machine-readable record medium as claimed in

claim 64

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a texture image photographing step in which the registered object is photographed under various illumination conditions and thereby texture images of the registered object are obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step and the texture images obtained in the texture image photographing step are stored as the registered data, and

the illumination correction step includes;

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose determined in the position/pose determination step and the 3-D shapes and the texture images of the registered object registered in the registration step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

73. A machine-readable record medium as claimed in

claim 72

, wherein the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

74. A machine-readable record medium as claimed in

claim 73

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting the principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

75. A machine-readable record medium as claimed in

claim 64

, wherein in the position/pose determination step, a predetermined position/pose is adopted as the position/pose of the target object in the input image to be used in the illumination correction step.

76. A machine-readable record medium as claimed in

claim 64

, wherein in the position/pose determination step, a position/pose inputted from outside is adopted as the position/pose of the target object in the input image to be used in the illumination correction step.

77. A machine-readable record medium as claimed in

claim 64

, wherein in the position/pose determination step, the position/pose of the target object in the input image obtained in the photographing step is estimated to be used in the illumination correction step.

78. A machine-readable record medium as claimed in

claim 77

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured;

a first feature point extraction step in which feature points of the registered object are extracted based on the 3-D shape measured in the 3-D shape measurement step and the reflectance measured in the reflectance measurement step and the positions of the extracted feature points are obtained; and

a data storage step in which the 3-D shapes measured in the 3-D shape measurement step, the reflectance measured in the reflectance measurement step and the feature point positions obtained in the first feature point extraction step are stored as the registered data, and

the position/pose determination step includes;

a second feature point extraction step in which feature points corresponding to those extracted in the first feature point extraction step are extracted from the input image obtained in the photographing step and the positions of the extracted feature points are obtained; and

a position/pose calculation step in which the position/pose of the target object in the input image is estimated based on the 3-D shape and the feature point positions stored in the data storage step and the input image feature point positions obtained in the second feature point extraction step.

79. A machine-readable record medium as claimed in

claim 78

, wherein the illumination correction step includes:

an image generation step in which images of each registered object in the same position/pose as the target object in the input image and under various illumination conditions are generated as illumination variation images by use of the position/pose estimated in the position/pose determination step and the 3-D shape and the reflectance stored in the data storage step; and

an illumination condition estimation step in which an image that is the most similar to the input image obtained in the photographing step is generated as the reference image by use of the illumination variation images generated in the image generation step.

80. A machine-readable record medium as claimed in

claim 79

, wherein:

the illumination correction step further includes an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated, and

in the illumination condition estimation step, the image that is the most similar to the input image is generated as the reference image from the illumination variation image space generated in the illumination variation image space generation step.

81. A machine-readable record medium as claimed in

claim 80

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors generated in the illumination variation image space generation step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

82. A machine-readable record medium as claimed in

claim 81

, wherein the illumination correction step further includes an illumination condition variation step in which various illumination conditions are set and generated to be used in the image generation step.

83. A machine-readable record medium as claimed in

claim 63

, wherein:

the registration step includes;

a 3-D shape measurement step in which the 3-D shape of each registered object is measured;

a reflectance measurement step in which the reflectance at each position on the surface of the 3-D shape of the registered object is measured;

an image generation step in which images of each registered object under various illumination conditions are generated as illumination variation images;

an illumination variation image space generation step in which an illumination variation image space which is spanned by the illumination variation images generated in the image generation step is generated; and

a data storage step in which the illumination variation image space generated in the illumination variation image space generation step is stored as the registered data, and

the comparison step includes;

a photographing step in which the target object is photographed and thereby an input image is obtained;

an illumination condition estimation step in which an image of each registered object that is the most similar to the input image is generated as a reference image from the illumination variation image space stored in the data storage step;

an image comparison step in which each reference image generated in the illumination condition estimation step is compared with the input image obtained in the photographing step and thereby an evaluation value concerning the similarity between the two images is calculated; and

a judgment step in which it is judged whether or not each of the registered objects registered in the registration step is the same as or similar to the target object photographed in the photographing step based on the evaluation value calculated in the image comparison step.

84. A machine-readable record medium as claimed in

claim 83

, wherein:

in the illumination variation image space generation step, basis vectors of a space that almost accommodates image variation due to the illumination variation are generated by conducting principal component analysis (PCA) to the illumination variation images generated in the image generation step, and

in the data storage step, the basis vectors generated in the illumination variation image space generation step are stored as the registered data, and

in the illumination condition estimation step, inner products are obtained between the input image obtained in the photographing step and each of the basis vectors stored in the data storage step, and the image that is the most similar to the input image is generated as the reference image by use of the basis vectors and based on the inner products.

85. A machine-readable record medium as claimed in

claim 65

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

86. A machine-readable record medium as claimed in

claim 66

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

87. A machine-readable record medium as claimed in

claim 72

, wherein in the 3-D shape measurement step, the 3-D shapes are obtained by reading data or drawings.

88. A machine-readable record medium as claimed in

claim 65

, wherein in the reflectance measurement step, the reflectance is obtained by reading data or drawings.

89. A machine-readable record medium as claimed in

claim 64

, wherein in the photographing step, the input image is obtained by scanning a film, a photograph or printed matter.

90. A machine-readable record medium as claimed in

claim 64

, wherein the judgment step is executed for judging which registered object matches the target object.

91. A machine-readable record medium as claimed in

claim 64

, wherein the judgment step is executed for searching for one or more registered objects that are similar to the target object.

92. A machine-readable record medium as claimed in

claim 63

, wherein the registered objects are automobiles.

93. A machine-readable record medium as claimed in

claim 63

, wherein the registered objects are human faces.