Method and apparatus for collating object

• US 20020097906A1
• Filed: 11/13/2001
• Published: 07/25/2002
• Est. Priority Date: 11/20/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ishiyama, Rui
• Granted Patent: US 7,321,370 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An object collation method comprising a registration procedure for registering the registered data of a registered object in a database, and a collation procedure for collating the input image of a target object with the registered data. The registration procedure includes a step of storing the three-dimensional shape of the registered object and a texture space defined by a texture group indicating the luminance and/or color information of each position of the object surface under various illumination conditions. The collation procedure includes the steps of: generating an illumination fluctuation space defined by the image group under the various illumination conditions, at the location and position of the target object in the input image from the three-dimensional shape and the texture space; and collating the target object and the registered object based on the distance between the illumination fluctuation space and the input image.

Claims

1. An object collation method comprising:

a registration procedure for registering registered data of a registered object in a database; and

a collation procedure for collating an input image of an target object with said registered data;

said registration procedure comprising the steps of;

storing a three-dimensional shape of the registered object; and

storing a texture space, which is an image space, defined by a texture group indicating luminance and/or color information of each position of a surface of the registered object under various illumination conditions;

said collation procedure comprising the steps of;

taking a two-dimensional image of the target object as the input image;

generating an illumination fluctuation space, which is an image space, defined by a image group under various illumination conditions at the location and position of the target object in said input image from said three-dimensional shape and said texture space; and

collating the target object and the registered object based on distance between said illumination fluctuation space and said input image.

2. An object collation method comprising:

a registration procedure for registering registered data of a registered object in a database; and

a collation procedure for collating an input image of a target object with said registered data;

said registration procedure comprising the steps of;

inputting a three-dimensional shape of the registered object and reflectivity of a surface of the registered object;

acquiring a texture group indicating luminance and color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity;

generating a texture space, which is an image space, defined by said texture group; and

storing said three-dimensional shape and said texture space; and

said collation procedure comprising the steps of;

taking a two-dimensional image of the target object as an input image;

presuming location and position of the target object in said input image;

generating an illumination fluctuation space, which is defined by an image group under the various illumination conditions, at same location and position as said presumed location and position, by using said three-dimensional shape and said texture space, said fluctuation space being an image space;

calculating distance between said input image and said illumination fluctuation space; and

collating the target object and the registered object based on said distance.

3. A method according to claim 2, wherein said texture is generated by computer graphics technique in said acquiring step.

4. A method according to claim 2,

wherein said inputting step comprises the step of measuring the three-dimensional shape of the registered object, and

wherein said step of generating the texture step comprises the step of photographing the luminance and color information of each location of the surface of the registered object practically by setting various illumination conditions, to generate the texture group.

7. A method according to claim 2,

wherein said step of generating the texture space comprises the step of analyzing said texture group statistically to determine a basic texture group of basic vectors of a space covering most of fluctuation factors appearing on the texture in accordance with the illumination conditions,

wherein said storing step comprises the step of storing said basic texture group as the texture group,

wherein said step of generating illumination fluctuation space comprises the step of determining basic vector groups of the illumination fluctuation space by using said three-dimensional shape, said basic texture group and said presumed location and position, and

wherein said calculating step comprises the steps of determining a correlation between said basic vector group and said input image;

generating a comparison image which is closest to said input image from the basic vector group based on said correlation; and

outputting similarity between said comparison image and said input image as the distance.

9. A method according to claim 2, wherein, an average shape which is an average of the three-dimensional shape of said registered object is outputted in said registration procedure, and said average shape is used as the three-dimensional shape of said registered object in said collation procedure.

11. A method according to claim 2, wherein said inputting step comprising the step of extracting said three-dimensional shape and/or said reflectivity from existing drawings and/or existing CAD data.

13. A method according to claim 2, wherein location information of a feature portion of the registered object is added to said registered data and is stored.

15. A method according to claim 2, wherein an object to be collated is an automobile.

17. A method according to claim 2, wherein an object to be collated is a human face.

5. An object collation method comprising:

a registration procedure for registering registered data of a registered object in a database; and

a collation procedure for collating an input image of a target object with said registered data, said registration procedure comprising the steps of;

inputting a three-dimensional shape of the registered object and reflectivity of a surface of the registered object;

acquiring a texture group indicating luminance and color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity;

generating a texture space, which id an image space, defined by said texture group; and

storing said three-dimensional shape and said texture space, and

said collation procedure comprising the steps of;

taking a two-dimensional image of the target object as an input image;

presuming location and position of the target object in said input image;

converting the input image into an input texture by using said three-dimensional shape and said presumed location and position;

calculating distance between said texture space and said texture obtained at said converting step; and

collating the target object and the registered object based on said distance.

6. A method according to claim 5,

wherein said inputting step comprises the step of measuring the three-dimensional shape of the registered object, and

wherein said generating step comprises the step of photographing the luminance and color information of each location of the surface of the registered object practically by setting various illumination conditions, to generate the texture group.

8. A method according to claim 5,

wherein said generating step comprises the step of analyzing said texture group statistically to determine a basic texture group of basic vectors of a space covering most of fluctuation factors appearing on the texture in accordance with the illumination conditions,

wherein said storing step comprises the step of storing said basic texture group as the texture group,

wherein said calculating step comprises the steps of determining a correlation between said basic texture group and said input image;

generating a comparison texture which is closest to said input texture from the basic texture group based on said correlation; and

outputting similarity between said comparison texture and said input texture as the distance.

10. A method according to claim 5, wherein, an average shape which is an average of the three-dimensional shape of said registered object is outputted in said registration procedure, and said average shape is used as the three-dimensional shape of said registered object in said collation procedure.

12. A method according to claim 5, wherein said inputting step comprising the step of extracting said three-dimensional shape and/or said reflectivity from existing drawings and/or existing CAD data.

14. A method according to claim 5, wherein location information of a feature portion of the registered object is added to said registered data and is stored.

16. A method according to claim 5, wherein an object to be collated is an automobile.

18. A method according to claim 5, wherein an object to be collated is a human face.

19. An object collation apparatus comprising:

registration means for registering a three-dimensional shape of a registered object and a texture which is defined by a texture group indicating luminance and color information of each location of the object surface under various illumination conditions, said texture space being an image space; and

collation means for taking a two-dimensional image of a target object as an input image;

for generating an illumination fluctuation space, which is defined by the image group under the various illumination conditions, at location and position of the target object in said input image from said three-dimensional shape and said texture space; and

for collating the target object and the registered object based on distance between said illumination fluctuation space and said input image, said illumination fluctuation space being an image space.

20. An object collation apparatus comprising:

registration means for registering registered data of a registered object; and

collation means for collating an input image of a target object with said registered data,

said registration means comprising;

means for inputting a three-dimensional shape of the registered object and reflectivity of a surface of the registered object;

means for acquiring a texture group indicating luminance and the color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity; and

texture space generation means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape and said texture space, and

said collation means comprising;

means for taking a two-dimensional image of the target object as an input image;

means for presuming location and position of the target object in said input image;

illumination fluctuation space generation means for generating an illumination fluctuation space, which is defined by the image group under the various illumination conditions, at same location and position as said presumed location and position, by using said three-dimensional shape and said texture space, said illumination fluctuation space being an image space;

means for calculating distance between said input image and said illumination fluctuation space; and

means for collating the target object and the registered object based on said distance.

21. An apparatus according to claim 20, wherein said texture group acquisition means comprises means for generates said texture by computer graphics technique.

22. An apparatus according to claim 20,

wherein said inputting means comprises means for measuring the three-dimensional shape of the registered object, and

wherein said texture group generation means comprises means for photographing the luminance and color information of each location of the surface of the registered object practically by setting various illumination conditions, to generate the texture group.

24. An apparatus according to claim 23,

wherein said inputting means comprises means for measuring the three-dimensional shape of the registered object, and

wherein said texture group generation means comprises means for photographing the luminance and color information of each location of the surface of the registered object practically by setting various illumination conditions, to generate the texture group.

25. An apparatus according to claim 20,

wherein said texture space generation means comprises means for generating the texture space by analyzing said texture group statistically to determined a basic texture group of basic vectors of a space covering most of fluctuation factors appearing on the texture in accordance with the illumination conditions,

wherein storing means comprises means for storing said three-dimensional shape and said basic texture group,

wherein said illumination fluctuation space generation means comprises means for determining the basic vector groups of the illumination fluctuation space by using said three-dimensional shape, said basic texture group and said presumed location and position, and

wherein calculating means comprises means for determining a correlation between said basic vector group and said input image for generating a comparison image which is closest to said input image from the basic vector group based on said correlation, and for outputting similarity between said comparison image and said input image as the distance.

26. An apparatus according to claim 23,

wherein said generating means comprises means for generating the texture space by analyzing said texture group statistically to determine a basic texture group of basic vectors of a space covering most of fluctuation factors appearing on the texture in accordance with the illumination conditions,

wherein said storing means comprises means for storing said three-dimensional shape and said basic texture group, and

wherein said calculating means comprises means for determining a correlation between said basic texture group and said input texture, for generating a comparison texture which is closest to said input texture from the basic texture group based on said correlation, and for outputting similarity between said comparison texture and said input texture as the distance.

27. An apparatus according to claim 20, wherein said registration means outputs, as an average shape, an average of the three-dimensional shape of said registered object, and said collation means uses said average shape as the three-dimensional shape of said registered object.

28. An apparatus according to claim 23, wherein said registration means outputs, as an average shape, an average of the three-dimensional shape of said registered object, and said collation means uses said average shape as the three-dimensional shape of said registered object.

29. An apparatus according to claim 20, wherein said inputting means comprises means for extracting said three-dimensional shape and/or said reflectivity from existing drawings and/or existing CAD data.

30. An apparatus according to claim 23, wherein said inputting means comprises means for extracting said three-dimensional shape and/or said reflectivity from existing drawings and/or existing CAD data.

31. An apparatus according to claim 20, wherein said registration means further comprises means for adding and registering the location information of a feature portion of the registered object.

32. An apparatus according to claim 23, wherein said registration means further comprises means for adding and registering location information of a feature portion of the registered object.

23. An object collation apparatus comprising:

registration means for registering registered data of a registered object in a database; and

collation means for collating the input image of a target object with said registered data,

registration means comprising;

means for inputting a three-dimensional shape of the registered object and reflectivity of the surface of the registered object;

means for acquiring a texture group indicating luminance and the color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity;

means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape and said texture space, and

said collation means comprising;

means for taking a two-dimensional image of the target object as an input image;

means for presuming location and position of the target object in said input image;

means for converting the input image into an input texture by using said three-dimensional shape and said presumed location and position;

means for calculating distance between said input texture and said texture space; and

means for collating the target object and the registered object based on said distance.

33. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for measuring a three-dimensional shape of a registered object and reflectivity of a surface of the registered object;

means for generating a texture group indicating luminance and color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity;

means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape and said texture space.

34. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for taking a two-dimensional image of a target object as an input image;

means for presuming location and position of the target object in said input image;

means for generating an illumination fluctuation space, which is defined by an image group under various illumination conditions, at same location and position as said presumed location and position, by using said three-dimensional shape of the registered objects and the texture space, said illumination fluctuation space being an image space;

means for calculating distance between said input image and said illumination fluctuation space; and

means for collating the target object and the registered object based on said distance.

35. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for measuring a three-dimensional shape of a registered object;

means for taking an image group of the registered object by setting illumination conditions practically and for generating a texture coordinate system by using said three-dimensional shape, thereby to output a texture group by converting said image group into said texture coordinate system;

means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape and said texture space.

36. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for taking a two-dimensional image of a target object as an input image;

means for presuming location and position of the target object in said input image;

means for converting and deforming each pixel of said input image into a texture coordinate by using the result of said presumption thereby to generate an input texture;

means for calculating distance between the texture space of the registered object and said input texture; and

means for collating the target object and the registered object based on said distance.

37. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for measuring a three-dimensional shape of a registered object;

means for translating to make three-dimensional shapes of a plurality of registered objects to align their centers of gravity and for setting sections normal to an axis at a proper interval to calculate the average shapes in the individual sections; and

means for generating a texture space, which is an image space, defined by a texture group; and

means for storing said average shapes and said texture space.

38. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for measuring a three-dimensional shape of a registered object and reflectivity of a surface of a registered object;

means for detecting location of a feature point and outputting a three-dimensional coordinate as the feature point location;

means for generating a texture group indicating luminance and color information of each location of the surface of the registered object under various illumination conditions by using said three-dimensional shape and said reflectivity;

means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape, said texture space and said feature point location.

39. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for taking a two-dimensional image of a target object as an input image;

means for presuming location and position of the target object and the parameters of a photography device, which are photographic conditions at a time of taking said input image, by using the three-dimensional shape of the registered object, the texture space and the feature point location;

means for generating an illumination fluctuation space, which is defined by the image group under the various illumination conditions, at same location and position as said presumed location and position, by using said three-dimensional shape of the registered objects and the texture space, said illumination fluctuation space being an image space;

means for calculating distance between said input image and said illumination fluctuation space; and

means for collating the target object and the registered object based on said distance.

40. A computer-readable recording medium comprising programs recorded therein for causing a computer to function as:

means for inputting a three-dimensional shape of a registered object and reflectivity;

means for generating a texture group indicating luminance and color information of each location of a surface of the registered object surface under various illumination conditions by using said three-dimensional shape and said reflectivity;

means for generating a texture space, which is an image space, defined by said texture group; and

means for storing said three-dimensional shape and said texture space.