Face meta-data creation and face similarity calculation

• US 20040197013A1
• Filed: 02/26/2004
• Published: 10/07/2004
• Est. Priority Date: 12/14/2001
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kamei, Toshio
• Granted Patent: US 7,593,551 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A face feature is extracted by a face feature extracting unit, a confidence index is extracted by a confidence index extracting unit, and then, they are outputted as a face meta-data. At a time of matching, by using the confidence index of the face meta-data, a distribution estimating unit estimates the data of a parameter and the like with regard to a posterior distribution when the confidence index is obtained. A distance calculating unit calculates a similarity between feature values. Consequently, the precision of face verification can be improved, thereby carrying out a practical face matching.

Claims

1. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a confidence index indicative of a confidence to said face feature, from said face image,

wherein said face feature and said confidence index are generated as a face meta-data.

11. The face meta-data generating apparatus according to claim 1, wherein, said face feature extracting unit extracts said face feature, by obtaining a pattern vector Λ

from said face image, and calculating said face feature using a basis matrix U specified by a subset of basis vectors of said pattern vector Λ and

a mean vector of said pattern vector Λ

.

12. The face meta-data generating apparatus according to claim 11, wherein said basis matrix U is determined by said partial basis vectors that is selected from basis vectors obtained by the principal component analysis of a training set of pattern vectors.

14. The face meta-data generating apparatus according to claim 11, wherein said pattern vector Λ

has pixels values in said face image as its elements.

15. The face meta-data generating apparatus according to claim 11, wherein said pattern vector Λ

Fourier components of said face image.

2. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a contrast index of said face image, as a confidence index indicative of a confidence to said face feature,

wherein said face feature and said confidence index are generated as a face meta-data.

3. The face meta-data generating apparatus according to claim 2, wherein said confidence index extracting unit extracts a difference between the maximum value and the minimum value of pixel values in said face image, as said contrast index of said face image.

4. The face meta-data generating apparatus according to claim 2, wherein said confidence index extracting unit extracts one of a standard deviation and a variance of the pixel values in said face image, as said contrast index of said face image.

5. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts an asymmetric of said face image, as a confidence index indicative of a confidence to said face feature,

wherein said face feature and said confidence index are generated as a face meta-data.

6. The face meta-data generating apparatus according to claim 5, wherein said confidence index extracting unit extracts the maximum value of differences between said face image and a flipped image obtained by flipping said face image, as the asymmetric index of said face image.

7. The face meta-data generating apparatus according to claim 5, wherein said confidence index extracting unit extracts one of the sum and the average of powers of pixel values of difference images between said face image and a flipped image obtained by flipping said face image, as the asymmetric index of said face image.

8. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a confidence index indicative of a confidence to said face feature, from said face image,

wherein said face feature and said confidence index are generated as a face meta-data,

said confidence index extracting unit comprises;

a first confidence index extracting unit which extracts a contrast index of said face image; and

a second confidence index extracting unit which extracts an asymmetric index of said face image.

9. The face meta-data generating apparatus according to claim 8, wherein said first confidence index extracting unit extracts a difference between the maximum value and the minimum value of pixel values in said face image, as the contrast index of said face image, and

said second confidence index extracting unit extracts the maximum value of differences between said face image and a flipped image obtained by flipping said face image, as the asymmetric index of said face image.

10. The face meta-data generating apparatus according to claim 8, wherein said first confidence index extracting unit extracts one of a standard deviation and a variance of the pixel values in said face image, as the contrast index of said face image, and

said second confidence index extracting unit extracts one of the sum and the average of powers of pixel values of difference images between said face image and a flipped image obtained by flipping said face image, as the asymmetric index of said face image.

13. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a confidence index indicative of a confidence to said face feature, from said face image,

wherein said face feature and said confidence index are generated as a face meta-data,

said face feature extracting unit, extracts said face feature by obtaining a pattern vector Λ

from said face image, and calculating said face feature using a basis matrix U and a mean vector of said feature vector Λ

,

said basis matrix U is obtained by the linear combination of;

a basis matrix A specified by a subset of basis vectors those are selected from basis vectors obtained by the principal component analysis of a training set; and

a basis matrix B specified by a flipped subset of basis vectors in which elements of said basis vectors are exchanged so as to correspond to a conversion of pixel values where a left side and a right side of said face image is flipped.

16. (Cancelled).

17. A face similarity calculating apparatus which calculates a similarity of a face on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to said face feature comprising:

a distribution estimating unit which estimates a parameter data with regard to a posterior distribution of a difference vector between said face features, by using said confidence index to said face feature; and

a distance calculating unit which calculates, as said similarity, a distance between said face features obtained from a likelihood of said posterior distribution by using said parameter data.

18. The face similarity calculating apparatus according to claim 17, wherein said distance calculating unit, when said posterior distribution of the difference vector between said face features is a normal distribution, calculates, as said similarity, an adaptive Mahalanobis distance obtained from said likelihood of the normal distribution in said confidence index.

19. The face similarity calculating apparatus according to claim 18, wherein said distribution estimating unit estimates a variance of each element of said difference vector to said confidence index, and

said distance calculating unit calculates said adaptive Mahalanobis distance by using the variance of said each element.

20. The face similarity calculating apparatus according to claim 19, wherein said distribution estimating unit includes a variance value table which stores in advance the variance of each element of said difference vector to said confidence index, and reads out a variance required to calculate said adaptive Mahalanobis distance by referring to said variance value table on the basis of said confidence index, and

said distance calculating unit calculates said adaptive Mahalanobis distance by using the variance of said each element.

21. The face similarity calculating apparatus according to claim 17, which calculates as said similarity said adaptive mixture Mahalanobis distance that is obtained from a likelihood of said mixture distribution in said confidence index, when said posterior distribution of a difference vector between said face features is a mixture distribution.

22. A face similarity calculating apparatus which calculates a similarity of a face on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to said face feature, which, by using said confidence index to said face feature, and estimating a parameter data with regard to a within-class distribution that is said posterior distribution of the difference vectors between said face features in a class which should be deemed to be coincident, and an between-class distribution that is a posterior distribution of the difference vectors between said face features in a class which should be deemed to be inconsistent, calculates, as said similarity, a distance between said face features obtained from a likelihood of a ratio between said within-class distribution and said between-class distribution based on said parameter data.

23. The face similarity calculating apparatus according to claim 22, which calculates as said similarity said adaptive discriminant distance obtained from a likelihood of a ratio between respective distributions in said confidence index, when said within-class distribution and said between-class distribution are respectively the normal distributions.

24. The face similarity calculating apparatus according to claim 23, which estimates a within-class variance and a between-class variance of each element of said difference vector to said confidence index, and calculates as said similarity said adaptive discriminant distance on the basis of said estimated variance.

25. The face similarity calculating apparatus according to claim 24, which the first variance value table for storing in advance said within-class variance and the second variance value table for storing in advance said between-class variance in order to estimate said within-class variance and said between-class variance of each element of the difference vector to said confidence index,

refers to said first and second variance value tables, respectively, on the basis of said confidence index estimates the variance, and calculates as said similarity said adaptive discriminant distance on the basis of said estimated variance.

26. The face similarity calculating apparatus according to claim 22, which calculates as said similarity said adaptive mixture discriminant distance obtained from a likelihood of a ratio between the respective mixture distributions in said confidence index by supposing that said within-class distribution and said between-class distribution are mixture distributions, respectively.

27. A face recognizing system comprising:

an image input unit which inputs the first face image;

a face meta-data generating unit which extracts a face feature from said first face image, extracts one of a contrast index and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said first face image, and generates said face feature and said confidence index as the first face meta-data;

a face meta-data storing unit which stores the second face meta-data;

a face similarity calculating unit which calculates a face similarity between said first face meta-data and each of said the second face meta-data; and

a control unit which controls said image input unit, said face meta-data generating unit and said face similarity calculating unit, in response to an input command.

28. A face recognizing method comprising:

generating one of a contrast index and an asymmetric index of a face image as a face feature extracted from a face image and a confidence index indicative of a confidence to said face feature, as the first face meta-data;

reading out each second face meta-data;

calculating a face similarity between each of said second face meta-data and said first face meta-data, on the basis of said confidence index and said face feature of each of said second face meta-data and said first face meta-data; and

a step of outputting a matching result of said face image in accordance with said face similarity.

29. (Cancelled)

30. A face meta-data generating method comprising:

(a) extracting a face feature from a face image;

(b) extracting a contrast index of said face image as a confidence index indicative of a confidence to said face feature from said face image; and

(c) generating said face feature and said confidence index as a face meta-data.

32. The face meta-data generating method according to claim 30, wherein said step (b) includes:

extracting one of a standard deviation and a variance of pixel values in said face image, as the contrast index of said face image.

31. A face meta-data generating method comprising:

(a) extracting a face feature from a face image;

(b) extracting an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said face image; and

(c) generating said face feature and said confidence index as a face meta-data.

33. A face meta-data generating method comprising:

(a) extracting a face feature from a face image;

(b) extracting an asymmetric index of said face image as confidence index indicative of a confidence to said face feature from said face image; and

(c) generating said face feature and said confidence index as a face meta-data.

34. The face meta-data generating method according to claim 33, wherein said step (b) includes:

extracting the maximum value of differences between said face image and a flipped image in which a left side and a right side of said face image is flipped, as the asymmetric index of said face image.

35. The face meta-data generating method according to claim 33, wherein said step (b) includes:

extracting one of the sum and the average of powers of pixel values of difference images between said face image and a flipped image, as the asymmetric index of said face image.

36. A face meta-data generating method comprising:

(a) extracting a face feature from a face image;

(b) extracting a contrast index of said face image and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said face image; and

(c) generating said face feature and said confidence index as a face meta-data.

37. The face meta-data generating method according to claim 36, wherein said step (b) includes:

extracting a difference between the maximum value and the minimum value of pixel values in said face image, as the contrast index of said face image; and

extracting the maximum value of differences between said face image and said flipped image as the asymmetric index of said face image.

38. The face meta-data generating method according to claim 36, wherein said step (b) includes:

extracting one of a standard deviation and a variance of the pixel values in said face image, as the contrast index of said face image, and

extracting one of the sum and the average of powers of pixel values of difference images between said face image and said flipped image, as the asymmetric index of said face image.

39. (Cancelled).

40. A face similarity calculating method for calculating said similarity on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to the face feature comprising:

estimating a parameter data with regard to a posterior distribution of a difference vector between said face features by using confidence indexes to face features to each of first and second face images; and

calculating, as a similarity, a distance between face features from a likelihood by using said parameter data with regard to a posterior distribution.

41. The face similarity calculating method according to claim 40, wherein supposing that the posterior distribution of the difference vector between said face features is a normal distribution, calculating an adaptive Mahalanobis distance obtained from the likelihood of the normal distribution in said confidence index as the similarity.

42. The face similarity calculating method according to claim 41, including:

estimating a variance of each element of a difference vector to a confidence index, and

calculating the adaptive Mahalanobis distance by using the variance of said each element.

43. The face similarity calculating method according to claim 42, storing in advance the variance of each element of the difference vector to the confidence index, in a variance value table, and referring to said variance value table through the confidence index, and thereby generating a variance value required to calculate the adaptive Mahalanobis distance.

44. The face similarity calculating method according to claim 40, wherein by assuming that the posterior distribution of the difference vector between the face features is a mixture distribution, calculating an adaptive mixture Mahalanobis distance obtained from the likelihood of the mixture distribution in said confidence index as the similarity.

45. A face similarity calculating method for calculating said similarity on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to the face feature, wherein by using a confidence index to a face feature to be compared, estimating a parameter data with regard to a posterior distribution (a within-class distribution) of difference vectors between face features of within-class which should be deemed to be coincident and a posterior distribution (a between-class distribution) of difference vectors between face features of between-class which should be deemed to be inconsistent, and a distance between face features obtained from a likelihood of a ratio between said within-class distribution and said between-class distribution is calculated as a similarity.

46. The face similarity calculating method according to claim 45, wherein by assuming that said within-class distribution and said between-class distribution are respectively the normal distributions, calculating an adaptive discriminant distance obtained from the likelihood of the ratio between the respective distributions in said confidence index as the similarity.

47. The face similarity calculating method according to claim 46, wherein by estimating a within-class variance and a between-class variance of each element of a difference vector to a confidence index, calculating the adaptive discriminant distance as the similarity.

48. The face similarity calculating method according to claim 47, including the first variance value table for storing in advance the within-class variance and the second variance value table for storing in advance the between-class variance, in order to estimate the within-class variance and the between-class variance of each element of the difference vector to the confidence index, and estimating the variance value needed in the adaptive discriminant distance by referring to the first and second variance value tables, respectively, through the confidence index.

49. The face similarity calculating method according to claim 45, wherein by assuming that said within-class distribution and said between-class distribution are respectively the mixture distributions, calculating an adaptive mixture discriminant distance obtained from a likelihood of a ratio between the respective mixture distributions in said confidence index as the similarity.

50. A recording medium for storing a meta-data generating program, including:

extracting a face feature from a face image;

extracting one of a contrast index and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature, from said face image; and

generating said face feature and said confidence index as a face meta-data.

51. A recording medium for storing a similarity calculating program, including:

extracting a face feature from said image;

extracting one of a contrast index and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature; and

calculating a similarity on the basis of said face feature and said confidence index.

52. A recording medium for storing a face verifying program, including:

generating one of a contrast index and an asymmetric index of a face image as a face feature extracted from an input image and a confidence index face indicative of a confidence to the face feature as a face meta-data;

storing generated face meta-data in a face meta-data storing unit;

inputting a face meta-data of a certain face image and each of said face meta-data stored in said face meta-data storing unit, and calculating a face similarity on the basis of a face feature and a confidence index indicative to a confidence to the face feature; and

outputting a face image matching result in accordance with said face similarity.