Face meta-data generation and face similarity calculation

• US 7,593,551 B2
• Filed: 12/16/2002
• Issued: 09/22/2009
• Est. Priority Date: 12/14/2001
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kamei, Toshio
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US10/488,213
• Publication Number:
  US 20040197013A1
• Time in Patent Office: 2,472 Days
• Field of Search: 382/118, 382/125, 382/228
• US Class Current: 382/118
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A face feature is extracted by a face feature extracting unit, a confidence index is extracted by a confidence index extracting unit, and then, they are outputted as a face meta-data. At a time of matching, by using the confidence index of the face meta-data, a distribution estimating unit estimates the data of a parameter and the like with regard to a posterior distribution when the confidence index is obtained. A distance calculating unit calculates a similarity between feature values. Consequently, the precision of face verification can be improved, thereby carrying out a practical face matching.

Claims

1. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a confidence index indicative of a confidence to said face feature, from said face image,

wherein said face feature and said confidence index are generated as a face meta-data,

said confidence index extracting unit comprises;

a first confidence index extracting unit which extracts a contrast index of said face image; and

a second confidence index extracting unit which extracts as an asymmetric index of said face image one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image.

2. The face meta-data generating apparatus according to claim 1, wherein said first confidence index extracting unit extracts one of a standard deviation and a variance of the pixel values in said face image, as the contrast index of said face image.

3. A face meta-data generating apparatus comprising:

a face feature extracting unit which extracts a face feature from a face image; and

a confidence index extracting unit which extracts a confidence index indicative of a confidence to said face feature, from said face image,

wherein said face feature and said confidence index are generated as a face meta-data,

said confidence index extracting unit comprising;

a first confidence index extracting unit which extracts a contrast index of said face image; and

a second confidence index extracting unit which extracts as an asymmetric index of said face image one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image,

said face feature extracting unit extracting said face feature by obtaining a pattern vector Λ

from said face image, and calculating said face feature using a basis matrix U and a mean vector of said feature vector Λ

,

said basis matrix U is obtained by the linear combination of;

a basis matrix A specified by a subset of basis vectors selected from basis vectors obtained by the principal component analysis of a training set; and

a basis matrix B specified by a flipped subset of basis vectors in which elements of said basis vectors are exchanged to correspond to a conversion of pixel values where a left side and a right side of said face image is flipped.

4. The face recta-data generating apparatus according to claim 3, wherein said pattern vector A has pixels values in said face image as its elements.

5. The face meta-data generating apparatus according to claim 3, wherein said pattern vector A has Fourier components of said face image.

6. A face similarity calculating apparatus which calculates a similarity of a face on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to said face feature comprising:

a distribution estimating unit which estimates a parameter data with regard to a posterior distribution of a difference vector between said face features, by using said confidence index to said face feature; and

a distance calculating unit which calculates, as said similarity, a distance between said face features obtained from a likelihood of said posterior distribution by using said parameter data, wherein said confidence index comprises a contrast index of said face image and, as an asymmetric index of said face image, one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image.

7. The face similarity calculating apparatus according to claim 6, wherein said distance calculating unit, when said posterior distribution of the difference vector between said face features is a normal distribution, calculates, as said similarity, an adaptive Mahalanobis distance obtained from said likelihood of the normal distribution in said confidence index.

8. The face similarity calculating apparatus according to claim 7, wherein said distribution estimating unit estimates a variance of each element of said difference vector to said confidence index, and

said distance calculating unit calculates said adaptive Mahalanobis distance by using the variance of said each element.

9. The face similarity calculating apparatus according to claim 8, wherein said distribution estimating unit includes a variance value table which stores in advance the variance of each element of said difference vector to said confidence index, and reads out a variance required to calculate said adaptive Mahalanobis distance by referring to said variance value table on the basis of said confidence index, and

said distance calculating unit calculates said adaptive Mahalanobis distance by using the variance of said each element.

10. The face similarity calculating apparatus according to claim 6, which calculates as said similarity said adaptive mixture Mahalanobis distance that is obtained from a likelihood of said mixture distribution in said confidence index, when said posterior distribution of a difference vector between said face features is a mixture distribution.

11. A face similarity calculating apparatus which calculates a similarity of a face on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to said face feature, which, by using said confidence index to said face feature, and estimating a parameter data with regard to a within-class distribution that is said posterior distribution of the difference vectors between said face features in a class which should be deemed to be coincident, and an between-class distribution that is a posterior distribution of the difference vectors between said face features in a class which should be deemed to be inconsistent, calculates, as said similarity, a distance between said face features obtained from a likelihood of a ratio between said within-class distribution and said between-class distribution based on said parameter data, wherein said confidence index comprises an asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image.

12. The face similarity calculating apparatus according to claim 11, which calculates as said similarity said adaptive discriminant distance obtained from a likelihood of a ratio between respective distributions in said confidence index, when said within-class distribution and said between-class distribution are respectively the normal distributions.

13. The face similarity calculating apparatus according to claim 12, which estimates a within-class variance and a between-class variance of each element of said difference vector to said confidence index, and calculates as said similarity said adaptive discriminant distance on the basis of said estimated variance.

14. The face similarity calculating apparatus according to claim 13, which the first variance value table for storing in advance said within-class variance and the second variance value table for storing in advance said between-class variance in order to estimate said within-class variance and said between-class variance of each element of the difference vector to said confidence index,

refers to said first and second variance value tables, respectively, on the basis of said confidence index estimates the variance, and calculates as said similarity said adaptive discriminant distance on the basis of said estimated variance.

15. The face similarity calculating apparatus according to claim 11, which calculates as said similarity said adaptive mixture discriminant distance obtained from a likelihood of a ratio between the respective mixture distributions in said confidence index by supposing that said within-class distribution and said between-class distribution are mixture distributions, respectively.

16. A face recognizing system comprising:

an image input unit which inputs a first face image;

a face meta-data generating unit which extracts a face feature from said first face image, extracts a contrast index of said first face image as a confidence index indicative of a confidence to said face feature from said first face image and an asymmetric index of said first face image as another confidence index, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image, and generates said face feature and said confidence indexes as the first face meta-data;

a face meta-data storing unit which stores the second face meta-data;

a face similarity calculating unit which calculates a face similarity between said first face meta-data and each of said second face meta-data; and

a control unit which controls said image input unit, said face meta-data generating unit, and said face similarity calculating unit, in response to an input command.

17. A face recognizing method comprising:

extracting a face feature from a first face image;

generating a contrast index of said first face image as a confidence index indicative of a confidence to said face feature, and an asymmetric index of said first face image as another confidence index to said face feature, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image, as the first face meta-data;

reading out each second face meta-data;

calculating a face similarity between each of said second face meta-data and said first face meta-data, on the basis of said confidence indexes and said face feature of each of said second face meta-data and said first face meta-data; and

outputting a matching result of said face image in accordance with said face similarity.

18. A face meta-data generating method for generating a face meta-data from a face image to be used for face image similarity evaluation comprising:

(a) extracting a face feature from a face image;

(b) extracting a contrast index of said face image and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said face image, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image; and

(c) generating said face feature and said confidence index as a face meta-data.

19. The face meta-data generating method according to claim 18, wherein said step (b) includes:

extracting a difference between the maximum value and the minimum value of pixel values in said face image, as the contrast index of said face image.

20. The face meta-data generating method according to claim 18, wherein said step (b) includes:

extracting one of a standard deviation and a variance of the pixel values in said face image, as the contrast index of said face image.

21. A face similarity calculating method for calculating said similarity on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to the face feature comprising:

estimating a parameter data with regard to a posterior distribution of a difference vector between said face features by using confidence indexes to face features to each of first and second face images; and

calculating, as a similarity, a distance between face features from a likelihood by using said parameter data with regard to a posterior distribution, wherein said confidence index comprises a contrast index of said face image and, as an asymmetric index of said face image, one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image.

22. The face similarity calculating method according to claim 21, wherein supposing that the posterior distribution of the difference vector between said face features is a normal distribution, calculating an adaptive Mahalanobis distance obtained from the likelihood of the normal distribution in said confidence index as the similarity.

23. The face similarity calculating method according to claim 22, including:

estimating a variance of each element of a difference vector to a confidence index, and

calculating the adaptive Mahalanobis distance by using the variance of said each element.

24. The face similarity calculating method according to claim 23, storing in advance the variance of each element of the difference vector to the confidence index, in a variance value table, and referring to said variance value table through the confidence index, and thereby generating a variance value required to calculate the adaptive Mahalanobis distance.

25. The face similarity calculating method according to claim 21, wherein by assuming that the posterior distribution of the difference vector between the face features is a mixture distribution, calculating an adaptive mixture Mahalanobis distance obtained from the likelihood of the mixture distribution in said confidence index as the similarity.

26. A face similarity calculating method for calculating said similarity on the basis of a face feature extracted from a face image and a confidence index indicative of a confidence to the face feature, comprising the steps of:

using a confidence index to a face feature to be compared and estimating a parameter data with regard to a within-class posterior distribution of difference vectors between face features of within-class deemed to be coincident and a between-class posterior distribution of difference vectors between face features of between-class deemed to be inconsistent; and

calculating a distance between face features obtained from a likelihood of a ratio between said within-class distribution and said between-class distribution as a similarity, wherein said confidence index comprises a contrast index of said face image and, as an asymmetric index of said face image, one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image.

27. The face similarity calculating method according to claim 26, further comprising the steps of:

calculating an adaptive discriminant distance obtained from the likelihood of the ratio between the within-class and between-class distributions in said confidence index as the similarity, when said within-class distribution and said between-class distribution are normal distributions.

28. The face similarity calculating method according to claim 27, further comprising the steps of:

estimating a within-class variance and a between-class variance of each element of a difference vector to a confidence index; and

calculating the adaptive discriminant distance as the similarity.

29. The face similarity calculating method according to claim 28, further comprising the steps of:

including the first variance value table for storing in advance the within-class variance and the second variance value table for storing in advance the between-class variance to estimate the within-class variance and the between-class variance of each element of the difference vector to the confidence index; and

estimating the variance value needed in the adaptive discriminant distance by referring to the first and second variance value tables, respectively, through the confidence index.

30. The face similarity calculating method according to claim 26, further comprising the steps of:

assuming that said within-class distribution is a mixture distribution and said between-class distribution is another mixture distribution and calculating an adaptive mixture discriminant distance obtained from a likelihood of a ratio between the mixture distributions in said confidence index as the similarity.

31. A computer readable medium having a stored computer readable program for operating on a computer for a meta-data generating program comprising instructions that cause the computer to perform the steps of:

extracting a face feature from a face image;

extracting a contrast index of said face image and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said face image, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image; and

generating said face feature and said confidence index as a face meta-data.

32. A computer readable medium having a stored computer readable program for operating on a computer for executing a similarity calculating program, comprising instructions that cause the computer to perform the steps of:

extracting a face feature from said face image;

extracting a contrast index of said image and an asymmetric index of said face image as a confidence index indicative of a confidence to said face feature from said face image, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image; and

calculating a similarity on the basis of said face feature and said confidence index.

33. A computer readable medium having a stored computer readable program for operating on a computer for executing a face verifying program, comprising instructions that cause the computer to perform the steps of:

generating a contrast index of a face image as a face feature extracted from an input image and an asymmetric index of said face image as the face feature extracted from the input image and a confidence index indicative of a confidence to the face feature as a face meta-data, said asymmetric index comprising one of a maximum value of differences between said face image and a flipped image obtained by flipping said face image, and one of a sum and an average of powers of pixel values of difference images between said face image and said flipped image;

storing generated face meta-data in a face meta-data storing unit;

inputting a face meta-data of a certain face image and each of said face meta-data stored in said face meta-data storing unit, and calculating a face similarity on the basis of the face feature and the confidence index indicative to the confidence to the face feature; and

outputting a face image matching result in accordance with said face similarity.