Recognition system, particularly for recognising people

• US 5,412,738 A
• Filed: 08/10/1993
• Issued: 05/02/1995
• Est. Priority Date: 08/11/1992
• Status: Expired due to Term

• Current Assignee: Fondazione Bruno Kessler
• Original Assignee: Istituto Trentino Di Cultura
• Inventors: Stringa, Luigi, Falavigna, Daniele, Poggio, Tomaso, Brunelli, Roberto
• Primary Examiner(s): Mancuso, Joseph
• Application Number:
  US08/103,700
• Time in Patent Office: 630 Days
• Field of Search: 382/2-5, 382/30, 382/34, 340/825.34, 348/152, 348/156, 348/143, 381/41, 381/42
• US Class Current: 382/115
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G10L 17/10 Multimodal systems, i.e. based on the integration of multiple recognition engines or fusion of expert systems

Abstract

An integrated, multisensory recognition (identification and verification) system is described. Acoustic features and visual features are integrated in order to identify people or to verify their identities. The integration of the speaker-identification and visual-features-identification functions improves both performance and reliability in the applications envisaged. Various architectures are described for the implementation both of the integration function and of the speaker-recognition and visual-features-recognition functions.

Claims

1. A system for recognizing people comprising:

means for sensing whether a person is to be recognized;

a first detector subsystem, responsive to said sensing means, including,

first detector means for detecting first features including static and dynamic data associated with the person to be recognized,

a first database including prestored static and dynamic data relating to at least one person to be recognized, and

means for comparing the detected static and dynamic data with the prestored static and dynamic data and obtaining first distance data;

a second detector subsystem, responsive to said sensing means, including,

second detector means for detecting second features associated with the person to be recognized,

a second database including prestored second features relating to at least one person to be recognized, and

means for comparing the detected second features and the prestored second features and obtaining second distance data; and

an integration subsystem including means for combining the first distance data and the second distance data resulting in recognition data, and determining whether the recognition data corresponds to a value related to the prestored static and dynamic data and the prestored second features associated with one person.

2. The system according to claim 1, wherein the combining means sums the first distance data and the second distance data to obtain the recognition data.

3. The system according to claim 1, wherein said integration subsystem recognizes a person whose detected first and second features correspond to the lowest value related to the prestored static and dynamic data and the prestored second features associated with one person.

4. The system according to claim 1, wherein said integration subsystem maps the first and second distance data in accordance with an exponential law and obtains the recognition data as a product of the mapped first and second distance data.

5. The system according to claim 4, wherein according to the exponential law, the first and second distance data are negative exponents so that the mapped first and second distance data are within a finite interval.

6. The system according to claim 4 or 5, wherein the mapped first and second distance data are normalized according to:

##EQU10##

7. The system according to claim 4, wherein the recognition data is a weighted merged score according to:

space="preserve" listing-type="equation">S(w).sub.i =S.sup.w.sub.vi *S.sub.si.sup.(l-w)

in which w is selected to maximize performance of said integration subsystem.

8. The system according to claim 1, wherein the first and second distance data have respective variances and are normalized with respect to their standard deviations, the recognition data being calculated from the normalized first and second distance data.

9. The system according to claim 1, wherein the first features correspond to speech features of the person to be recognized and the second features correspond to facial features of the person to be recognized.

10. The system according to claim 1, wherein at least one of said first detector subsystem and said second detector subsystem further includes threshold means for comparing the first or second distance data with at least one threshold value to prevent recognition when the first or second distance data exceeds the at least one threshold.

11. The system according to claim 1, wherein said sensing means detects changes in an environment proximate to the system indicating presence of a person to be recognized.

12. The system according to claim 11, wherein said sensing means receives signals from at least one of said first and second detectors to detect environmental changes.

13. The system according to claim 11, wherein said sensing means receives signals from said second detector to detect environmental changes represented by the detected second features which correspond to facial features of the person to be recognized, said sensing means further including means for activating said first and second subsystems to recognize a person when an image of the detected second features has stabilized for a predetermined period of time.

14. The system according to claim 1, wherein said first subsystem obtains the first distance data in accordance with:

##EQU11## in which θ

_ik is a first vectorial parameter identifying the first features of the person to be recognized, Ψ

_ik is a second vectorial parameter identifying a corresponding one of the prestored static and dynamic data in said first database, σ

_k² is a mean variance of a k-th component of the parametric vector (θ

_i, Ψ

_i) and p is a predetermined constant factor.

15. The system according to claim 14, wherein p is 8.

16. The system according to claim 14, wherein the dynamic data is a first order time derivative of the static data.

17. The system according to claim 16, wherein the static data is based on a time-window analysis of a speech signal.

18. The system according to claim 1, wherein the detected second features of the person to be recognized and the prestored second features include at least one of the following features:

eyebrow thickness, distances of eyebrows from an interocular axis corresponding with eyes, arch description of at least one eyebrow, vertical position of a nose, nose width, vertical position of a mouth, lip width, lip thickness, lower jaw description according to radii originating from a mouth center, face width at nose level, and face width at cheekbone level.

19. The system according to claim 18, wherein the second subsystem obtains the second distance data based on a covariance matrix of the second features of the person to be recognized.

20. The system according to claim 1, wherein the detected second features of the person to be recognized and the prestored second features relate to digital representations of portions of a face including at least one of the following portions of the face:

the eyes, nose, mouth, and entire face;

the digital representations relating to luminous intensity reflected by the at least one portion of the face, the second distance data being obtained by correlating homologous data relating to the detected second features of the person to be recognized and the prestored second features for the at least one portion of the face.

21. The system according to claim 1, wherein the detected second features of the person to be recognized and the prestored second features indicate distribution of greyness levels in a facial image.

22. The system according to claim 21, wherein the second subsystem further comprises means for computing directional derivatives of the facial image.

23. The system according to claim 22, wherein the directional derivatives are converted into binary form prior to being compared.

24. The system according to claim 21, wherein the second subsystem obtains the second distance data in accordance with:

##EQU12## in which dI(y,x)=I(y,x)-I(y-1,x-1) is the directional derivative of image I(x,y) to be recognized and P_ij (y,x) is the i-th image of the j-th of the prestored second features and N_j is the number of images in the j-th class.