Personal access control system using speech and face recognition

• US 4,449,189 A
• Filed: 11/20/1981
• Issued: 05/15/1984
• Est. Priority Date: 11/20/1981
• Status: Expired due to Term

• Current Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Ruell, Hartwig E., Feix, Wolfgang H.
• Primary Examiner(s): Kemeny, E. S. Matt
• Application Number:
  US06/323,657
• Time in Patent Office: 907 Days
• Field of Search: 381/41, 381/42, 382/2, 364/513.5, 364/513
• US Class Current: 704/272
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition

Abstract

A method and an apparatus are disclosed for identifying an individual through a combination of both speech and face recognition. The voice signature of an interrogated person uttering a key word into a microphone is compared in a pattern matcher with the previously stored voice signature of a known person uttering the same key word to obtain a first similarity score. At the same time, when a key event in the utterance of the key word by the interrogated person occurs, a momentary image of that person'"'"'s mouth region onto which a grid pattern has been projected is optically recorded and compared with the previously stored corresponding momentary image of the same known person to obtain a second similarity score. The two similarity scores are analyzed to verify that the identity of the interrogated person is that of the known person.

Claims

1. A method for identifying an individual through a combination of speech and face recognition which comprises:

(a) defining a characteristic sequence of features of the voice in response to the utterance of a predetermined key word by the individual to be identified;

(b) forming a momentary image of a voice-utterance varying portion of the individual'"'"'s face upon the occurrence of a key event in the utterance of the key word; and

(c) using both the defined sequence of features and the momentary image in order to determine the identity or non-identity of the individual.

3. A method according to claims 1 or 2, which further comprises communicating the predetermined key word to the individual in response to a request.

4. A method according to claims 1 or 2, which further comprises projecting a grid pattern onto the voice-utterance varying portion of the individual'"'"'s face.

5. A method according to claim 1, wherein the momentary image forming step comprises:

(a) scanning the voice-utterance varying portion of the individual'"'"'s face with an imaging device and thereby creating a sequence of momentary images; and

(b) detecting the occurrence of the key event and storing that momentary image in the sequence of momentary images which corresponds to the moment of occurrence of the key event;

and wherein the identity determining step comprises;

(a) computing a first similarity score between the defined sequence of the features and a reference sequence of features;

(b) computing a second similarity score between the stored momentary image and a known reference momentary image; and

(c) determining the identity or non-identity of the individual by evaluating the computed first and second similarity score.

2. A method for identifying an individual through a combination of speech and face recognition which comprises

(a) defining a characteristic sequence of features of the voice in response to the utterance of a predetermined key word by the individual to be identified;

(b) forming a sequence of momentary images of a voice-utterance varying portion of the individual'"'"'s face upon the occurrence of a sequence of key events in the utterance of the key word; and

(c) using both the defining sequence of features and the sequence of momentary images to determine the identity or non-identity of the individual.

6. Apparatus for identifying an individual through a combination of speech and face recognition which comprises:

(a) means for defining a characteristic sequence of features of the voice in response to the utterance of a predetermined key word by the individual to be identified;

(b) means for forming a momentary image of a voice-utterance varying portion of the individual'"'"'s face upon the occurrence of a key event in the utterance of the key word; and

(c) identification means connected to both the defining means and the momentary image-forming means for using both the defined sequence of features and the momentary image in order to determine the identity or non-identity of the individual.

7. Apparatus according to claim 6, which further comprises:

means associated with the defining means for communicating the predetermined key word to the individual in response to a request.

8. Apparatus according to claims 6 or 7 wherein the momentary image forming means comprises means for projecting a grid pattern onto the voice utterance varying portion of the individual'"'"'s face.

9. Apparatus according to claim 6,

(a) wherein the momentary image-forming means comprises:

means for scanning the voice-utterance varying portion of the individual'"'"'s face to create a sequence of momentary images, and

means for detecting the occurrence of the key event and storing the momentary image in the sequence of momentary images which corresponds to the moment of occurrence of the key event; and

(b) wherein the identification means comprises;

a first pattern matcher connected to compute a first similarity score between the sequence of features and a reference sequence of features, and

a second pattern matcher connected to compute a second similarity score between the stored momentary image and a reference momentary image; and

(c) wherein the identification means comprises means for determining the identity or non-identity of the individual by evaluating the computed first and second similarity scores.

10. Apparatus according to claim 6, wherein the defining means comprises:

(a) a microphone for developing signals corresponding to the utterance of the predetermined key word by the individual;

(b) a preamplifier connected to amplify the signals developed by the microphone, and

(c) an extractor which is coupled to the pre-amplifier to define the sequence of features from the amplified signals.

11. Apparatus according to claim 6, wherein the momentary image forming means comprises:

(a) an electronic camera for developing scanning signals corresponding to a sequence of momentary images of the voice-utterance varying portion of the individual'"'"'s face;

(b) a detector being connected to the electronic camera for converting the signals developed by the electronic camera into digital signals;

(c) a memory coupled to the detector for receiving the digital signals; and

(d) a key event detecting unit being connected to control the memory so that the memory stores the digital signals of the momentary image in the sequence of momentary images that corresponds to the moment of occurrence of the key event.

12. Apparatus according to claim 6, wherein the identification means comprises:

(a) a first buffer for storing a reference sequence of features;

(b) a first pattern matcher coupled to the defining means and to the first buffer for computing a first similarity score between the sequence of features defined by the defining means and the reference sequence of features;

(c) a second buffer for storing a reference momentary image;

(d) a second pattern matcher coupled to the momentary image forming means and the second buffer for computing a second similarity score between the momentary image formed by the momentary image forming means and the reference momentary image;

(e) a third buffer, coupled to communicate with the momentary image forming means, for storing a reference set of parameters used by the momentary image forming means to define the occurrence of a key event;

(f) a microprocessor control unit for storing the reference sequence of features, the reference momentary image and the reference set of parameters and coupled to communicate this stored information respectively to the first, second and third buffers; and

also coupled to the pattern matchers for determining the identity or non-identity of the individual by evaluating the computed first and second similarity scores; and

(g) communicating means associated with the microprocessor control unit for communicating the predetermined key word to the individual in response to a request.

13. Apparatus according to claim 6, wherein the momentary image forming means includes a key event detecting unit to detect the occurrence of the key event in the utterance of the key word, the key event detecting unit comprising:

(a) an integrator coupled to receive the sequence of features from the defining means in the form of an electro-acoustical signal defined in response to the utterance of a key word by the individual and serving to form a time-varying signal which is a function of the amplitude of the electro-acoustical signal;

(b) a control unit connected to the integrator for detecting the beginning of the key word by analyzing the time-varying signal formed by the integrator;

(c) a slope detector coupled to the control unit and the integrator to receive the time-dependent signal for detecting characteristic slope features; and

(d) a comparator coupled to the slope detector and to the control unit for comparing the characteristic slope features detected by the slope detector with predetermined reference slope features defining the key event; and

(e) means coupled to the comparator to store the momentary image of a voice-utterance varying portion of the individual'"'"'s face corresponding to the key event in the utterance of the predetermined key word when coincidence between the detected slope features and the reference slope features is determined by the comparator.