Lip-password based speaker verification system

• US 9,159,321 B2
• Filed: 02/25/2013
• Issued: 10/13/2015
• Est. Priority Date: 02/27/2012
• Status: Active Grant

• Current Assignee: Hong Kong Baptist University
• Original Assignee: Hong Kong Baptist University
• Inventors: Cheung, Yiu-ming, Liu, Xin
• Primary Examiner(s): Yang, Qian
• Application Number:
  US13/776,615
• Publication Number:
  US 20130226587A1
• Time in Patent Office: 960 Days
• Field of Search: 704/273, 382/118
• US Class Current: 1/1
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis

Abstract

A lip-based speaker verification system for identifying a speaker using a modality of lip motions; wherein an identification key of the speaker comprising one or more passwords; wherein the one or more passwords are embedded into lip motions of the speaker; wherein the speaker is verified by underlying dynamic characteristics of the lip motions; and wherein the speaker is required to match the one or more passwords embedded in the lip motions with registered information in a database. That is, in the case where the target speaker saying the wrong password or even in the case where an impostor knowing and saying the correct password, the nonconformities will be detected and the authentications/accesses will be denied.

Claims

1. A lip-based speaker verification system for identifying a speaker, comprising one or more computer processors for executing a process of verification of identity of the speaker using one modality of lip motions;

wherein an identification key of the speaker comprising one or more passwords;

wherein the one or more passwords are embedded into lip motions of the speaker;

wherein the speaker is verified by underlying dynamic characteristics of the lip motions and extracted area-based features wherein the extracted area-based features further comprise teeth, tongue and oral cavity during the utterance; and

wherein the speaker is required to match the one or more passwords embedded in the lip motions with registered information in a database such that the matching between the dynamic characteristics of the speaker lip motions and the extracted area-based features with the one or more passwords is verified by using one or more multi-boosted hidden Markov models (HMMs);

wherein the process comprises the steps of;

(1) extracting visual features for each lip frame;

(2) performing lip motion segmentation of D to yield D={D₁, D₂, . . . , D_p} where D denotes the one or more passwords, and p is the number of password components;

(3) for each value of m=1, . . . , p, performing the steps of;

(3.1) getting a training set D_m^T={X₁^T, X₂^T, . . . , X_Na^T} of the speaker and D_m^I={X₁¹, X₂¹, . . . , X_Nb^I} of an imposer, and forming a novel training set using a data sharing scheme (DSS);

(3.2) initializing w_i,j^T, w_i,j^I, r and ε

⁰ respectively with;

2. The system of claim 1, wherein the one or more passwords are synchronously embedded into the lip motions, such that the verification of the speaker'"'"'s identity by the underlying dynamic characteristics of the lip motions and the matching of the one or more passwords embedded in the lip motion is performed simultaneously.

3. The system according to claim 2;

wherein the modality of lip motions is completely insensitive to background noise;

wherein acquisition of lip motions of the speaker is insusceptible to distance between the speaker'"'"'s lips and an acquisition means of the system;

wherein the system is usable by a mute person; and

wherein the system can operate in silence and in an obscure manner.

4. The system according to claim 2, wherein the system is included in one or more security systems.

5. The system according to claim 2, wherein the system is implemented across one or more computing hardware platform in one or more locations.

6. The system according to claim 2, wherein the process of verification of identity of the speaker is implemented in software that is executable on one or more hardware platform.

7. The system according to claim 1;

wherein the modality of lip motions is completely insensitive to background noise;

wherein acquisition of lip motions of the speaker is insusceptible to distance between the speaker'"'"'s lips and an acquisition means of the system;

wherein the system is usable by a mute person; and

wherein the system can operate in silence and in an obscure manner.

8. The system according to claim 1, wherein the system is included in one or more security systems.

9. The system according to claim 1, wherein the system is implemented across one or more computing hardware platform in one or more locations.

10. The system according to claim 1, wherein the process of verification of identity of the speaker is implemented in software that is executable on one or more hardware platform.

11. The system according to claim 1, wherein the process further comprises the steps of:

(4) given a test lip-password sequence V={v₁, v₂, . . . , v_p}, verifying each subunit via