System and method for generating challenge utterances for speaker verification

• US 10,121,476 B2
• Filed: 03/21/2016
• Issued: 11/06/2018
• Est. Priority Date: 11/24/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Zeljkovic, Ilija, Mishra, Taniya, Stent, Amanda, Syrdal, Ann K., Wilpon, Jay
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/075,333
• Publication Number:
  US 20160203821A1
• Time in Patent Office: 960 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media relating to speaker verification. In one aspect, a system receives a first user identity from a second user, and, based on the identity, accesses voice characteristics. The system randomly generates a challenge sentence according to a rule and/or grammar, based on the voice characteristics, and prompts the second user to speak the challenge sentence. The system verifies that the second user is the first user if the spoken challenge sentence matches the voice characteristics. In an enrollment aspect, the system constructs an enrollment phrase that covers a minimum threshold of unique speech sounds based on speaker-distinctive phonemes, phoneme clusters, and prosody. Then user utters the enrollment phrase and extracts voice characteristics for the user from the uttered enrollment phrase. The system generates a user profile, based on the voice characteristics, for generating random challenge sentences according to a grammar.

Claims

1. A method comprising:

collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech;

identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability;

identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics;

creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics;

generating, by a system comprising a processor and based on the flagged voice characteristics in the user profile, a random challenge sentence according to a challenge grammar; and

prompting the user to utter the random challenge sentence to verify an identity of the user.

2. The method of claim 1, further comprising:

collecting additional user speech;

identifying additional voice characteristics from the additional user speech; and

updating the user profile based on the additional voice characteristics.

3. The method of claim 1, wherein the voice characteristics are associated with different intonation patterns.

4. The method of claim 1, wherein the random challenge sentence is generated according to one of a plurality of sentence frames.

5. The method of claim 1, further comprising adding, by the system, personal information associated with the user to the user profile.

6. The method of claim 1, further comprising adding, by the system, personal contacts associated with the user to the user profile.

7. A non-transitory computer-readable storage medium storing instructions which, responsive to being executed by a processor, cause the processor to perform operations comprising:

collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech;

identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability;

identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics;

creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics;

generating, based on the flagged voice characteristics of interest in the user profile, a random challenge sentence according to a challenge grammar; and

prompting the user to utter the random challenge sentence to verify an identity of the user.

8. The non-transitory computer-readable storage medium of claim 7, wherein the voice characteristics are associated with different intonation patterns.

9. The non-transitory computer-readable storage medium of claim 7, wherein the random challenge sentence is generated according to one of a plurality of sentence frames.

10. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, responsive to being executed by the processor, result in the processor performing operations comprising;

collecting user speech uttered while a user is using a speech service associated with a multi-platform automatic speech recognition engine to yield collected user speech;

identifying speech of interest from the collected user speech, wherein the speech of interest has high speaker discriminatory ability;

identifying, from the speech of interest, voice characteristics for the user, the voice characteristics comprising one or more of speaker-distinctive phonemes, phoneme clusters, and prosody, to yield identified voice characteristics;

creating a user profile for the user by flagging the identified voice characteristics in the user profile as important for generating challenge sentences to yield flagged voice characteristics;

generating, based on the flagged voice characteristics in the user profile, a random challenge sentence according to a challenge grammar; and

prompting the user to utter the random challenge sentence to verify an identity of the user.

11. The system of claim 10, wherein the operations further comprise:

collecting additional user speech;

identifying additional voice characteristics from the additional user speech; and

updating the user profile based on the additional voice characteristics.

12. The system of claim 10, wherein the voice characteristics are associated with different intonation patterns.

13. The system of claim 10, wherein the random challenge sentence is generated according to one of a plurality of sentence frames.

14. The system of claim 10, wherein the operations further comprise adding personal information associated with the user to the user profile.

15. The system of claim 10, wherein the operations further comprise adding personal contacts associated with the user to the user profile.

16. The system of claim 10, wherein the generating of the random challenge sentence comprises:

identifying, using the voice characteristics, a plurality of more valuable phonemes determined to be more valuable than second phonemes for verifying an asserted identity; and

generating the random challenge sentence, based on the voice characteristics, wherein the random challenge sentence comprises the plurality of more valuable phonemes.