System and method for identifying speakers

• US 10,013,972 B2
• Filed: 07/19/2016
• Issued: 07/03/2018
• Est. Priority Date: 06/23/2008
• Status: Active Grant

• Current Assignee: Knapp Investment Company Limited
• Original Assignee: J Nicholas and Kristin Gross Trust U/A/D APR 13 2010
• Inventors: Gross, John Nicholas
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US15/214,064
• Publication Number:
  US 20160328548A1
• Time in Patent Office: 714 Days
• Field of Search: 704246, 704243, 704244, 7042564, 704273, 705 18, 705 1469, 379168, 455410, 709225, 726 2, 726 23, 726 7
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04M 2203/2027 Live party detection

Abstract

An electronic challenge system is used to control access to resources by using a spoken test to identify an origin of a voice. The test is based on a series of questions posed during an interactive dialog session with the entity attempting access.

Claims

1. A computer-implemented method of controlling an entity'"'"'s access to a first user account comprising:

[a] storing a first voice print for the first entity for the first user account at the computing system based on a first speech utterance articulating a first challenge response phrase;

[b] selecting and presenting a first challenge question to a second entity during an access attempt to the first user account;

[c] receiving a second speech utterance by said second entity corresponding to a first answer articulated in the form of a first sentence phrase to said first challenge question during said account access attempt;

[d] processing said second speech utterance with said computing system to determine speech characteristics of said second entity; and

repeating steps [b]-[d] to conduct an interactive question and answer dialog session with the second entity to collect and process a plurality of second speech utterance answers articulated in the form of a second plurality of sentence phrases for a corresponding plurality of speech based challenge questions to determine said speech characteristics of said second entity;

wherein said plurality of speech based challenge questions are presented in a format that includes at least a first challenge question and a second follow-up challenge question that relates to and is based on a first answer given by said second entity to said first challenge question;

[e] performing a comparison with the computing system of said speech characteristics of said second entity to said first voice print to determine whether said second speech utterance also originated from said first entity; and

[f] controlling whether said second entity is allowed to access the account based on said comparison.

2. The method of claim 1 wherein said challenge questions are selected based on an initial identification of the first entity so as to maximize a detection capability of the speech recognizer.

3. The method of claim 1 wherein at least a gender and demographic are used to select a speech based challenge question.

4. The method of claim 1 wherein said second follow-up challenge question is generated dynamically during the session by the computing system based on content presented by the entity in said first answer.

5. The method of claim 1 wherein said plurality of speech based challenge questions include questions requesting details about the entity or activities undertaken by the entity.

6. The method of claim 5 wherein said plurality of speech based challenge questions include questions regarding events attended by the entity.

7. The method of claim 5 wherein said plurality of speech based challenge questions include questions regarding the entity'"'"'s job occupation.

8. The method of claim 1 wherein said plurality of second speech utterance answers are checked against a database of black listed entities.

9. The method of claim 1 wherein said plurality of second speech utterance answers are compared to answers derived from a collaborative filtering process.

10. The method of claim 1 wherein said second entity is added to a blacklist if it fails said interactive question and answer dialog session.

11. The method of claim 1 wherein the computing system increases a number of utterances solicited from said second entity to increase a likelihood of detection of an improper entity.

12. The method of claim 1 including a step:

generating a confidence score that said second speech utterance also originated from said first entity.

13. The method of claim 1 including a step:

selecting said speech based challenge questions by measuring and ranking a discrimination capability by said speech recognition engine for distinguishing different entities uttering answers to said questions.

14. The method of claim 1 wherein a profile is created for the first entity which includes voice characteristics of said first entity, including speech acoustic characteristic and speech prosodic characteristics.

15. The method of claim 1 wherein phoneme content is selected for discrimination of said first entity during an enrollment process based on demographics and/or an identity of said first entity.

16. The method of claim 1 wherein a minimum number of unique speech sounds sufficient to distinguish between entities are selected for and presented in said a first challenge question.

17. The method of claim 1 wherein said of speech based challenge questions are selected based on an expected human processing and articulation time.

18. The method of claim 1 wherein an articulation time required by such second entity to provide said second speech utterance is also considered to determine if such utterance originated from said first entity.

19. The method of claim 1 wherein processing of said second speech utterance and controlling of access to said account is performed by multiple different verification servers.

20. The method of claim 1 further including steps:

i) computing a first prosodic score for said first challenge question based on target prosodic characteristics; and

     2) computing a second prosodic score for said second speech utterance based on measured prosodic characteristics; and

     3) comparing said first prosodic score with said second prosodic score as part of said step of determining whether said second entity is allowed access to the account.

21. The method of claim 1 further including a step:

presenting said speech based challenge questions so as to increase prosodic elements in said second speech utterances.

22. The method of claim 1 wherein said speech based challenge questions are selected based on an identity of said first entity so as to maximize a difference in prosodic characteristics to other entities articulating said optimized spoken challenge sentence.

23. The method of claim 1 wherein answers to said speech based challenge questions are ranked and sorted according to a score provided by human test subjects identifying whether a response was vocalized by a human or a computer.

24. A computer-implemented challenge system for controlling an entity'"'"'s access to a first user account comprising:

one or more software routines implemented in a computer readable medium and configured;

[a] to store a first voice print for the first entity for the first user account at the computing system based on a said first speech utterance articulating a first challenge response sentence phrase;

[b] to select and present a first challenge question to a second entity during an access attempt to the first user account;

[c] to receive a second speech utterance by said second entity corresponding to a first answer articulated in the form of a first sentence phrase to said first challenge question during said account access attempt;

[d] to process said second speech utterance to determine speech characteristics of said second entity; and

repeat steps [b]-[d] during an interactive question and answer dialog session with the second entity to collect and process a plurality of second speech utterance answers articulated in the form of a second plurality of sentence phrases for a corresponding plurality of speech based challenge questions to determine said speech characteristics of said second entity;

wherein said plurality of speech based challenge questions are presented in a format that includes at least a first challenge question and a second follow-up challenge question that relates to and is based on a first answer given by said second entity to said first challenge question;

[e] to perform a comparison of said speech characteristics of said second entity to said first voice print to determine whether said second speech utterance also originated from said first entity; and

[f] to control whether said second entity is allowed to access the account based on said comparison.

25. The system of claim 24 wherein said challenge questions are selected by said one or more software routines based on an initial identification of the first entity so as to maximize a detection capability of the speech recognizer.

26. The system of claim 24 wherein at least a gender and demographic are used to select a speech based challenge question.

27. The system of claim 24 wherein said second follow-up challenge question is generated dynamically during the session by the computing system based on content in said first answer.

28. The system of claim 24 wherein said plurality of speech based challenge questions include questions requesting details about the entity or activities undertaken by the entity.

29. The system of claim 24 wherein said one or more software routines are configured to select said speech based challenge questions by measuring and ranking a discrimination capability by said speech recognition engine for distinguishing different entities uttering answers to said questions.

30. The system of claim 24 wherein said one or more software routines are configured to create a profile for the first entity which includes voice characteristics of said first entity, including speech acoustic characteristic and speech prosodic characteristics.

31. The system of claim 24 herein said one or more software routines are configured to select phoneme content for discrimination of said first entity during an enrollment process based on demographics and/or an identity of said first entity.

32. The system of claim 24 wherein said one or more software routines are configured to select and present a minimum number of unique speech sounds sufficient to distinguish between entities in said first challenge question.

33. A computer-implemented method of controlling an entity'"'"'s access to a first user account comprising:

[a] storing a first voice print for the first entity for the first user account at the computing system based on a first speech utterance articulating a first challenge response sentence phrase;

[b] selecting and presenting a first challenge question to a second entity during an access attempt to the first user account, which first challenge question relates to activities performed by the second entity;

[c] receiving a second speech utterance by said second entity corresponding to a first answer articulated in the form of a first sentence phrase to said first challenge question during said account access attempt;

[d] processing said second speech utterance with said computing system to determine speech characteristics of said second entity; and

repeating steps [b]-[d] during an interactive question and answer dialog session with the second entity to collect and process a plurality of second speech utterance answers articulated in the form of a second plurality of phrases for a corresponding plurality of speech based challenge questions to determine said speech characteristics of said second entity;

wherein said plurality of speech based challenge questions are presented in a format that includes at least a first challenge question and a second follow-up challenge question that relates to and is based on a first answer given by said second entity to said first challenge question;

[e] performing an acoustic comparison with the computing system of said speech characteristics of said second entity to said first voice print to determine whether said second speech utterance also originated from said first entity; and

wherein said comparison is made based on an entirety of said second plurality of second speech utterance answers provided during said interactive question and answer dialog and without requiring a password or passphrase from the second entity;

[f] controlling whether said second entity is allowed to access the account based solely on said acoustic comparison to said first voice print.