System and method for discriminating between speakers for authentication

• US 10,276,152 B2
• Filed: 07/19/2016
• Issued: 04/30/2019
• Est. Priority Date: 06/23/2008
• Status: Active Grant

• Current Assignee: Knapp Investment Company Limited
• Original Assignee: J Nicholas and Kristin Gross
• Inventors: Gross, John Nicholas
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US15/214,027
• Publication Number:
  US 20160328547A1
• Time in Patent Office: 1,015 Days
• Field of Search: 704246, 704243, 704244, 7042564, 704273, 705 18, 705 1469, 379168, 455410, 709225, 726 2, 726 23, 726 7
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04M 2203/2027 Live party detection

Abstract

An audible based electronic challenge system is used to control access to resources by using a test to identify an origin of a voice. The test is based on optimized text sentences selected for their discrimination capability in identifying different speakers.

Claims

1. A computer-implemented method of controlling a

human entity'"'"'s access to an account comprising:

processing biometric information with a speech recognition engine from a plurality of different entities to generate a first set of voice prints for individual ones of such plurality of different entities when articulating utterances of a first spoken text based challenge sentence;

wherein a first voice print is associated with a first entity articulating said first spoken text based challenge sentence for a first user account;

generating a challenge set of additional spoken text based challenge sentences with a computing system for controlling an entity'"'"'s access to the account;

generating a second set of voice prints for said additional spoken text based challenge sentences with a computing system;

wherein said second set of voice prints include both articulated voiceprints and reference voiceprints generated by the computing system for said additional spoken text based challenge sentences;

training said speech recognition engine to distinguish between different entities uttering such challenge set of additional spoken text based challenge sentences;

selecting said first spoken text challenge sentence with the computing system to be articulated as a first speech utterance by an unverified entity accessing the first user account;

storing a first access attempt voice print for the unverified entity at the computing system for the spoken challenge sentence as articulated by the unverified entity;

selecting a first challenge sentence at random from the set of additional spoken text based challenge sentences with the computing system and presenting such sentence to said unverified entity during an access attempt to the first user account;

receiving a second speech utterance from said unverified entity in response to said first challenge sentence;

processing said second speech utterance with said computing system to determine speech characteristics of said unverified entity including a second access attempt voice print; and

processing said first access attempt voice print, said second access attempt voice print, said first voice print, said first set of voice prints and said second set of voice prints with the computing system to determine whether said first speech utterance and second speech utterance originated from said first entity; and

controlling whether said unverified entity is allowed to access the account based on determining whether said unverified entity and said first entity are the same person.

2. The method of claim 1 including a step:

optimizing spoken challenge sentences by measuring and ranking a discrimination capability by said speech recognition engine for distinguishing different entities uttering said set of candidate spoken text based challenge sentences.

3. The method of claim 1 wherein an additional operation of recognition of words within said second speech utterance is performed to identify content of an answer in such utterance and confirm that it matches an expected answer.

4. The method of claim 3 wherein an accuracy of said answer is measured by comparing it to an answer determined from collaborative filtering.

5. The method of claim 1 wherein an interactive dialog of multiple questions and associated answers is performed with said unverified entity to solicit multiple second speech utterances as part of said comparison step as part of an entity identity challenge test prior to granting access to the account.

6. The method of claim 1 wherein said first voice print is obtained during an enrollment process and stored in a profile for said first entity.

7. The method of claim 6 wherein said profile includes biometric information identifying voice characteristics of said first entity, including speech acoustic characteristic and speech prosodic characteristics.

8. The method of claim 6 wherein word phoneme content is selected for discrimination of said first entity during an enrollment process based on demographics and/or an identity of said first entity.

9. The method of claim 1 wherein a minimum number of unique speech sounds sufficient to distinguish between entities are selected for and presented in said first optimized spoken challenge sentence.

10. The method of claim 1 wherein words are selected from a corpus of content for inclusion in said first optimized spoken challenge sentence based on their phonetic distinctiveness.

11. The method of claim 1 wherein said one or more spoken challenge sentences are selected based on an expected human processing and articulation time.

12. The method of claim 1 wherein an articulation time required by such unverified entity to provide said second speech utterance is also considered to determine if such utterance originated from said first entity.

13. The method of claim 1 wherein processing of said second speech utterance and controlling of access to said account is performed by multiple different verification servers.

14. The method of claim 1 wherein said first spoken challenge sentence differs from a second spoken challenge sentence presented to said unverified entity.

15. The method of claim 1 wherein demographic information determined by the computing system for the first entity is used to generate a random challenge sentence constructed according to a challenge grammar to maximize speaker discriminatory ability of the random challenge sentence by the speech recognizer for a particular demographic group.

16. The method of claim 1 further including steps:

i) computing a first prosodic score for said first optimized spoken challenge sentence based on target prosodic characteristics; and

     2) computing a second prosodic score for said second speech utterance based on measured prosodic characteristics; and

     3) comparing said first prosodic score with said second prosodic score as part of said step of determining whether said second entity is allowed access to the account.

17. The method of claim 16 further including a step:

constructing and presenting an optimized spoken challenge sentence so as to increase prosodic elements in said second speech utterance.

18. The method of claim 1 wherein an optimized spoken challenge sentence is selected based on an identity of said first entity so as to maximize a difference in prosodic characteristics to other entities articulating said optimized spoken challenge sentence.

19. The method of claim 1 wherein one or more optimized spoken challenge sentences are ranked and sorted according to a score provided by human test subjects identifying whether a challenge item was vocalized by a human or a computer.

20. The method of claim 1 wherein the reference voiceprints are generated by averaging representative human speech vectors.

21. The method of claim 1 wherein the reference voiceprints are generated pseudo speech vectors.

22. The method of claim 1 wherein the reference voiceprints are based on averaged samples of machine based articulations.

23. The system of claim 1 wherein the reference voiceprints include averaged samples of machine based articulations.

24. A challenge system for controlling access to an

electronic account on a computing system comprising:

one or more software routines implemented in a computer readable medium and adapted to cause the challenge system to;

process biometric information from a plurality of different entities to generate a first set of voice prints for individual ones of such plurality of different entities when articulating utterances of a first spoken text based challenge sentence;

associate a first voice print with a first entity articulating said first spoken text based challenge sentence;

generate a challenge set of additional spoken text based challenge sentences for controlling an entity'"'"'s access to the account;

generate a second set of voice prints for said additional spoken text based challenge sentences;

wherein said second set of voice prints include both articulated voiceprints and reference voiceprints generated for said additional spoken text based challenge sentences;

implement a speech recognition engine to distinguish between different entities uttering such challenge set of additional spoken text based challenge sentences;

select a first spoken challenge sentence to be articulated as a first speech utterance by an unverified entity accessing the first user account;

store a first access attempt voice print for the unverified entity at the computing system for the spoken challenge sentence as articulated by the unverified entity;

select a first challenge sentence at random from the set of additional spoken text based challenge sentences and present such sentence to said unverified entity during an access attempt to the first user account;

receive a second speech utterance from said unverified entity in response to said first challenge sentence;

process said second speech utterance to determine speech characteristics of said unverified entity including a second access attempt voice print; and

process said first access attempt voice print, said second access attempt voice print, said first voice print, said first set of voice prints and said second set of voice prints to determine whether said second speech utterance also originated from said first entity; and

control whether unverified entity is allowed to access the account based on a determination of whether said unverified entity and said first entity are the same person.

25. The system of claim 24 wherein the one or more software routines are further configured to conduct an interactive dialog of multiple questions and associated sentence based answers with said unverified entity to solicit multiple second speech utterances as part of said comparison step as part of an entity identity challenge test prior to granting access to the account.

26. The system of claim 24 wherein the one or more software routines are further configured to select word phoneme content for discriminating said first entity during an enrollment process based on demographics and/or an identity of said first entity.

27. The system of claim 24 wherein the one or more software routines are further configured to select a minimum number of unique speech sounds determined to be sufficient to distinguish between entities articulating said first spoken challenge sentence.

28. The system of claim 24 wherein the one or more software routines are further configured to select words from a corpus of content for inclusion in said first spoken challenge sentence based on their phonetic distinctiveness.

29. The system of claim 24 wherein the one or more software routines are further configured such that demographic information determined by the computing system for the first entity is used to generate a random challenge sentence constructed according to a challenge grammar to maximize speaker discriminatory ability of the random challenge sentence by the speech recognizer for a particular demographic group.

30. The system of claim 24 wherein the one or more software routines are further configure to compute a first prosodic score for said first spoken challenge sentence based on target prosodic characteristics;

and

     2) compute a second prosodic score for said second speech utterance based on measured prosodic characteristics; and

     3) compare said first prosodic score with said second prosodic score as part of said step of determining whether said second entity is allowed access to the account.

31. The system of claim 24 wherein the one or more software routines are further configured to construct and presenting a first optimized spoken challenge sentence so as to increase prosodic elements in said second speech utterance.

32. The system of claim 24 wherein the spoken challenge sentences are optimized by ranking and sorting according to a score provided by human test subjects identifying whether a challenge item was vocalized by a human or a computer.

33. The system of claim 24 wherein the one or more software routines are further configured to generated the reference voiceprints by averaging representative human speech vectors.

34. The system of claim 24 wherein the one or more software routines are further configured to generated the reference voiceprints by generating pseudo speech vectors.