Dynamic threshold for speaker verification

• US 9,502,039 B2
• Filed: 06/21/2016
• Issued: 11/22/2016
• Est. Priority Date: 06/24/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Foerster, Jakob Nicolaus, Casado, Diego Melendo
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/188,047
• Publication Number:
  US 20160300575A1
• Time in Patent Office: 154 Days
• Field of Search: 704/250, 704/233, 704/246, 704/251, 704/275, 704/231, 704/236, 704/E15.001, 704/E15.005
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 17/08 Use of distortion metrics or a particular distance between probe pattern and reference templates
• CPC Class Codes: G10L 17/12 Score normalisation
• CPC Class Codes: G10L 17/14 Use of phonemic categorisation or speech recognition prior to speaker recognition or verification
• CPC Class Codes: G10L 17/20 Pattern transformations or operations aimed at increasing system robustness, e.g. against channel noise or different working conditions
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04M 3/385 using speech signals

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a dynamic threshold for speaker verification are disclosed. In one aspect, a method includes the actions of receiving, for each of multiple utterances of a hotword, a data set including at least a speaker verification confidence score, and environmental context data. The actions further include selecting from among the data sets, a subset of the data sets that are associated with a particular environmental context. The actions further include selecting a particular data set from among the subset of data sets based on one or more selection criteria. The actions further include selecting, as a speaker verification threshold for the particular environmental context, the speaker verification confidence score. The actions further include providing the speaker verification threshold for use in performing speaker verification of utterances that are associated with the particular environmental context.

Claims

1. A computer-implemented method comprising:

receiving, by a computing device that uses voice-based speaker identification, audio data corresponding to an utterance by the user of a predefined hotword;

in response to a false rejection of the audio data corresponding to the utterance, prompting the user to verify their identification using a technique other than voice-based speaker identification;

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that the audio data corresponding to the utterance was falsely rejected;

receiving data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected; and

in response to receiving the data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected, using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

2. The method of claim 1, wherein prompting the user to verify their identification using a technique other than voice-based speaker identification comprises:

prompting the user for a password.

3. The method of claim 1, comprising:

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected.

4. The method of claim 3, comprising:

receiving additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected; and

in response to receiving the additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected, using the additional, previously received audio data in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

5. The method of claim 3, wherein prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected comprises:

providing, for display and while prompting the user, data indicating a timestamp that the additional utterance was received.

6. The method of claim 1, comprising:

identifying an environmental context associated with receiving the audio data corresponding to the utterance by the user of the predefined hotword;

wherein using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected comprises using the environmental context in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

7. The method of claim 6, wherein the environmental context specifies a loudness of the utterance, an amount of noise detected immediately preceding receipt of the utterance, or signal-to-noise ratio of a loudness of an audio signal that encodes the utterance.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a computing device that uses voice-based speaker identification, audio data corresponding to an utterance by the user of a predefined hotword;

in response to a false rejection of the audio data corresponding to the utterance, prompting the user to verify their identification using a technique other than voice-based speaker identification;

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that the audio data corresponding to the utterance was falsely rejected;

receiving data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected; and

in response to receiving the data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected, using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

9. The system of claim 8, wherein prompting the user to verify their identification using a technique other than voice-based speaker identification comprises:

prompting the user for a password.

10. The system of claim 8, wherein the operations further comprise:

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected.

11. The system of claim 10, wherein the operations further comprise:

receiving additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected; and

in response to receiving the additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected, using the additional, previously received audio data in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

12. The system of claim 10, wherein prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected comprises:

providing, for display and while prompting the user, data indicating a timestamp that the additional utterance was received.

13. The system of claim 8, wherein the operations further comprise:

identifying an environmental context associated with receiving the audio data corresponding to the utterance by the user of the predefined hotword;

wherein using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected comprises using the environmental context in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

14. The system of claim 13, wherein the environmental context specifies a loudness of the utterance, an amount of noise detected immediately preceding receipt of the utterance, or signal-to-noise ratio of a loudness of an audio signal that encodes the utterance.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a computing device that uses voice-based speaker identification, audio data corresponding to an utterance by the user of a predefined hotword;

in response to a false rejection of the audio data corresponding to the utterance, prompting the user to verify their identification using a technique other than voice-based speaker identification;

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that the audio data corresponding to the utterance was falsely rejected;

receiving data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected; and

in response to receiving the data indicating that the user has confirmed that the audio data corresponding to the utterance was falsely rejected, using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

16. The medium of claim 15, wherein prompting the user to verify their identification using a technique other than voice-based speaker identification comprises:

prompting the user for a password.

17. The medium of claim 15, wherein the operations further comprise:

in response to the user successfully verifying their identification using the technique other than voice-based speaker identification, prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected.

18. The medium of claim 17, wherein the operations further comprise:

receiving additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected; and

in response to receiving the additional data indicating that the user has indicated that the additional, previously received audio data corresponding to the additional utterance was properly accepted, properly rejected, falsely accepted, or falsely rejected, using the additional, previously received audio data in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.

19. The medium of claim 17, wherein prompting the user to confirm that additional, previously received audio data corresponding to an additional utterance by the user of the predefined hotword was properly accepted, properly rejected, falsely accepted, or falsely rejected comprises:

providing, for display and while prompting the user, data indicating a timestamp that the additional utterance was received.

20. The medium of claim 15, wherein the operations further comprise:

identifying an environmental context associated with receiving the audio data corresponding to the utterance by the user of the predefined hotword;

wherein using the audio data in determining whether audio data corresponding to subsequently received utterances by the user of the predefined hotword are to be accepted or rejected comprises using the environmental context in determining whether the audio data corresponding to the subsequently received utterances by the user of the predefined hotword are to be accepted or rejected.