DETERMINING HOTWORD SUITABILITY

• US 20160133259A1
• Filed: 01/20/2016
• Published: 05/12/2016
• Est. Priority Date: 07/03/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rubin, Andrew E., Schalkwyk, Johan, San Martin, Maria Carolina Parada
• Granted Patent: US 10,002,613 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/46 by designing passwords or checking the strength of passwords
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining hotword suitability. In one aspect, a method includes receiving speech data that encodes a candidate hotword spoken by a user, evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, generating a hotword suitability score for the candidate hotword based on evaluating the speech data or a transcription of the candidate hotword, using one or more predetermined criteria, and providing a representation of the hotword suitability score for display to the user.

Claims

1. (canceled)

2. A computer-implemented method comprising:

providing, for output, a prompt for a user to speak a candidate term that is to be designated a custom hotword, wherein a hotword is a term that indicates to a voice command processing subsystem on a mobile device that one or more terms that are uttered immediately after the hotword are to be processed as voice commands;

receiving one or more utterances of the candidate hotword in response to the prompt; and

providing, in response to the one or more utterances of the candidate hotword, an indication whether the candidate hotword has been accepted as a hotword.

3. The computer-implemented method of claim 2,

wherein providing the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, an indication that the candidate hotword has been accepted as a hotword, and

wherein, after providing the indication that the candidate hotword has been accepted as a hotword, the computer-implemented method further comprises:

receiving an utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) a particular set of one or more terms; and

determining that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands.

4. The computer-implemented method of claim 3,

wherein receiving one or more utterances of the candidate hotword in response to the prompt comprises:

while the mobile device is in an activated state, receiving one or more utterances of the candidate hotword in response to the prompt, and

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises;

while the mobile device is in a dormant or idle state, receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms.

5. The computer-implemented method of claim 3,

wherein receiving one or more utterances of the candidate hotword in response to the prompt comprises:

while the mobile device is unlocked, receiving one or more utterances of the candidate hotword in response to the prompt, and

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises;

while the mobile device is locked, receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms.

6. The computer-implemented method of claim 3, further comprising:

after providing the indication that the candidate hotword has been accepted as a hotword, receiving data confirming that the candidate hotword is to be designated as a custom hotword;

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises, receiving, after receiving data confirming that the candidate hotword is to be designated as a custom hotword, the utterance of (i) the candidate hotword having been accepted as a hotword; and

wherein determining that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands comprises determining, based at least on receiving data confirming that the candidate hotword is to be designated as a custom hotword, that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands.

7. The computer-implemented method of claim 6, wherein receiving data confirming that the candidate hotword is to be designated as a custom hotword comprises receiving data indicating user input provided to the mobile device in response to the indication.

8. The computer-implemented method of claim 2, wherein providing the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, (i) an indication that the candidate hotword has not been accepted as a hotword and (ii) a prompt for the user to speak another, different candidate term that is to be designated a custom hotword.

9. The computer-implemented method of claim 2,

wherein providing, for output, the prompt for the user to speak a candidate term that is to be designated a custom hotword comprises:

providing, for output by the mobile device, a prompt for the user to speak a candidate term that is to be designated a custom hotword, and

wherein providing, in response to the one or more utterances of the candidate hotword, the indication of whether the candidate hotword has been accepted as a hotword comprises;

providing, in response to the one or more utterances of the candidate hotword, an indication of whether the candidate hotword has been accepted as a hotword for output by the mobile device.

10. The computer-implemented method of claim 2, wherein providing, in response to the one or more utterances of the candidate hotword, the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, an indication of whether a confidence value that reflects a level of confidence in the candidate hotword representing a suitable hotword has satisfied a predetermined threshold.

11. The computer-implemented method of claim 10, further comprising:

providing, in response to the one or more utterances of the candidate hotword, one or more graphical or textual representations of the confidence value for display on the mobile device.

12. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

providing, for output, a prompt for a user to speak a candidate term that is to be designated a custom hotword, wherein a hotword is a term that indicates to a voice command processing subsystem on a mobile device that one or more terms that are uttered immediately after the hotword are to be processed as voice commands;

receiving one or more utterances of the candidate hotword in response to the prompt; and

providing, in response to the one or more utterances of the candidate hotword, an indication whether the candidate hotword has been accepted as a hotword.

13. The computer program product of claim 12,

wherein providing the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, an indication that the candidate hotword has been accepted as a hotword, and

wherein, after providing the indication that the candidate hotword has been accepted as a hotword, the operations further comprise:

receiving an utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) a particular set of one or more terms; and

determining that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands.

14. The computer program product of claim 13,

wherein receiving one or more utterances of the candidate hotword in response to the prompt comprises:

while the mobile device is in an activated state, receiving one or more utterances of the candidate hotword in response to the prompt, and

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises;

while the mobile device is in a dormant or idle state, receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms.

15. The computer program product of claim 13,

wherein receiving one or more utterances of the candidate hotword in response to the prompt comprises:

while the mobile device is unlocked, receiving one or more utterances of the candidate hotword in response to the prompt, and

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises;

while the mobile device is locked, receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms.

16. The computer program product of claim 13, wherein the operations further comprise:

after providing the indication that the candidate hotword has been accepted as a hotword, receiving data confirming that the candidate hotword is to be designated as a custom hotword;

wherein receiving the utterance of (i) the candidate hotword having been accepted as a hotword, immediately followed by (ii) the particular set of one or more terms comprises, receiving, after receiving data confirming that the candidate hotword is to be designated as a custom hotword, the utterance of (i) the candidate hotword having been accepted as a hotword; and

wherein determining that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands comprises determining, based at least on receiving data confirming that the candidate hotword is to be designated as a custom hotword, that the particular set of one or more terms uttered immediately after the candidate hotword having been accepted as a hotword are to be processed as one or more voice commands.

17. The computer program product of claim 16, wherein receiving data confirming that the candidate hotword is to be designated as a custom hotword comprises receiving data indicating user input provided to the mobile device in response to the indication.

18. The computer program product of claim 12, wherein providing the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, (i) an indication that the candidate hotword has not been accepted as a hotword and (ii) a prompt for the user to speak another, different candidate term that is to be designated a custom hotword.

19. The computer program product of claim 12,

wherein providing, for output, the prompt for the user to speak a candidate term that is to be designated a custom hotword comprises:

providing, for output by the mobile device, a prompt for the user to speak a candidate term that is to be designated a custom hotword, and

wherein providing, in response to the one or more utterances of the candidate hotword, the indication of whether the candidate hotword has been accepted as a hotword comprises;

providing, in response to the one or more utterances of the candidate hotword, an indication of whether the candidate hotword has been accepted as a hotword for output by the mobile device.

20. The computer program product of claim 12, wherein providing, in response to the one or more utterances of the candidate hotword, the indication of whether the candidate hotword has been accepted as a hotword comprises providing, in response to the one or more utterances of the candidate hotword, an indication of whether a confidence value that reflects a level of confidence in the candidate hotword representing a suitable hotword has satisfied a predetermined threshold.

21. The computer program product of claim 20, wherein the operations further comprise:

providing, in response to the one or more utterances of the candidate hotword, one or more graphical or textual representations of the confidence value for display on the mobile device.

22. A computer-implemented method comprising:

receiving a request to designate a particular term that is not currently designated as a hotword, as a hotword;

determining that the particular term satisfies one or more hotword suitability criteria; and

in response to determining that the particular term satisfies one or more hotword suitability criteria, determining whether subsequently received audio data includes sounds that are characteristic of an utterance of the particular term.

23. The computer-implemented method of claim 22, further comprising:

determining that the particular term includes a particular quantity of syllables or phones; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular quantity of syllables or phones, that the particular term satisfies one or more hotword suitability criteria.

24. The computer-implemented method of claim 22, further comprising:

determining that the particular term includes one or more particular phones; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the one or more particular phones included in the particular term, that the particular term satisfies one or more hotword suitability criteria.

25. The computer-implemented method of claim 22, further comprising:

determining that the particular term is a particular edit distance from one or more terms; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular edit distance between the particular term and the one or more terms, that the particular term satisfies one or more hotword suitability criteria.

26. The computer-implemented method of claim 22, further comprising:

determining that the particular term corresponds to a particular quantity of pronunciations; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular quantity of pronunciations, that the particular term satisfies one or more hotword suitability criteria.

27. The computer-implemented method of claim 22, further comprising:

before determining that the particular term satisfies one or more hotword suitability criteria, receiving audio data that encodes an utterance of one or more words, at least a portion of which represent the particular term.

28. The computer-implemented method of claim 27, further comprising:

determining that the particular term is encoded in a particular quantity of frames of the audio data that encodes the utterance of one or more words; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular quantity of frames, that the particular term satisfies one or more hotword suitability criteria.

29. The computer-implemented method of claim 27, wherein receiving audio data that encodes the utterance of one or more words comprises:

receiving audio data that encodes an utterance of one or more words spoken by a user of a client device.

30. The computer-implemented method of claim 29, further comprising:

in response to determining that subsequently received audio data includes sounds that are characteristic of an utterance of the particular term, providing, to an application running on the client device, one or more commands for waking up the client device.

31. The computer-implemented method of claim 29, further comprising:

in response to determining that subsequently received audio data includes sounds that are characteristic of an utterance of the particular term, providing, to an application running on the client device, one or more commands for unlocking the client device.

32. The computer-implemented method of claim 29, further comprising:

in response to determining that subsequently received audio data includes sounds that are characteristic of an utterance of the particular term, providing, to an application running on the client device, one or more commands for applying settings of the user'"'"'s preference.

33. The computer-implemented method of claim 29, further comprising:

in response to determining that the particular term satisfies one or more hotword suitability criteria, providing an indication that the particular term satisfies one or more hotword suitability criteria for output at the client device.

34. The computer-implemented method of claim 29, further comprising:

obtaining additional audio data encoding one or more utterances of the particular term spoken by the user of the client device;

determining, based on the received audio data and the additional audio data, that the user'"'"'s pronunciation of the particular term varies to a particular extent from utterance to utterance; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular extent to which the user'"'"'s pronunciation of the particular word varies from utterance to utterance, that the particular term satisfies one or more hotword suitability criteria.

35. The computer-implemented method of claim 22, further comprising:

determining a confidence value that reflects a level of confidence in the particular term representing a suitable hotword; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based on the confidence value, that the particular term satisfies one or more hotword suitability criteria.

36. The computer-implemented method of claim 35, further comprising:

generating one or more textual or graphical representations of the confidence value that reflects a level of confidence in the particular term representing a suitable hotword; and

providing the one or more textual or graphical representations for output at one or more computing devices.

37. The computer-implemented method of claim 36, further comprising:

generating a textual or graphical indication that the particular term satisfies one or more hotword suitability criteria; and

providing the textual or graphical indication for output at the one or more computing devices.

38. The computer-implemented method of claim 22, wherein determining whether subsequently received audio data includes sounds that are characteristic of an utterance of the particular term comprises:

determining, using a speech recognizer, whether subsequently received audio data includes sounds that are characteristic of an utterance of the particular term.

39. The computer-implemented method of claim 38, further comprising:

determining that a particular quantity of training examples of the particular term are available to the speech recognizer; and

wherein determining that the particular term satisfies one or more hotword suitability criteria, comprises determining, based at least on the particular quantity of training examples, that the particular term satisfies one or more hotword suitability criteria.