HOTWORD RECOGNITION

• US 20170110130A1
• Filed: 06/08/2016
• Published: 04/20/2017
• Est. Priority Date: 10/16/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Foerster, Jakob Nicolaus
• Granted Patent: US 9,928,840 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G07C 9/25 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G07C 9/37 using biometric data, e.g. fingerprints, iris scans or voice recognition
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving audio data corresponding to an utterance, determining that the audio data corresponds to a hotword, generating a hotword audio fingerprint of the audio data that is determined to correspond to the hotword, comparing the hotword audio fingerprint to one or more stored audio fingerprints of audio data that was previously determined to correspond to the hotword, detecting whether the hotword audio fingerprint matches a stored audio fingerprint of audio data that was previously determined to correspond to the hotword based on whether the comparison indicates a similarity between the hotword audio fingerprint and one of the one or more stored audio fingerprints that satisfies a predetermined threshold, and in response to detecting that the hotword audio fingerprint matches a stored audio fingerprint, disabling access to a computing device into which the utterance was spoken.

Claims

1. (canceled)

2. A computer-implemented method comprising:

receiving, by a mobile computing device, an initial audio input corresponding to an utterance of an initial portion of a hotword;

while a remaining portion of the hotword is being uttered and before the hotword is fully uttered, providing, by the mobile computing device, verification audio for output through a speaker of the mobile computing device; and

selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered.

3. The computer-implemented method of claim 2, wherein the verification audio comprises one of a plurality of unique ultrasonic audio samples.

4. The computer-implemented method of claim 2, wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

5. The computer-implemented method of claim 2, further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that one of either the initial audio input corresponding to an utterance of an initial portion of a hotword or the additional audio input corresponding to an utterance of the remaining portion of a hotword comprises audio data corresponding to verification audio different than the verification audio provided for output through the speaker of the mobile computing device; and

disabling access to one or more resources of the mobile computing device.

6. The computer-implemented method of claim 5, wherein disabling access to one or more resources of the mobile computing device comprises one or more of:

preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

7. The computer-implemented method of claim 2, further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword,

wherein selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered is based at least in part on determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword.

8. The computer-implemented method of claim 2, wherein the verification audio is stored in a database in association with the initial audio input corresponding to an utterance of an initial portion of a hotword.

9. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving, by a mobile computing device, an initial audio input corresponding to an utterance of an initial portion of a hotword;

while a remaining portion of the hotword is being uttered and before the hotword is fully uttered, providing, by the mobile computing device, verification audio for output through a speaker of the mobile computing device; and

selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered.

10. The system of claim 9, wherein the verification audio comprises one of a plurality of unique ultrasonic audio samples.

11. The system of claim 9, wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

12. The system of claim 9, the operations further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that one of either the initial audio input corresponding to an utterance of an initial portion of a hotword or the additional audio input corresponding to an utterance of the remaining portion of a hotword comprises audio data corresponding to verification audio different than the verification audio provided for output through the speaker of the mobile computing device; and

disabling access to one or more resources of the mobile computing device.

13. The system of claim 12, wherein disabling access to one or more resources of the mobile computing device comprises one or more of:

preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

14. The system of claim 9, the operations further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword,

wherein selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered is based at least in part on determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword.

15. The system of claim 9, wherein the verification audio is stored in a database in association with the initial audio input corresponding to an utterance of an initial portion of a hotword.

16. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a mobile computing device, an initial audio input corresponding to an utterance of an initial portion of a hotword;

while a remaining portion of the hotword is being uttered and before the hotword is fully uttered, providing, by the mobile computing device, verification audio for output through a speaker of the mobile computing device; and

selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered.

17. The computer-readable storage device of claim 16, wherein the verification audio comprises one of a plurality of unique ultrasonic audio samples.

18. The computer-readable storage device of claim 16, wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

19. The computer-readable storage device of claim 16, the operations further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that one of either the initial audio input corresponding to an utterance of an initial portion of a hotword or the additional audio input corresponding to an utterance of the remaining portion of a hotword comprises audio data corresponding to verification audio different than the verification audio provided for output through the speaker of the mobile computing device; and

disabling access to one or more resources of the mobile computing device.

20. The computer-readable storage device of claim 19, wherein disabling access to one or more resources of the mobile computing device comprises one or more of:

preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

21. The computer-readable storage device of claim 16, the operations further comprising:

receiving, by the mobile computing device, an additional audio input corresponding to an utterance of the remaining portion of a hotword;

determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword,

wherein selectively enabling access to one or more resources of the mobile computing device after the hotword is fully uttered is based at least in part on determining that the initial audio input corresponding to an utterance of an initial portion of a hotword and the additional audio input corresponding to an utterance of the remaining portion of a hotword, collectively, correspond to the entire hotword.