Initiating actions based on partial hotwords

• US 8,768,712 B1
• Filed: 12/04/2013
• Issued: 07/01/2014
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US14/096,359
• Time in Patent Office: 209 Days
• Field of Search: 704/270, 704/270.1, 704/275
• US Class Current: 704/275
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, receiving audio data; determining that an initial portion of the audio data corresponds to an initial portion of a hotword; in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and causing one or more actions of the subset to be performed.

Claims

1. A computer-implemented method comprising:

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a hotword;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, by one or more computers, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and

causing one or more actions of the subset to be performed.

2. The computer-implemented method of claim 1, wherein determining comprises:

identifying one or more acoustic features of the initial portion of the audio data;

comparing the one or more acoustic features of the initial portion of the audio data to one or more acoustic features associated with one or more hotwords stored in a database; and

based on the comparing, determining that the initial portion of the audio data corresponds to the initial portion of the hotword of the one or more hotwords stored in the database.

3. The computer-implemented method of claim 1, wherein determining comprises:

obtaining a transcription of the initial portion of the audio data, the audio data including a speech utterance;

comparing the transcription of the initial portion of the audio to one or more transcriptions associated with one or more hotwords stored in a database; and

based on the comparing, determining that the initial portion of the audio corresponding to the initial portion of the hotword of the one or more hotwords stored in the database.

4. The computer-implemented method of claim 1, wherein the initial portion of the hotword includes less than the entire hotword.

5. The computer-implemented method of claim 1, wherein the one or more actions of the subset include actions that are not visible to a user of a computing device performing the actions.

6. The computer-implemented method of claim 1, wherein the one or more actions of the subset of includes actions that are performed in a background computing environment of a computing device performing the actions.

7. The computer-implemented method of claim 1, wherein causing the one or more actions of the subset to be performed include transmitting an interrupt signal to an operating system executing on a computing device to perform the one or more actions.

8. The computer-implemented method of claim 7, wherein the interrupt signal causes the operating system to initialize a camera functionality of the computing device.

9. The computer-implemented method of claim 7, wherein the interrupt signal causes the operating system to activate the computing device from a sleep state.

10. The computer-implemented method of claim 7, wherein the interrupt signal causes the operating system to initialize a network connection on the computer device.

11. The computer-implemented method of claim 7, wherein the interrupt signal causes the operating system to preload an application on the computing device associated with the hotword.

12. The computer-implemented method of claim 1, further comprising:

receiving a subsequent portion of the audio data;

determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively correspond to the entire hotword;

in response to determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively, correspond to the entire hotword, selecting, from among the set of one or more actions that are performed when the entire hotword is detected, an additional subset of the one or more actions; and

causing the one or more actions of the additional subset to be performed.

13. The computer-implemented method of claim 12, wherein the set of the one or more actions includes the subset of the one or more actions and the additional subset of the one more actions.

14. The computer-implemented method of claim 1, further comprising:

receiving a subsequent portion of the audio data;

determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively, do not correspond to the entire hotword; and

in response to determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively, do not correspond to the entire hotword, causing the one or more actions of the subset to be reversed.

15. The computer-implemented method of claim 14, wherein causing the one or more actions of the subset to be reversed includes:

selecting an additional subset of one or more actions that when performed, reverse the one or more actions of the subset of the one or more actions; and

causing the one or more actions of the addition subset to be performed.

16. The computer-implemented method of claim 1, further comprising:

prior to receiving the audio data, receiving (i) data indicating the initial portion of the audio data corresponds to the initial portion of the hotword and (ii) data indicating the subset of the actions associated with the hotword.

17. The computer-implemented method of claim 1, wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

18. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a hotword;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and

causing one or more actions of the subset to be performed.

19. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a hotword;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and

causing one or more actions of the subset to be performed.

20. The computer-readable storage device of claim 19, the operations further comprising:

receiving a subsequent portion of the audio data;

determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively correspond to the entire hotword;

in response to determining that the initial portion of the audio data and the subsequent portion of the audio data, collectively, correspond to the entire hotword, selecting, from among the set of one or more actions that are performed when the entire hotword is detected, an additional subset of the one or more actions; and

causing the one or more actions of the additional subset to be performed.