Initiating actions based on partial hotwords

• US 9,508,342 B2
• Filed: 06/23/2016
• Issued: 11/29/2016
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US15/191,046
• Publication Number:
  US 20160300568A1
• Time in Patent Office: 159 Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, receiving audio data; determining that an initial portion of the audio data corresponds to an initial portion of a hotword; in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and causing one or more actions of the subset to be performed.

Claims

1. A computer-implemented method comprising:

receiving, by a mobile computing device that is operating in a sleep mode, sounds corresponding to a beginning portion of a predetermined hotword that is associated with waking the mobile computing device to an awake mode; and

before receiving sounds corresponding to a remaining portion of the predetermined hotword that is associated with waking the mobile computing device to an awake mode, initiating, by the mobile computing device, processing to begin waking the mobile computing device to the awake mode without initiating processing to complete waking the mobile computing device to the awake mode.

2. The computer-implemented method of claim 1, further comprising:

receiving sounds corresponding to the remaining portion of the predetermined hotword that is associated with waking the mobile computing device to the awake mode;

determining that the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword;

in response to determining that the beginning portion of a predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to complete waking the mobile computing device to the awake mode.

3. The computer-implemented method of claim 1, further comprising:

receiving additional sounds;

determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword; and

in response to determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to return the mobile computing device to the sleep mode.

4. The computer-implemented method of claim 1, wherein the awake mode comprises one or more of:

activating a display of the mobile computing device, initializing a camera functionality of the mobile computing device, initializing a network connection on the mobile computing device, preloading an application on the mobile computing device, and displaying a user interface on a display of the mobile computing device.

5. The computer-implemented method of claim 1, wherein the processing to begin waking the mobile computing device to the awake mode comprises processing that is not discernible to a user of the mobile computing device.

6. The computer-implemented method of claim 1, wherein the predetermined hotword comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

7. The computer-implemented method of claim 2, wherein determining that the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword comprises at least one of:

(i) comparing one or more acoustic features of the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword to one or more acoustic features associated with one or more hotwords stored in a database, or (ii) comparing a transcription of the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword to one or more transcriptions associated with one or more hotwords stored in a database.

8. A system comprising:

one or more processors; and

one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising;

receiving, by a mobile computing device that is operating in a sleep mode, sounds corresponding to a beginning portion of a predetermined hotword that is associated with waking the mobile computing device to an awake mode; and

before receiving sounds corresponding to a remaining portion of the predetermined hotword that is associated with waking the mobile computing device to an awake mode, initiating, by the mobile computing device, processing to begin waking the mobile computing device to the awake mode without initiating processing to complete waking the mobile computing device to the awake mode.

9. The system of claim 8, the operations further comprising:

receiving sounds corresponding to the remaining portion of the predetermined hotword that is associated with waking the mobile computing device to the awake mode;

determining that the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword;

in response to determining that the beginning portion of a predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to complete waking the mobile computing device to the awake mode.

10. The system of claim 8, the operations further comprising:

receiving additional sounds;

determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword; and

in response to determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to return the mobile computing device to the sleep mode.

11. The system of claim 8, wherein the awake mode comprises one or more of:

activating a display of the mobile computing device, initializing a camera functionality of the mobile computing device, initializing a network connection on the mobile computing device, preloading an application on the mobile computing device, and displaying a user interface on a display of the mobile computing device.

12. The system of claim 8, wherein the processing to begin waking the mobile computing device to the awake mode comprises processing that is not discernible to a user of the mobile computing device.

13. The system of claim 8, wherein the predetermined hotword comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

14. The system of claim 9, wherein determining that the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword comprises at least one of:

(i) comparing one or more acoustic features of the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword to one or more acoustic features associated with one or more hotwords stored in a database, or (ii) comparing a transcription of the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword to one or more transcriptions associated with one or more hotwords stored in a database.

15. A non-transitory computer-readable storage device storing instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a mobile computing device that is operating in a sleep mode, sounds corresponding to a beginning portion of a predetermined hotword that is associated with waking the mobile computing device to an awake mode; and

before receiving sounds corresponding to a remaining portion of the predetermined hotword that is associated with waking the mobile computing device to an awake mode, initiating, by the mobile computing device, processing to begin waking the mobile computing device to the awake mode without initiating processing to complete waking the mobile computing device to the awake mode.

16. The non-transitory computer-readable storage device of claim 15, the operations further comprising:

receiving sounds corresponding to the remaining portion of the predetermined hotword that is associated with waking the mobile computing device to the awake mode;

determining that the beginning portion of the predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword;

in response to determining that the beginning portion of a predetermined hotword and the subsequent portion of the predetermined hotword, collectively correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to complete waking the mobile computing device to the awake mode.

17. The non-transitory computer-readable storage device of claim 15, the operations further comprising:

receiving additional sounds;

determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword; and

in response to determining that the sounds corresponding to the beginning portion of the predetermined hotword and the additional sounds, collectively do not correspond to the entire predetermined hotword, initiating, by the mobile computing device, processing to return the mobile computing device to the sleep mode.

18. The non-transitory computer-readable storage device of claim 15, wherein the awake mode comprises one or more of:

activating a display of the mobile computing device, initializing a camera functionality of the mobile computing device, initializing a network connection on the mobile computing device, preloading an application on the mobile computing device, and displaying a user interface on a display of the mobile computing device.

19. The non-transitory computer-readable storage device of claim 15, wherein the processing to begin waking the mobile computing device to the awake mode comprises processing that is not discernible to a user of the mobile computing device.

20. The non-transitory computer-readable storage device of claim 15, wherein the predetermined hotword comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.