Initiating actions based on partial hotwords

• US 9,502,026 B2
• Filed: 01/08/2016
• Issued: 11/22/2016
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US14/991,092
• Publication Number:
  US 20160125875A1
• Time in Patent Office: 319 Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, receiving audio data; determining that an initial portion of the audio data corresponds to an initial portion of a hotword; in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and causing one or more actions of the subset to be performed.

Claims

1. A computer-implemented method comprising:

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a predefined phrase that is associated with waking a mobile device from a stand-by state;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, selecting a set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by state;

causing the one or more actions of the set to be performed;

after determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, determining that a remaining portion of the audio data corresponds to a remaining portion of the predefined phrase;

in response to determining that the remaining portion of the audio data corresponds to the remaining portion of the predefined phrase, selecting additional actions of the set of actions that are associated with waking the mobile device from the stand-by state; and

causing the additional actions of the set of actions to be performed.

2. The computer-implemented method of claim 1, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more predefined phrases stored in a database, the one or predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

3. The computer-implemented method of claim 1, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more predefined phrases stored in a database, the one or more predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

4. The computer-implemented method of claim 1, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises determining that the audio data corresponds to a full predefined phrase that is associated with waking a mobile device from a stand-by state.

5. The computer-implemented method of claim 1, wherein the set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by state include actions whose respective results are not discernible to a user of a computing device performing the actions.

6. The computer-implemented method of claim 1, wherein the predefined phrase comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

7. A system comprising:

one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising;

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a predefined phrase that is associated with waking a mobile device from a stand-by state;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, selecting a set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by state;

causing the one or more actions of the set to be performed;

after determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, determining that a remaining portion of the audio data corresponds to a remaining portion of the predefined phrase;

in response to determining that the remaining portion of the audio data corresponds to the remaining portion of the predefined phrase, selecting additional actions of the set of actions that are associated with waking the mobile device from the stand-by state; and

causing the additional actions of the set of actions to be performed.

8. The system of claim 7, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more predefined phrases stored in a database, the one or predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

9. The system of claim 7, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more predefined phrases stored in a database, the one or more predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

10. The system of claim 7, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises determining that the audio data corresponds to a full predefined phrase that is associated with waking a mobile device from a stand-by state.

11. The system of claim 7, wherein the set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by-state include actions whose respective results are not discernible to a user of a computing device performing the actions.

12. The system of claim 7, wherein the predefined phrase comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

13. A non-transitory computer-readable medium storing instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving audio data;

determining that an initial portion of the audio data corresponds to an initial portion of a predefined phrase that is associated with waking a mobile device from a stand-by state;

in response to determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, selecting a set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by-state;

causing the one or more actions of the set to be performed;

after determining that the initial portion of the audio data corresponds to the initial portion of the predefined phrase that is associated with waking the mobile device from the stand-by state, determining that a remaining portion of the audio data corresponds to a remaining portion of the predefined phrase;

in response to determining that the remaining portion of the audio data corresponds to the remaining portion of the predefined phrase, selecting additional actions of the set of actions that are associated with waking the mobile device from the stand-by state; and

causing the additional actions of the set of actions to be performed.

14. The computer-readable medium of claim 13, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more predefined phrases stored in a database, the one or predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

15. The computer-readable medium of claim 13, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more predefined phrases stored in a database, the one or more predefined phrases including the predefined phrase that is associated with waking a mobile device from a stand-by state; and

based on the comparing, determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state of the one or more predefined phrases stored in the database.

16. The computer-readable medium of claim 13, wherein determining that the audio data corresponds to the predefined phrase that is associated with waking a mobile device from a stand-by state comprises determining that the audio data corresponds to a full predefined phrase that is associated with waking a mobile device from a stand-by state.

17. The computer-readable medium of claim 13, wherein the set of one or more actions that have been pre-associated with the initial portion of the predefined phrase and that are a subset of a set of actions that are associated with waking the mobile device from the stand-by-state include actions whose respective results are not discernible to a user of a computing device performing the actions.

18. The computer-readable medium of claim 13, wherein the predefined phrase comprises a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.