Initiating actions based on partial hotwords

• US 9,805,719 B2
• Filed: 03/23/2017
• Issued: 10/31/2017
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US15/466,979
• Publication Number:
  US 20170193998A1
• Time in Patent Office: 222 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, receiving audio data; determining that an initial portion of the audio data corresponds to an initial portion of a hotword; in response to determining that the initial portion of the audio data corresponds to the initial portion of the hotword, selecting, from among a set of one or more actions that are performed when the entire hotword is detected, a subset of the one or more actions; and causing one or more actions of the subset to be performed.

Claims

1. A computer-implemented method comprising:

receiving audio data;

determining, by a component of a mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of an entire predetermined activation term, that the audio data encodes a likely utterance of only a prefix predetermined activation term; and

in response to determining, by the component of the mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of the entire predetermined activation term, that the audio data encodes the likely utterance of a prefix only of the predetermined activation term, activating a deactivated hardware component of the mobile computing device.

2. The method of claim 1, wherein determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more prefixes of predetermined activation terms stored in a database, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

3. The method of claim 1, wherein determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more prefixes of predetermined activation terms, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

4. The method of claim 1, wherein activating the deactivated hardware component of the mobile computing device comprises providing an interrupt signal that initializes the deactivated hardware component of the mobile computing device.

5. The method of claim 1, wherein activating the deactivated hardware component of the mobile computing device comprises:

selecting one or more actions that are associated with only the prefix of the predetermined activation term; and

causing the one or more actions to be performed.

6. The method of claim 5, wherein the one or more actions include one or more actions whose respective results are not discernible to a user of the computing device performing the one or more actions.

7. The method of claim 1, comprising:

determining, by the component of a mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of an entire predetermined activation term, that the audio data encodes a likely utterance of the entire predetermined activation term; and

after determining that the audio data encodes a likely utterance of the entire predetermined activation term, determining that the entire predetermined activation term includes a particular activation term that triggers semantic interpretation of an additional term of one or more terms that follow the particular activation term.

8. A system comprising:

one or more processors and one or more storage device storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising;

receiving audio data;

determining, by a component of a mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of an entire predetermined activation term, that the audio data encodes a likely utterance of only a prefix of the predetermined activation term; and

in response to determining, by the component of the mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of the entire predetermined activation term, that the audio data encodes the likely utterance of only a prefix of the predetermined activation term, activating a deactivated hardware component of the mobile computing device.

9. The system of claim 8, wherein the operation of determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more prefixes of predetermined activation terms stored in a database, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

10. The system of claim 8, wherein the operation of determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more prefixes of predetermined activation terms, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

11. The system of claim 8, wherein the operation of activating the deactivated hardware component of the mobile computing device comprises providing an interrupt signal that initializes the deactivated hardware component of the mobile computing device.

12. The system of claim 8, wherein the operation of activating the deactivated hardware component of the mobile computing device comprises:

selecting one or more actions that are associated with only the prefix of the predetermined activation term; and

causing the one or more actions to be performed.

13. The system of claim 12, wherein the one or more actions include one or more actions whose respective results are not discernible to a user of the computing device performing the one or more actions.

14. The system of claim 8, wherein the operations comprise:

determining, by the component of a mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of an entire predetermined activation term, that the audio data encodes a likely utterance of the entire predetermined activation term; and

after determining that the audio data encodes a likely utterance of the entire predetermined activation term, determining that the entire predetermined activation term includes a particular activation term that triggers semantic interpretation of an additional term of one or more terms that follow the particular activation term.

15. A non-transitory computer-readable medium storing instructions executable by one or more computers which, upon execution, cause the one or more computers to perform operations comprising:

receiving audio data;

determining, by a component of a mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of an entire predetermined activation term, that the audio data encodes a likely utterance of only a prefix of the predetermined activation term; and

in response to determining, by the component of the mobile computing device that remains activated when the mobile computing device is in a low power state and before the component determines whether the audio data encodes a likely utterance of the entire predetermined activation term, that the audio data encodes the likely utterance of only a prefix of the predetermined activation term, activating a deactivated hardware component of the mobile computing device.

16. The non-transitory computer-readable medium of claim 15, wherein the operation of determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

identifying one or more acoustic features of the audio data;

comparing the one or more acoustic features of the audio data to one or more acoustic features associated with one or more prefixes of predetermined activation terms stored in a database, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

17. The non-transitory computer-readable medium of claim 15, wherein the operation of determining that the audio data encodes a likely utterance of only the prefix of the predetermined activation term comprises:

obtaining a transcription of the audio data;

comparing the transcription of the audio data to one or more transcriptions associated with one or more prefixes of predetermined activation terms, the one or more prefixes of predetermined activation terms including only the prefix of the predetermined activation term; and

based on the comparing, determining that the audio data corresponds to only the prefix of the predetermined activation term.

18. The non-transitory computer-readable medium of claim 15, wherein the operation of activating the deactivated hardware component of the mobile computing device comprises providing an interrupt signal that initializes the deactivated hardware component of the mobile computing device.

19. The non-transitory computer-readable medium of claim 15, wherein the operation of activating the deactivated hardware component of the mobile computing device comprises:

selecting one or more actions that are associated with only the prefix of the predetermined activation term; and

causing the one or more actions to be performed.

20. The non-transitory computer-readable medium of claim 19, wherein the one or more actions include one or more actions whose respective results are not discernible to a user of the computing device performing the one or more actions.