Hotword detection on multiple devices

• US 9,424,841 B2
• Filed: 03/17/2015
• Issued: 08/23/2016
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Foerster, Jakob Nicolaus, Gruenstein, Alexander H.
• Primary Examiner(s): Hang, Vu B
• Application Number:
  US14/659,861
• Publication Number:
  US 20160104483A1
• Time in Patent Office: 525 Days
• Field of Search: 704/231, 704/240, 704/270, 704/275, 704/251
• US Class Current: 1/1
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving, by a computing device, audio data that corresponds to an utterance. The actions further include determining a likelihood that the utterance includes a hotword. The actions further include determining a loudness score for the audio data. The actions further include based on the loudness score, determining an amount of delay time. The actions further include, after the amount of delay time has elapsed, transmitting a signal that indicates that the computing device will initiate speech recognition processing on the audio data.

Claims

1. A computer-implemented method comprising:

receiving, by one or more processors of a computing device, audio data that corresponds to an utterance;

determining that the utterance likely includes a particular, predefined hotword;

in response to determining that the utterance likely includes the particular, predefined hotword, determining score that reflects a loudness of the audio data;

determining a duration of a delay period, wherein the duration of the delay period is inversely proportional to the loudness of the audio data;

activating a mode in which the computing device temporarily listens, for the duration of the delay period, for a predetermined audio signal that indicates that another computing device is commencing speech recognition processing on the audio data;

after the duration of the delay period has elapsed without hearing the predetermined audio signal from another computing device, deactivating the mode and transmitting the predetermined audio signal that indicates that the computing device is commencing speech recognition processing on the audio data; and

after transmitting the predetermined audio signal, processing at least a portion of the audio data using an automated speech recognizer on the computing device.

2. The method of claim 1, comprising:

receiving, by the one or more processors of the computing device, additional audio data that corresponds to an additional utterance;

determining that the additional utterance likely includes the particular, predefined hotword;

in response to determining that the additional utterance likely includes the particular, predefined hotword, determining a second score that reflects a second loudness of the additional audio data;

determining a second duration of a second delay period, wherein the second duration of the second delay period is inversely proportional to the second loudness of the additional audio data;

activating the mode in which the computing device temporarily listens, for the second duration of the second delay period, for the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data;

before the second duration of a second delay period has elapsed, receiving the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data; and

in response to receiving the predetermined audio signal;

deactivating processing of the additional audio data.

3. The method of claim 2, comprising:

based on receiving the predetermined audio signal, determining that an activation state of the computing device is an inactive state.

4. The method of claim 1, comprising:

after transmitting the predetermined audio signal, determining that an activation state of the computing device is an active state.

5. The method of claim 1, wherein the predetermined audio signal comprises an ultrasonic signal.

6. The method of claim 1, wherein the predetermined audio signal is received by another computing device and indicates to the other computing device to not commence speech recognition processing on the audio data.

7. The method of claim 1, comprising:

based on determining that the utterance likely includes a particular, predefined hotword, preparing to receive the predetermined audio signal that indicates that another computing device will commence speech recognition processing on the audio data.

8. The method of claim 1, wherein the duration of the delay period is zero when the score that reflects a loudness of the audio data satisfies a threshold.

9. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a computing device, audio data that corresponds to an utterance;

determining that the utterance likely includes a particular, predefined hotword;

in response to determining that the utterance likely includes the particular, predefined hotword, determining score that reflects a loudness of the audio data;

determining a duration of a delay period, wherein the duration of the delay period is inversely proportional to the loudness of the audio data;

activating a mode in which the computing device temporarily listens, for the duration of the delay period, for a predetermined audio signal that indicates that another computing device is commencing speech recognition processing on the audio data;

after the duration of the delay period has elapsed without hearing the predetermined audio signal from another computing device, deactivating the mode and transmitting the predetermined audio signal that indicates that the computing device is commencing speech recognition processing on the audio data; and

after transmitting the predetermined audio signal, processing at least a portion of the audio data using an automated speech recognizer on the computing device.

10. The system of claim 9, wherein the operations further comprise:

receiving, by the computing device, additional audio data that corresponds to an additional utterance;

determining that the additional utterance likely includes the particular, predefined hotword;

in response to determining that the additional utterance likely includes the particular, predefined hotword, determining a second score that reflects a second loudness of the additional audio data;

determining a second duration of a second delay period, wherein the second duration of the second delay period is inversely proportional to the second loudness of the additional audio data;

activating the mode in which the computing device temporarily listens, for the second duration of the second delay period, for the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data;

before the second duration of a second delay period has elapsed, receiving the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data; and

in response to receiving the predetermined audio signal;

deactivating processing of the additional audio data.

11. The system of claim 9, wherein the operations further comprise:

based on receiving the predetermined audio signal, determining that an activation state of the computing device is an inactive state.

12. The system of claim 9, wherein the predetermined audio signal comprises an ultrasonic signal.

13. The system of claim 9, wherein the predetermined audio signal is received by another computing device and indicates to the other computing device to not commence speech recognition processing on the audio data.

14. The system of claim 9, wherein the operations further comprise:

based on determining that the utterance likely includes a particular, predefined hotword, preparing to receive the predetermined audio signal that indicates that another computing device will commence speech recognition processing on the audio data.

15. The system of claim 9, wherein the duration of the delay period is zero when the score that reflects a loudness of the audio data satisfies a threshold.

16. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a computing device, audio data that corresponds to an utterance;

determining that the utterance likely includes a particular, predefined hotword;

in response to determining that the utterance likely includes the particular, predefined hotword, determining score that reflects a loudness of the audio data;

determining a duration of a delay period, wherein the duration of the delay period is inversely proportional to the loudness of the audio data;

activating a mode in which the computing device temporarily listens, for the duration of the delay period, for a predetermined audio signal that indicates that another computing device is commencing speech recognition processing on the audio data;

after the duration of the delay period has elapsed without hearing the predetermined audio signal from another computing device, deactivating the mode and transmitting the predetermined audio signal that indicates that the computing device is commencing speech recognition processing on the audio data; and

after transmitting the predetermined audio signal, processing at least a portion of the audio data using an automated speech recognizer on the computing device.

17. The medium of claim 16, wherein the operations further comprise:

receiving, by the computing device, additional audio data that corresponds to an additional utterance;

determining that the additional utterance likely includes the particular, predefined hotword;

in response to determining that the additional utterance likely includes the particular, predefined hotword, determining a second score that reflects a second loudness of the additional audio data;

determining a second duration of a second delay period, wherein the second duration of the second delay period is inversely proportional to the second loudness of the additional audio data;

activating the mode in which the computing device temporarily listens, for the second duration of the second delay period, for the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data;

before the second duration of a second delay period has elapsed, receiving the predetermined audio signal that indicates that the other computing device is commencing speech recognition processing on the additional audio data; and

in response to receiving the predetermined audio signal;

deactivating processing of the additional audio data.

18. The medium of claim 16, wherein the predetermined audio signal comprises an ultrasonic signal.

19. The medium of claim 16, wherein the duration of the delay period is zero when the score that reflects a loudness of the audio data satisfies a threshold.

20. The medium of claim 16, wherein the predetermined audio signal is received by another computing device and indicates to the other computing device to not commence speech recognition processing on the audio data.