Hotword detection on multiple devices

• US 10,714,093 B2
• Filed: 03/22/2019
• Issued: 07/14/2020
• Est. Priority Date: 08/24/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Melendo Casado, Diego, Gruenstein, Alexander H., Foerster, Jakob Nicolaus
• Primary Examiner(s): Vo, Huyen X
• Application Number:
  US16/362,284
• Publication Number:
  US 20190287534A1
• Time in Patent Office: 480 Days
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H05K 999/99 dummy group

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving audio data that corresponds to an utterance. The actions further include determining that the utterance likely includes a particular, predefined hotword. The actions further include transmitting (i) data indicating that the computing device likely received the particular, predefined hotword, (ii) data identifying the computing device, and (iii) data identifying a group of nearby computing devices that includes the computing device. The actions further include receiving an instruction to commence speech recognition processing on the audio data. The actions further include in response to receiving the instruction to commence speech recognition processing on the audio data, processing at least a portion of the audio data using an automated speech recognizer on the computing device.

Claims

1. A computer-implemented method comprising:

receiving, by a first computing device, from a second computing device that is in a vicinity of the first computing device, data indicating that the second computing device is configured to respond to a particular, predefined hotword;

receiving, by the first computing device, audio data of an utterance that includes a particular, predefined hotword;

based on receiving, from the second computing device that is in the vicinity of the first computing device, the data indicating that the second computing device is configured to respond to the particular, predefined hotword, providing, to a third computing device, data indicating that the first computing device received the particular, predefined hotword, data indicating that the second computing device is in the vicinity of the first computing device, and the data indicating that the second computing device is configured to respond to the particular, predefined hotword; and

based on receiving, from the third computing device, an instruction to perform a command that follows the particular, predefined hotword, performing, by the first computing device, the command.

2. The method of claim 1, further comprising:

determining, by the first computing device, that the utterance likely includes the particular, predefined hotword,

wherein providing the data indicating that the first computing device received the particular, predefined hotword and the data indicating that the second computing device is in the vicinity of the first computing device and configured to respond to the particular, predefined hotword is further based on determining that the utterance likely includes the particular, predefined hotword.

3. The method of claim 1, wherein the first computing device is configured to respond to the particular, predefined hotword.

4. The method of claim 1, further comprising, based on the second computing device that is configured to respond to the particular, predefined hotword being in a vicinity of the first computing device, providing, by the first computing device, data identifying the first computing device.

5. The method of claim 1, wherein the third computing device transmits, to the second computing device, an instruction to suppress performance of the command that follows the particular, predefined hotword.

6. The method of claim 1, wherein:

the first computing device includes a first microphone that receives the audio data of the utterance, and

the second computing device includes a second microphone that receives the audio data of the utterance.

7. The method of claim 1, wherein the third computing device does not include a microphone.

8. The method of claim 1, wherein the third computing device is not configured to respond to the particular, predefined hotword.

9. The method of claim 1, wherein the first computing device is configured to respond to the particular, predefined hotword.

10. A system comprising:

one or more computers; and

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a first computing device, from a second computing device that is in a vicinity of the first computing device, data indicating that the second computing device is configured to respond to a particular, predefined hotword;

receiving, by the first computing device, audio data of an utterance that includes a particular, predefined hotword;

based on receiving, from the second computing device that is in the vicinity of the first computing device, the data indicating that the second computing device is configured to respond to the particular, predefined hotword, providing, to a third computing device, data indicating that the first computing device received the particular, predefined hotword, data indicating that the second computing device is in the vicinity of the first computing device, and the data indicating that the second computing device is configured to respond to the particular, predefined hotword; and

based on receiving, from the third computing device, an instruction to perform a command that follows the particular, predefined hotword, performing, by the first computing device, the command.

11. The system of claim 10, wherein the operations further comprise:

determining, by the first computing device, that the utterance likely includes the particular, predefined hotword,

wherein providing the data indicating that the first computing device received the particular, predefined hotword and the data indicating that the second computing device is in the vicinity of the first computing device and configured to respond to the particular, predefined hotword is further based on determining that the utterance likely includes the particular, predefined hotword.

12. The system of claim 10, wherein the operations further comprise, based on the second computing device that is configured to respond to the particular, predefined hotword being in a vicinity of the first computing device, providing, by the first computing device, data identifying the first computing device.

13. The system of claim 10, wherein the third computing device transmits, to the second computing device, an instruction to suppress performance of the command that follows the particular, predefined hotword.

14. The system of claim 10, wherein:

the first computing device includes a first microphone that receives the audio data of the utterance, and

the second computing device includes a second microphone that receives the audio data of the utterance.

15. The system of claim 10, wherein the third computing device does not include a microphone and is not configured to respond to the particular, predefined hotword.

16. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a first computing device, from a second computing device that is in a vicinity of the first computing device, data indicating that the second computing device is configured to respond to a particular, predefined hotword;

receiving, by the first computing device, audio data of an utterance that includes a particular, predefined hotword;

based on receiving, from the second computing device that is in the vicinity of the first computing device, the data indicating that the second computing device is configured to respond to the particular, predefined hotword, providing, to a third computing device, data indicating that the first computing device received the particular, predefined hotword, data indicating that the second computing device is in the vicinity of the first computing device, and the data indicating that the second computing device is configured to respond to the particular, predefined hotword; and

based on receiving, from the third computing device, an instruction to perform a command that follows the particular, predefined hotword, performing, by the first computing device, the command.