Hotword detection on multiple devices

• US 10,593,330 B2
• Filed: 10/26/2018
• Issued: 03/17/2020
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US16/171,495
• Publication Number:
  US 20190130914A1
• Time in Patent Office: 508 Days
• Field of Search: 704254, 704275, 704273, 704270, 704257, 704256, 704251, 704250, 704247, 704246, 704244, 704243, 704235, 704233, 704231, 704228, 704224, 704 2, 4554141, 434157, 391 711, 379 8803, 379 8801, 367124
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving, by a first computing device, audio data that corresponds to an utterance. The actions further include determining a first value corresponding to a likelihood that the utterance includes a hotword. The actions further include receiving a second value corresponding to a likelihood that the utterance includes the hotword, the second value being determined by a second computing device. The actions further include comparing the first value and the second value. The actions further include based on comparing the first value to the second value, initiating speech recognition processing on the audio data.

Claims

1. A computer-implemented method comprising:

receiving, by a computing device that is in a low power mode, audio data that includes an utterance of a particular, predefined hotword;

while the computing device remains in the low power mode, and in response to receiving the audio data that includes the utterance of the particular, predefined hotword, transmitting, by the computing device, a message;

while the computing device remains in the low power mode, receiving, by the computing device and from an additional computing device that is in a low power mode, an additional message; and

based on the message and the additional message, determining, by the computing device, to exit the low power mode.

2. The method of claim 1, comprising:

based on the message and the additional message, performing, by the computing device, automated speech recognition processing on the audio data.

3. The method of claim 1, wherein:

the computing device is configured to exit the low power mode in response to detecting an utterance of the particular, predefined hotword, and

the additional computing device is configured to exit the low power mode in response to detecting an utterance of the particular, predefined hotword.

4. The method of claim 1, comprising:

determining, by the computing device, that the audio data includes an utterance of a particular, predefined hotword without performing automated speech recognition processing on the audio data.

5. The method of claim 1, comprising:

determining, by the computing device, a hotword score that reflects a likelihood that the audio data includes an utterance of the particular, predefined hotword,

wherein the message includes the hotword score.

6. The method of claim 1, comprising:

determining, by the computing device, a hotword score that reflects a likelihood that the audio data includes an utterance of the particular, predefined hotword; and

determining, by the computing device, that the hotword score satisfies a threshold,

wherein transmitting the message is based on determining that the hotword score satisfies the threshold.

7. The method of claim 1, comprising:

comparing, by the computing device, the message to the additional message;

determining to exit the low power mode based on comparing the message to the additional message.

8. The method of claim 1, wherein the first computing device transmits the message for a particular amount of time.

9. A system comprising:

one or more computers; and

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a computing device that is in a low power mode, audio data that includes an utterance of a particular, predefined hotword;

while the computing device remains in the low power mode, and in response to receiving the audio data that includes the utterance of the particular, predefined hotword, transmitting, by the computing device, a message;

while the computing device remains in the low power mode, receiving, by the computing device and from an additional computing device that is in a low power mode, an additional message; and

based on the message and the additional message, determining, by the computing device, to exit the low power mode.

10. The system of claim 9, wherein the operations comprise:

based on the message and the additional message, performing, by the computing device, automated speech recognition processing on the audio data.

11. The system of claim 9, wherein:

the computing device is configured to exit the low power mode in response to detecting an utterance of the particular, predefined hotword, and

the additional computing device is configured to exit the low power mode in response to detecting an utterance of the particular, predefined hotword.

12. The system of claim 9, wherein the operations comprise:

determining, by the computing device, that the audio data includes an utterance of a particular, predefined hotword without performing automated speech recognition processing on the audio data.

13. The system of claim 9, wherein the operations comprise:

determining, by the computing device, a hotword score that reflects a likelihood that the audio data includes an utterance of the particular, predefined hotword,

wherein the message includes the hotword score.

14. The system of claim 9, wherein the operations comprise:

determining, by the computing device, a hotword score that reflects a likelihood that the audio data includes an utterance of the particular, predefined hotword; and

determining, by the computing device, that the hotword score satisfies a threshold,

wherein transmitting the message is based on determining that the hotword score satisfies the threshold.

15. The system of claim 9, wherein the operations comprise:

comparing, by the computing device, the message to the additional message;

determining to exit the low power mode based on comparing the message to the additional message.

16. The system of claim 9, wherein the first computing device transmits the message for a particular amount of time.

17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a computing device that is in a low power mode, audio data that includes an utterance of a particular, predefined hotword;

while the computing device remains in the low power mode, and in response to receiving the audio data that includes the utterance of the particular, predefined hotword, transmitting, by the computing device, a message;

while the computing device remains in the low power mode, receiving, by the computing device and from an additional computing device that is in a low power mode, an additional message; and

based on the message and the additional message, determining, by the computing device, to exit the low power mode.

18. The medium of claim 17, wherein the operations comprise:

based on the message and the additional message, performing, by the computing device, automated speech recognition processing on the audio data.