Hotword detection on multiple devices

• US 9,514,752 B2
• Filed: 04/01/2016
• Issued: 12/06/2016
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US15/088,477
• Publication Number:
  US 20160217790A1
• Time in Patent Office: 249 Days
• Field of Search: 704/254, 704/275, 704/273, 704/270, 704/257, 704/256, 704/251, 704/250, 704/247, 704/246, 704/244, 704/243, 704/235, 704/233, 704/231, 704/228, 704/224, 704/2, 455/414.1, 434/157, 391/71.1, 379/88.03, 379/88.01, 367/124
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for hotword detection on multiple devices are disclosed. In one aspect, a method includes the actions of receiving, by a first computing device, audio data that corresponds to an utterance. The actions further include determining a first value corresponding to a likelihood that the utterance includes a hotword. The actions further include receiving a second value corresponding to a likelihood that the utterance includes the hotword, the second value being determined by a second computing device. The actions further include comparing the first value and the second value. The actions further include based on comparing the first value to the second value, initiating speech recognition processing on the audio data.

Claims

1. A computer-implemented method comprising:

receiving, by a first computing device that (i) is in sleep mode and (ii) is configured to disable the sleep mode and respond to voice commands upon receipt of a particular, predefined hotword, audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword;

while the first computing device remains in the sleep mode, and in response to receiving the audio data that corresponds to the utterance of a voice command that is preceded by the particular, predefined hotword, transmitting a message by the first computing device to a second computing device that (i) is in short-range electronic communication with the first computing device, and (ii) is also configured to respond to voice commands that are preceded by the particular, predefined hotword; and

determining, by the first computing device, to remain in the sleep mode and not respond to the voice command despite receiving the audio data that corresponds to the utterance of the voice command that is preceded by the particular predefined hotword, based at least on transmitting the message.

2. The method of claim 1, comprising:

receiving an additional message from the second computing device,

wherein determining to remain in the sleep mode and not respond to the voice command is based at least in part on the additional message from the second computing device.

3. The method of claim 1, comprising:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword,

wherein the message includes the hotword score.

4. The method of claim 1, comprising:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword; and

determining that the hotword score satisfies a threshold,

wherein transmitting the message is based on determining that the hotword score satisfies the threshold.

5. The method of claim 1, wherein the first computing device transmits the message without performing speech recognition on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword.

6. The method of claim 1, comprising:

determining that the second computing device is configured to respond to voice commands that are preceded by the particular, predefined hotword,

wherein transmitting the message is based on determining that the second computing device is configured to respond to voice commands that are preceded by the particular, predefined hotword.

7. The method of claim 1, wherein the first computing device transmits the message for a particular amount of time.

8. The method of claim 1, comprising:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword;

receiving, from the second computing device, an additional message that includes an additional hotword score that is determined by the second computing device; and

comparing the hotword score with the additional hotword score,

wherein determining to remain in the sleep mode and not respond to the voice command is based on comparing the hotword score with the additional hotword score.

9. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a first computing device that (i) is in sleep mode and (ii) is configured to disable the sleep mode and respond to voice commands upon receipt of a particular, predefined hotword, audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword;

while the first computing device remains in the sleep mode, and in response to receiving the audio data that corresponds to the utterance of a voice command that is preceded by the particular, predefined hotword, transmitting a message by the first computing device to a second computing device that (i) is in short-range electronic communication with the first computing device, and (ii) is also configured to respond to voice commands that are preceded by the particular, predefined hotword; and

determining, by the first computing device, to remain in the sleep mode and not respond to the voice command despite receiving the audio data that corresponds to the utterance of the voice command that is preceded by the particular predefined hotword, based at least on transmitting the message.

10. The system of claim 9, wherein the operations further comprise:

receiving an additional message from the second computing device,

wherein determining to remain in the sleep mode and not respond to the voice command is based at least in part on the additional message from the second computing device.

11. The system of claim 9, wherein the operations further comprise:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword,

wherein the message includes the hotword score.

12. The system of claim 9, wherein the operations further comprise:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword; and

determining that the hotword score satisfies a threshold,

wherein transmitting the message is based on determining that the hotword score satisfies the threshold.

13. The system of claim 9, wherein the first computing device transmits the message without performing speech recognition on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword.

14. The system of claim 9, wherein the operations further comprise:

determining that the second computing device is configured to respond to voice commands that are preceded by the particular, predefined hotword,

wherein transmitting the message is based on determining that the second computing device is configured to respond to voice commands that are preceded by the particular, predefined hotword.

15. The system of claim 9, wherein the first computing device transmits the message for a particular amount of time.

16. The system of claim 9, wherein the operations further comprise:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword;

receiving, from the second computing device, an additional message that includes an additional hotword score that is determined by the second computing device; and

comparing the hotword score with the additional hotword score,

wherein determining to remain in the sleep mode and not respond to the voice command is based on comparing the hotword score with the additional hotword score.

17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a first computing device that (i) is in sleep mode and (ii) is configured to disable the sleep mode and respond to voice commands upon receipt of a particular, predefined hotword, audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword;

while the first computing device remains in the sleep mode, and in response to receiving the audio data that corresponds to the utterance of a voice command that is preceded by the particular, predefined hotword, transmitting a message by the first computing device to a second computing device that (i) is in short-range electronic communication with the first computing device, and (ii) is also configured to respond to voice commands that are preceded by the particular, predefined hotword; and

determining, by the first computing device, to remain in the sleep mode and not respond to the voice command despite receiving the audio data that corresponds to the utterance of the voice command that is preceded by the particular predefined hotword, based at least on transmitting the message.

18. The medium of claim 17, wherein the operations further comprise:

receiving an additional message from the second computing device,

wherein determining to remain in the sleep mode and not respond to the voice command is based at least in part on the additional message from the second computing device.

19. The medium of claim 17, wherein the operations further comprise:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword,

wherein the message includes the hotword score.

20. The medium of claim 17, wherein the operations further comprise:

determining a hotword score that is based on the audio data that corresponds to an utterance of a voice command that is preceded by the particular, predefined hotword; and

determining that the hotword score satisfies a threshold,

wherein transmitting the message is based on determining that the hotword score satisfies the threshold.