Voice detection by multiple devices

• US 10,152,969 B2
• Filed: 07/15/2016
• Issued: 12/11/2018
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Reilly, Jonathon, Burlingame, Gregory, Butts, Christopher, Kadri, Romi, Lang, Jonathan P.
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US15/211,748
• Publication Number:
  US 20180018964A1
• Time in Patent Office: 879 Days
• Field of Search: 704235, 704246, 704270, 7042701, 704275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/34 Adaptation of a single recogniser for parallel processing, e.g. by use of multiple processors or cloud computing
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Disclosed herein are example techniques for voice detection by multiple NMDs. An example implementation may involve receiving a set of voice recordings from a set of NMDs, and identifying a subset of voice recordings from which to determine a given voice command. The example implementation may further involve causing the identified subset of voice recordings to be analyzed to determine the given voice command.

Claims

1. A first networked microphone device (NMD) comprising:

one or more amplifiers configured to drive one or more speakers;

a microphone array;

a network interface;

one or more processors;

tangible, non-transitory computer-readable media having stored therein instructions executable by the one or more processors to cause the first NMD to perform a method comprising;

continuously recording, via the microphone array, audio into a buffer;

detecting, in the recorded audio, a wake-word;

in response to detecting the wake-word, (i) listening, via the microphone array, for a voice command following the wake-word in the recorded audio and (ii) sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via a local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for a pre-defined time period;

querying, via the network interface, one or more servers of a particular voice assistant service with the voice command following the detected wake-word within the recorded audio;

receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a playback command corresponding to the voice command; and

playing back audio content according to the playback command via the one or more amplifiers configured to drive one or more speakers.

2. The first NMD of claim 1, wherein the voice command includes an indication of a period of time for the first NMD to listen for the voice command, and wherein the method further comprises sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for the period of time indicated by the voice command.

3. The first NMD of claim 2, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the period of time indicated by the voice command has fully elapsed.

4. The first NMD of claim 1, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period has fully elapsed.

5. The first NMD of claim 1, wherein the playback command comprises a command to play back particular audio content in a first zone that includes the first NMD and a second zone that includes a second NMD, and wherein the method further comprises:

instructing, via the network interface, the second NMD of the second zone to play back the audio content according to the playback command in synchrony with playback of the audio content by the first NMD of the first zone.

6. The first NMD of claim 1, wherein a first zone of a media playback system includes the first NMD, and wherein the first zone is configured into a zone group with a second zone that includes one or more playback devices, and wherein playing back the audio content according to the playback command comprises playing back the audio content in synchrony with one or more playback devices of the second zone.

7. The first NMD of claim 1, wherein a first zone of a media playback system includes the first NMD and a second NMD in a bonded zone configuration in which the first NMD and the second NMD play respective channels of the audio content, and wherein playing back the audio content according to the playback command comprises playing back a first channel of the audio content in synchrony the second NMD playing back a second channel of the audio content.

8. Tangible, non-transitory, computer-readable media having instructions encoded therein, wherein the instructions, when executed by one or more processors, cause a first networked microphone device (NMD) to perform a method comprising:

continuously recording, via a microphone array of the first NMD, audio into a buffer;

detecting, in the recorded audio, a wake-word;

in response to detecting the wake-word, (i) listening, via a microphone of the first NMD, for a voice command following the wake-word in the recorded audio and (ii) sending, via a network interface, instructions to one or more second NMDs connected via to the first NMD via a local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for a pre-defined time period;

querying, via a network interface of the first NMD, one or more servers of a particular voice assistant service with the voice command following the detected wake-word within the recorded audio;

receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a playback command corresponding to the voice command; and

playing back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers.

9. The tangible, computer readable media of claim 8, wherein the voice command includes an indication of a period of time for the first NMD to listen for the voice command, and wherein the method further comprises sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for the period of time indicated by the voice command.

10. The tangible, computer readable media of claim 9, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the period of time indicated by the voice command has fully elapsed.

11. The tangible, computer readable media of claim 8, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period has fully elapsed.

12. The tangible, computer readable media of claim 8, wherein the playback command comprises a command to play back particular audio content in a first zone that includes the first NMD and a second zone that includes a second NMD, and wherein the method further comprises:

instructing, via the network interface, the second NMD of the second zone to play back the audio content according to the playback command in synchrony with playback of the audio content by the first NMD of the first zone.

13. The tangible, computer readable media of claim 8, wherein a first zone of a media playback system includes the first NMD, and wherein the first zone is configured into a zone group with a second zone that includes one or more playback devices, and wherein playing back the audio content according to the playback command comprises playing back the audio content in synchrony with one or more playback devices of the second zone.

14. The tangible, computer readable media of claim 8, wherein a first zone of a media playback system includes the first NMD and a second NMD in a bonded zone configuration in which the first NMD and the second NMD play respective channels of the audio content, and wherein playing back the audio content according to the playback command comprises playing back a first channel of the audio content in synchrony the second NMD playing back a second channel of the audio content.

15. A method comprising:

a first networked microphone device (NMD) continuously recording, via a microphone array of the first NMD, audio into a buffer;

the first NMD detecting, in the recorded audio, a wake-word;

in response to detecting the wake-word, the first NMD (i) listening, via a microphone of the first NMD, for a voice command following the wake-word in the recorded audio and (ii) sending, via a network interface, instructions to one or more second NMDs connected via to the first NMD via a local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for a pre-defined time period;

the first NMD querying, via a network interface of the first NMD, one or more servers of a particular voice assistant service with the voice command following the detected wake-word within the recorded audio;

the first NMD receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a playback command corresponding to the voice command; and

the first NMD playing back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers.

16. The method of claim 15, wherein the voice command includes an indication of a period of time for the first NMD to listen for the voice command, and wherein the method further comprises sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for the period of time indicated by the voice command.

17. The method of claim 15, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period indicated by the voice command has fully elapsed.

18. The method of claim 17, wherein the method further comprises:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command, and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period has fully elapsed.

19. The method of claim 15, wherein a first zone of a media playback system includes the first NMD, and wherein the first zone is configured into a zone group with a second zone that includes one or more playback devices, and wherein playing back the audio content according to the playback command comprises playing back the audio content in synchrony with one or more playback devices of the second zone.

20. The method of claim 15, wherein a first zone of a media playback system includes the first NMD and a second NMD in a bonded zone configuration in which the first NMD and the second NMD play respective channels of the audio content, and wherein playing back the audio content according to the playback command comprises playing back a first channel of the audio content in synchrony the second NMD playing back a second channel of the audio content.