Voice Detection By Multiple Devices

• US 20190341045A1
• Filed: 05/20/2019
• Published: 11/07/2019
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Reilly, Jonathon, Burlingame, Gregory, Butts, Christopher, Kadri, Romi, Lang, Jonathan P.
• Granted Patent: US 10,699,711 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/34 Adaptation of a single recogniser for parallel processing, e.g. by use of multiple processors or cloud computing
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Disclosed herein are example techniques for voice detection by multiple NMDs. An example implementation may involve one or more servers receiving, via a network interface, data representing multiple audio recordings of a voice input spoken by a given user, each audio recording recorded by a respective NMD of the multiple NMDs, wherein the voice input comprises a detected wake-word. Based on respective sound pressure levels of the multiple audio recordings of the voice input, the servers (i) select a particular NMD of the multiple NMDs and (ii) forego selection of other NMDs of the multiple NMDs. The servers send, via the network interface to the particular NMD, data representing a playback command that corresponds to a voice command in the voice input represented in the multiple audio recordings, wherein the data representing the playback command causes the particular NMD to play back audio content according to the playback command.

Claims

1. A first networked microphone device (NMD) comprising:

one or more amplifiers configured to drive one or more speakers;

a microphone array;

a network interface;

one or more processors;

data storage having stored therein instructions executable by the one or more processors to cause the first NMD to perform functions comprising;

recording, via the microphone array, audio into a buffer;

monitoring the recorded audio in the buffer for wake words;

when a wake-word is detected in the recorded audio, querying, via the network interface, one or more servers of a particular voice assistant service with a voice command following the detected wake-word within the recorded audio;

receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a voice response corresponding to the voice command;

in response to receiving the voice response corresponding to the voice command, sending, via the network interface to one or more second NMDs connected via to the first NMD via a local area network, instructions to cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs; and

playing back the voice response via the one or more amplifiers configured to drive one or more speakers.

2. The first NMD of claim 1, wherein the functions further comprise:

selecting the first NMD to handle the voice response; and

foregoing selection of the one or more second NMDs to handle the voice response.

3. The first NMD of claim 2, wherein the first NMD is in a synchrony group with the one or more second NMDS, and wherein selecting the first NMD to handle the voice response comprises selecting the first NMD handle the voice response based on state information indicating that the first NMD is group coordinator of the synchrony group, wherein the group coordinator is configured to provide, to group members, at least one of (i) synchrony timing information and (ii) synchrony audio information.

4. The first NMD of claim 1, wherein one or more servers of the particular voice assistant service select the first NMD to handle the voice response and forego selection of the one or more second NMDs to handle the voice response.

5. The first NMD of claim 4, wherein the one or more servers of the particular voice assistant service select the first NMD to handle the voice response and forego selection of the one or more second NMDs to handle the voice response based on respective sound pressure levels of the voice command in multiple recording corresponding to respective NMDS of the first NMD and the one or more second NMDs.

6. The first NMD of claim 1, wherein sending instructions to cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs comprising sending instructions that cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for a pre-defined time period.

7. The first NMD of claim 6, wherein the functions further comprise

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command;

and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period has fully elapsed.

8. The first NMD of claim 1, wherein the functions further comprise:

forming a synchrony group including the first NMD and the one or more second NMDS, wherein playing back the voice response comprises playing back the voice response in synchrony with the one or more second NMDS.

9. A method to be performed by first networked microphone device (NMD), the method comprising:

recording, via a microphone array of the first NMD, audio into a buffer;

monitoring the recorded audio in the buffer for wake words;

when a wake-word is detected in the recorded audio, querying, via a network interface of the first NMD, one or more servers of a particular voice assistant service with a voice command following the detected wake-word within the recorded audio;

receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a voice response corresponding to the voice command;

in response to receiving the voice response corresponding to the voice command, sending, via the network interface to one or more second NMDs connected via to the first NMD via a local area network, instructions to cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs; and

playing back the voice response via one or more amplifiers configured to drive one or more speakers, the first NMD comprising the one or more amplifiers and the one or more speakers.

10. The method of claim 9, further comprising:

selecting the first NMD to handle the voice response; and

foregoing selection of the one or more second NMDs to handle the voice response.

11. The method of claim 10, wherein the first NMD is in a synchrony group with the one or more second NMDS, and wherein selecting the first NMD to handle the voice response comprises selecting the first NMD handle the voice response based on state information indicating that the first NMD is group coordinator of the synchrony group, wherein the group coordinator is configured to provide, to group members, at least one of (i) synchrony timing information and (ii) synchrony audio information.

12. The method of claim 9, wherein one or more servers of the particular voice assistant service select the first NMD to handle the voice response and forego selection of the one or more second NMDs to handle the voice response.

13. The method of claim 12, wherein the one or more servers of the particular voice assistant service select the first NMD to handle the voice response and forego selection of the one or more second NMDs to handle the voice response based on respective sound pressure levels of the voice command in multiple recording corresponding to respective NMDS of the first NMD and the one or more second NMDs.

14. The method of claim 9, wherein sending instructions to cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs comprising sending instructions that cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs for a pre-defined time period.

15. The method of claim 14, further comprising:

determining, based on the recorded audio in the buffer, that the first NMD is no longer receiving the voice command; and

based on determining that the first NMD is no longer receiving the voice command, sending, via the network interface, instructions to one or more second NMDs connected via to the first NMD via the local area network, the instructions causing the one or more second NMDs to start recording audio via respective microphone arrays of the one or more second NMDs before the pre-defined time period has fully elapsed.

16. The method of claim 9, further comprising:

forming a synchrony group including the first NMD and the one or more second NMDS, wherein playing back the voice response comprises playing back the voice response in synchrony with the one or more second NMDS.

17. A tangible, non-transitory, computer-readable medium storing instructions that, when executed by one or more processors, cause a first networked microphone device (NMD) to perform functions comprising:

recording, via a microphone array of the first NMD, audio into a buffer;

monitoring the recorded audio in the buffer for wake words;

when a wake-word is detected in the recorded audio, querying, via a network interface of the first NMD, one or more servers of a particular voice assistant service with a voice command following the detected wake-word within the recorded audio;

receiving, from one or more servers of the particular voice assistant service via the network interface in response to the query, a voice response corresponding to the voice command;

in response to receiving the voice response corresponding to the voice command, sending, via the network interface to one or more second NMDs connected via to the first NMD via a local area network, instructions to cause the one or more second NMDs to stop recording audio via respective microphone arrays of the one or more second NMDs; and

playing back the voice response via one or more amplifiers configured to drive one or more speakers, the first NMD comprising the one or more amplifiers and the one or more speakers.

18. The tangible, non-transitory, computer-readable medium of claim 17, wherein the functions further comprise:

selecting the first NMD to handle the voice response; and

foregoing selection of the one or more second NMDs to handle the voice response.

19. The tangible, non-transitory, computer-readable medium of claim 18, wherein the first NMD is in a synchrony group with the one or more second NMDS, and wherein selecting the first NMD to handle the voice response comprises selecting the first NMD handle the voice response based on state information indicating that the first NMD is group coordinator of the synchrony group, wherein the group coordinator is configured to provide, to group members, at least one of (i) synchrony timing information and (ii) synchrony audio information.

20. The tangible, non-transitory, computer-readable medium of claim 17, wherein one or more servers of the particular voice assistant service select the first NMD to handle the voice response and forego selection of the one or more second NMDs to handle the voice response based on respective sound pressure levels of the voice command in multiple recording corresponding to respective NMDS of the first NMD and the one or more second NMDs.