Devices, systems, and methods for distributed voice processing

• US 10,867,604 B2
• Filed: 02/08/2019
• Issued: 12/15/2020
• Est. Priority Date: 02/08/2019
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Smith, Connor Kristopher, Tolomei, John, Lee, Betty
• Primary Examiner(s): Azad, Abul K
• Application Number:
  US16/271,560
• Publication Number:
  US 20200258513A1
• Time in Patent Office: 676 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Systems and methods for distributed voice processing are disclosed herein. In one example, the method includes detecting sound via a microphone array of a first playback device and analyzing, via a first wake-word engine of the first playback device, the detected sound. The first playback device may transmit data associated with the detected sound to a second playback device over a local area network. A second wake-word engine of the second playback device may analyze the transmitted data associated with the detected sound. The method may further include identifying that the detected sound contains either a first wake word or a second wake word based on the analysis via the first and second wake-word engines, respectively. Based on the identification, sound data corresponding to the detected sound may be transmitted over a wide area network to a remote computing device associated with a particular voice assistant service.

Claims

1. A method comprising:

detecting sound via a microphone array of a first playback device;

transmitting data associated with the detected sound from the first playback device to a second playback device over a local area network;

analyzing, via a wake word engine of the second playback device, the transmitted data associated with the detected sound for identification of a wake word;

identifying that the detected sound contains the wake word based on the analysis via the wake word engine;

based on the identification, transmitting sound data corresponding to the detected sound from the second playback device to a remote computing device over a wide area network, wherein the remote computing device is associated with a particular voice assistant service;

receiving via the second playback device a response from the remote computing device, wherein the response is based on the detected sound;

transmitting a message from the second playback device to the first playback device over the local area network, wherein the message is based on the response from the remote computing device and includes instructions to perform an action; and

performing the action via the first playback device.

2. The method of claim 1, wherein the action is a first action and the method further comprises performing a second action via the second playback device, wherein the second action is based on the response from the remote computing device.

3. The method of claim 1, further comprising disabling a wake word engine of the first playback device in response to the identification of the wake word via the wake word engine of the second playback device.

4. The method of claim 3, further comprising enabling a wake word engine of the first playback device after the second playback device receives the response from the remote computing device.

5. The method of claim 4, wherein the wake word is a second wake word, and wherein the wake word engine of the first playback device is configured to detect a first wake word that is different than the second wake word.

6. The method of claim 1, wherein the first playback device is configured to communicate with the remote computing device associated with the particular voice assistant service.

7. The method of claim 1, wherein the remote computing device is a first remote computing device and the voice assistant service is a first voice assistant service, and wherein the first playback device is configured to detect a wake word associated with a second voice assistant service different than the first voice assistant service.

8. A first playback device comprising:

one or more processors;

a computer-readable medium storing instructions that, when executed by the one or more processors, cause the first playback device to perform operations comprising;

receiving, from a second playback device over a local area network, data associated with sound detected via a microphone array of the second playback device;

analyzing, via a wake word engine of the first playback device, the data associated with the detected sound for identification of a wake word;

identifying that the detected sound contains the wake word based on the analysis via the wake word engine;

based on the identification, transmitting sound data corresponding to the detected sound to a remote computing device over a wide area network, wherein the remote computing device is associated with a particular voice assistant service;

receiving a response from the remote computing device, wherein the response is based on the detected sound; and

transmitting a message to the second playback device over the local area network, wherein the message is based on the response from the remote computing device and includes instructions for the second playback device to perform an action.

9. The first playback device of claim 8, wherein the action is a first action and the operations further comprise performing a second action via the first playback device, wherein the second action is based on the response from the remote computing device.

10. The first playback device of claim 8, wherein the operations further comprise disabling a wake word engine of the second playback device in response to the identification of the wake word via the wake word engine of the first playback device.

11. The first playback device of claim 10, wherein the operations further comprise enabling the wake word engine of the second playback device after the first playback device receives the response from the remote computing device.

12. The first playback device of claim 11, wherein the wake word is a first wake word, and wherein the wake word engine of the second playback device is configured to detect a second wake word that is different than the first wake word.

13. The first playback device of claim 8, wherein the second playback device is configured to communicate with the remote computing device associated with the particular voice assistant service.

14. The first playback device of claim 8, wherein the remote computing device is a first remote computing device and the voice assistant service is a first voice assistant service, and wherein the second playback device is configured to detect a wake word associated with a second voice assistant service different than the first voice assistant service.

15. A system, comprising:

a first playback device comprising;

one or more processors;

a microphone array; and

a first computer-readable medium storing instructions that, when executed by the one or more processors, cause the first playback device to perform first operations, the first operations comprising;

detecting sound via the microphone array;

transmitting data associated with the detected sound to a second playback device over a local area network;

the second playback device comprising;

one or more processors; and

a second computer-readable medium storing instructions that, when executed by the one or more processors, cause the second playback device to perform second operations, the second operations comprising;

analyzing, via a wake word engine of the second playback device, the transmitted data associated with the detected sound from the first playback device for identification of a wake word;

identifying that the detected sound contains the wake word based on the analysis via the wake word engine;

based on the identification, transmitting sound data corresponding to the detected sound to a remote computing device over a wide area network, wherein the remote computing device is associated with a particular voice assistant service;

receiving a response from the remote computing device, wherein the response is based on the detected sound; and

transmitting a message to the first playback device over the local area network, wherein the message is based on the response from the remote computing device and includes instructions to perform an action,

wherein the first computer-readable medium of the first playback device causes the first playback device to perform the action from the instructions received from the second playback device.

16. The system of claim 15, wherein the action is a first action and the second operations further comprise performing a second action via the second playback device, wherein the second action is based on the response from the remote computing device.

17. The system of claim 15, wherein the second operations further comprise disabling a wake word engine of the first playback device in response to the identification of the wake word via the wake word engine of the second playback device.

18. The system of claim 17, wherein the second operations further comprise enabling the wake word engine of the first playback device after the second playback device receives the response from the remote computing device.

19. The system of claim 15, wherein the first playback device is configured to communicate with the remote computing device associated with the particular voice assistant service.

20. The system of claim 15, wherein the remote computing device is a first remote computing device and the voice assistant service is a first voice assistant service, and wherein the first playback device is configured to detect a wake word associated with a second voice assistant service different than the first voice assistant service.