Arbitration-based voice recognition

• US 10,614,807 B2
• Filed: 11/01/2018
• Issued: 04/07/2020
• Est. Priority Date: 10/19/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Beckhardt, Steven, Lin, Ted
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US16/178,122
• Publication Number:
  US 20190074008A1
• Time in Patent Office: 523 Days
• Field of Search: 704275
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A system comprises a first network microphone device (NMD) communicatively coupled to a second NMD via a network interface of the first NMD, wherein the first NMD is configured to: receive, from the second NMD via the network interface of the first NMD, an arbitration message comprising (i) a first measure of confidence associated with a voice input detected by the second NMD and (ii) an identifier associated with at least a portion of the voice input detected by the second NMD, determine that the first measure of confidence is greater than a second measure of confidence, wherein the second measure of confidence is associated with a voice input detected by the first NMD via at least one microphone of the second NMD, and based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the playback of the audio content by at least one of the first or the second NMDs.

Claims

1. A first network microphone device (NMD) comprising:

one or more audio processing components configured to cause the first NMD to play back audio content;

a network interface configured to communicatively couple the first NMD to a second NMD that is configured to play back audio content;

one or more processors;

at least one microphone; and

a tangible, non-transitory, computer-readable medium having instructions stored thereon that are executable by the one or more processors to cause the first NMD device to;

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD;

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via the network interface, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

2. The first NMD of claim 1, wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein the instructions that are executable to cause the first NMD to perform voice recognition further comprise instructions that are executable to cause the first NMD to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

3. The first NMD of claim 2, wherein the voice data is indicative of a wakeword.

4. The first NMD of claim 2, further comprising instructions that are executable to cause the first NMD to:

receive, from the server, a voice response to the voice message and play back the voice response.

5. The first NMD of claim 2, wherein the voice message is a first voice message, and wherein the arbitration message further comprises a value indicating an interval of time that the second NMD will wait before transmitting, to the server, a second voice message that is based on the voice input detected by the second NMD.

6. The first NMD of claim 1, wherein the arbitration message comprises a header that further comprises (i) voice data that is based on the voice input detected by the second NMD, (ii) an identifier associated with a source of the voice input detected by the second NMD, and (iii) a timestamp value indicating a time at which the arbitration message was transmitted by the second NMD.

7. The first NMD of claim 1, wherein the first NMD is configured to, via the one or more audio processing components, play back a first audio channel of the audio content in synchrony with playback of a second audio channel of the audio content by the second NMD.

8. The first NMD of claim 1, wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.

9. A non-transitory computer-readable storage medium storing instructions thereon that, when executed, cause at least one processor of a first network microphone device (NMD) to:

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD;

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via a network interface of the first NMD, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

10. The non-transitory computer-readable storage medium of claim 9, wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein the instructions that are executable to cause the first NMD to perform voice recognition further comprise instructions that are executable to cause the first NMD to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

11. The non-transitory computer-readable storage medium of claim 10, wherein the voice data is indicative of a wakeword.

12. The non-transitory computer-readable storage medium of claim 10, further comprising instructions that are executable to cause the first NMD to:

receive, from the server, a voice response to the voice message and play back the voice response.

13. The non-transitory computer-readable medium of claim 10, wherein the voice message is a first voice message, and wherein the arbitration message further comprises a value indicating an interval of time that the second NMD will wait before transmitting, to the server, a second voice message that is based on the voice input detected by the second NMD.

14. The non-transitory computer-readable storage medium of claim 9, wherein the arbitration message comprises a header that further comprises (i) voice data that is based on the voice input detected by the second NMD, (ii) an identifier associated with a source of the voice input detected by the second NMD, and (iii) a timestamp value indicating a time at which the arbitration message was transmitted by the second NMD.

15. The non-transitory computer-readable storage medium of claim 9, further comprising instructions that, when executed, cause the at least one processor to:

via one or more audio processing components of the first NMD, play back a first audio channel of the audio content in synchrony with playback of a second audio channel of the audio content by the second NMD.

16. The non-transitory computer-readable medium of claim 9, wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.

17. A system comprising:

a first network microphone device (NMD) communicatively coupled to a second NMD via a network interface of the first NMD, wherein the first NMD is configured to;

while operating in a mode in which the first NMD is configured to play back audio content in synchrony with the second NMD;

detect a voice input via the at least one microphone;

determine a first measure of confidence associated with the voice input detected by the first NMD;

receive, from the second NMD via the network interface of the first NMD, an arbitration message comprising (i) a second measure of confidence associated with a voice input detected by the second NMD and (ii) the voice input detected by the second NMD;

determine that the second measure of confidence is greater than the first measure of confidence;

based on the determination, perform voice recognition based on the voice input detected by the second NMD, wherein the voice input detected by the second NMD comprises a command to control the synchronous playback of the audio content by the first NMD and the second NMD; and

after performing voice recognition based on the voice input detected by the second NMD, executing the command to control the synchronous playback of the audio content by the first NMD and the second NMD.

18. The system of claim 17, wherein the arbitration message further comprises voice data that is based on the voice input detected by the second NMD, and wherein to perform voice recognition, the first NMD is configured to:

transmit a voice message that comprises the voice data to a server via a network for voice processing.

19. The system of claim 18, wherein the first NMD is further configured to:

receive, from the server, a voice response to the voice message and play back the voice response.

20. The system of claim 17, wherein the voice input detected by the second NMD comprises one or more of (i) a wakeword or (ii) at least one command.