Arbitration-Based Voice Recognition

• US 20180108351A1
• Filed: 10/19/2016
• Published: 04/19/2018
• Est. Priority Date: 10/19/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Beckhardt, Steven, Lin, Ted
• Granted Patent: US 10,181,323 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A first voice input is detected and a first measure of confidence associated with the first voice input is determined. A message is received comprising a second measure of confidence associated with detection of the first voice input by a network device. A determination is made whether the first measure of confidence is greater than the second measure of confidence. Based on the determination that the first measure of confidence is greater than the second measure of confidence, a second voice input is sent to a server.

Claims

1. (canceled)

2. (canceled)

3. (canceled)

4. (canceled)

5. (canceled)

9. (canceled)

10. (canceled)

11. A method comprising:

detecting, with a first network device that comprises a first microphone device (NMD), a voice input;

determining, by the first NMD, a first measure of confidence associated with the first voice input;

transmitting a message to a second network device, wherein the message comprises the first measure of confidence;

receiving, from the first NMD and by the second network device, the message comprising the first measure of confidence;

after receiving the message, determining, by the second network device, that the first measure of confidence is greater than a second measure of confidence, wherein the second measure of confidence is associated with a voice input that has been detected by a second NMD; and

based on the determination that the first measure of confidence is greater than the second measure of confidence, causing the first NMD to perform voice recognition.

16. The method of claim 11, further comprising waiting for expiration of a timer before sending the voice input to the server.

17. The method of claim 11, further comprising receiving a voice response to the voice input from the server and audibly playing the voice response.

18. The method of claim 11, wherein the message is a first message, the method further comprising sending a second message which comprises the first measure of the confidence associated with the detected voice input to the network device.

30. The method of claim 11, comprising:

detecting the voice input via a microphone of a third NMD, wherein the third network device comprising the second NMD.

31. The method of claim 11, wherein the first measure of confidence and the second measure of confidence are based on at least one of:

an audio envelope of a wakeword of the voice input, or

a frequency spectrum content of a wakeword of the voice input.

32. The method of claim 16, comprising:

determining a time interval of the timer based on a learning algorithm that balances arbitration time and latency concerns of the voice input.

12. (canceled)

13. (canceled)

14. (canceled)

15. (canceled)

19. (canceled)

20. (canceled)

21. A system comprising a first network device and a second network device communicatively coupled to the first network device over a network, wherein:

the first network device comprises a first network microphone device (NMD), wherein the first NMD comprises at least one processor and tangible, non-transitory computer-readable memory comprising instructions that, when executed, cause the first NMD to perform first operations comprising;

detecting a voice input;

determining a first measure of confidence associated with the voice input captured detected by the first network device; and

transmitting a message to the second network device, wherein the message comprises the first measure of confidence; and

the second network device comprises at least one processor and tangible, non-transitory computer-readable memory comprising instructions that, when executed, cause the second network device to perform second operations comprising;

receiving from the first NMD the message comprising the first measure of confidence;

after receiving the message, determining that the first measure of confidence is greater than a second measure of confidence, wherein the second measure of confidence is associated with a voice input that has been detected by a second NMD; and

based on the determination that the first measure of confidence is greater than the second measure of confidence, causing the first NMD to perform voice recognition.

6. The system of claim 21, wherein the message is a first message, the first operations comprising sending to the second network device a second message which comprises the first measure of the confidence associated with the detected first voice input.

7. The system of claim 21, wherein the second operations comprise:

waiting for expiration of a timer before sending the voice input to a server.

8. The first network device of claim 21, further comprising computer instructions for receiving a voice response to the voice input from the server and audibly playing the voice response.

22. The system of claim 21, wherein the second network device comprises the second NMD, and wherein the second operations further comprise detecting the voice input via a microphone of the second NMD.

23. The system of claim 21, wherein the second network device comprises a playback device.

24. The system of claim 21, comprising:

a third network device comprising the second NMD, wherein the third network device comprises at least one processor and tangible, non-transitory computer-readable memory comprising instructions that, when executed, cause the third network device to perform third operations comprising;

detecting the voice input via a microphone of the third NMD.

25. The system of claim 21, wherein the second network device comprises a remote server that is accessible via the Internet by the first network device.

26. The system of claim 21, wherein the first measure of confidence and the second measure of confidence are based on at least one of:

an audio envelope of a wakeword of the voice input, or

a frequency spectrum content of a wakeword of the voice input.

27. The system of claim 7, wherein a time interval of the timer is determined based on a learning algorithm that balances arbitration time and latency concerns of the voice input.

28. The system of claim 21, wherein the second network device comprises the second NMD, and wherein the second operations further comprise detecting the voice input via a microphone of the second NMD.

29. The system of claim 21, wherein the second network device comprises a playback device.