MEDIATION OF WAKEWORD RESPONSE FOR MULTIPLE DEVICES

• US 20170090864A1
• Filed: 09/28/2015
• Published: 03/30/2017
• Est. Priority Date: 09/28/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Jorgovanovic, Milos
• Granted Patent: US 9,996,316 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/0332 involving modification of waveforms
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals
• CPC Class Codes: H04M 11/007 with remote control systems remote control in general G08C
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

Devices, systems, and methods provide synchronization of multiple voice-controlled devices to establish priority of one of the devices to respond to an acoustic signal, preventing other devices from responding to a single user command. Each device is configured to mediate its response to a wakeword by, after detecting the wakeword in an audio input, generating a data packet and attempting to send it to the other devices over a communication channel that prevents data collision. One device will succeed in sending the packet, while the other devices cannot until they receive the first device'"'"'s packet, indicating that the first device has priority. Additionally, the devices may process their audio inputs to determine a signal quality, the signal qualities are shared between all devices, and the device with the best signal quality assumes priority.

Claims

1. A computing device comprising:

a network interface;

a microphone;

a processor that converts an acoustic signal received by the microphone into a first audio input; and

memory storing program instructions, the computing device executing the program instructions to;

determine a first signal strength value representing a first signal-to-noise ratio of the first audio input;

determine that the first audio input comprises a wakeword;

generate a local data packet comprising the first signal strength value;

send the local data packet to an access point device communicatively coupled to the computing device;

receive a first data packet from the access point device during a predetermined time period after determining that the first audio input comprises the wakeword, the first data packet comprising a second signal strength value representing a second signal-to-noise ratio of a second audio input, wherein the second audio input is generated from the acoustic signal by a second computing device that is configured to process the second audio input in response to detecting the wakeword included in the second audio input;

determine that the first signal strength value is greater than the second signal strength value;

determine that the first audio input further comprises a command;

determine that the predetermined time period after determining that the first audio input comprises the wakeword has expired; and

responsive to determining that the predetermined time period after determining that the first audio input comprises the wakeword has expired, perform an action associated with the command.

2. The computing device of claim 1, wherein the memory stores an expected packet transmission time, and wherein the predetermined time period has a duration that is at least as long as the expected packet transmission time multiplied by a number of remote devices communicatively coupled to the access point device.

3. The computing device of claim 1, wherein the computing device further executes the program instructions to:

receive a second data packet from the access point device during the predetermined time period after determining that the first audio input comprises the wakeword, the second data packet comprising a third signal strength value representing a third signal-to-noise ratio of a third audio input, wherein the third audio input is generated from the acoustic signal by a third computing device that is configured to process the third audio input in response to detecting the wakeword included in the third audio input; and

before determining that the first audio input comprises the command;

determine that the first signal strength value is less than the third signal strength value;

refrain from executing the program instructions to determine that the first audio input comprises the command; and

refrain from executing the program instructions to perform the action.

4. The computing device of claim 1, wherein the computing device further executes the program instructions to:

receive a second data packet from the access point device at a first time during the predetermined time period after determining that the first audio input comprises the wakeword, the second data packet comprising a third signal strength value representing a third signal-to-noise ratio of a third audio input, wherein the third audio input is generated from the acoustic signal by a third computing device that is configured to process the third audio input in response to detecting the wakeword included in the third audio input;

receive the local data packet from the access point device at a second time during the predetermined time period after determining that the first audio input comprises the wakeword;

determine that the first signal strength value is equal to the third signal strength value; and

before determining that the first audio input comprises the command;

determine that the first time is prior to the second time;

refrain from executing the program instructions to determine that the first audio input comprises the command; and

refrain from executing the program instructions to perform the action.

5. A computing device, comprising:

a processor that receives a first audio input representing an acoustic signal; and

memory storing program instructions, the processor executing the program instructions to;

determine that the first audio input comprises a wakeword;

determine a first quality value of the first audio input;

receive a second quality value generated from a second audio input representing the acoustic signal, the second audio input obtained by a second device;

determine that the first quality value is greater than the second quality value;

determine that the first audio input further comprises a command; and

process the command.

6. The computing device of claim 5, wherein the first quality value comprises a first signal-to-noise ratio of the first audio input and the second quality value comprises a second signal-to-noise ratio of the second audio input.

7. The computing device of claim 5, wherein the first quality value comprises a first automatic gain control setting used to generate the first audio input, and the second quality value comprises a second automatic gain control setting used to generate the second audio input.

8. The computing device of claim 5, wherein the processor executes the program instructions to:

maintain, for a predetermined time period after determining that the first audio input comprises the wakeword, a connection to an access point device via a communication channel; and

receive one or more quality values over the communication channel during the predetermined time period, the one or more quality values including the second quality value; and

subsequent to the predetermined time period elapsing, process the command.

9. The computing device of claim 8, wherein the processor further executes the program instructions to:

receive a third quality value of the one or more quality values;

during the predetermined time period;

determine that the third quality value is greater than the first quality value; and

cease receiving additional quality values of the one or more quality values; and

refrain from executing the program instructions to process the command.

10. The computing device of claim 5, wherein the computing device is communicatively coupled to an access point device and executes the program instructions to further:

generate a local data packet comprising the first quality value and an instruction for the access point device to send the local data packet to the second device; and

send the local data packet to the access point device.

11. The computing device of claim 10, wherein the instruction further includes information for the access point device to send the local data packet to a group of devices comprising the computing device, the second device, and a third device, and wherein before the computing device executes the program instructions to process the command, the computing device executes the program instructions to:

receive, from the access point device at a first time, a third data packet comprising a third quality value generated from a third audio input by the third device, the third audio input representing the acoustic signal;

receive the local data packet from the access point device at a second time;

determine that the first quality value is equal to the third quality value;

determine that the first time is prior to the second time; and

refrain from executing the program instructions to process the command.

12. The computing device of claim 5, wherein the computing device executes the program instructions to further:

establish a first communication channel directly with the second device;

generate a local data packet comprising the first quality value;

send the local data packet to the second device via the communication channel; and

receive a second data packet from the second device via the communication channel, the second data packet comprising the second quality value.

13. The computing device of claim 12, wherein before the computing device executes the program instructions to process the command, the computing device executes the program instructions to:

establish a second communication channel directly with a third device;

receive, over the second communication channel from the third device at a first time, a third data packet comprising a third quality value generated from a third audio input representing the acoustic signal, the third audio input obtained by the third device;

send, at a second time, the local data packet to the third device over the second communication channel;

determine that the first quality value is equal to the third quality value;

determine that the first time is prior to the second time; and

refrain from executing the program instructions to process the command.

14. The computing device of claim 5, wherein before the computing device executes the program instructions to process the command, the computing device executes the program instructions to:

determine that the first quality value is less than a maximum quality value of a plurality of quality values including the second quality value; and

refrain from executing the program instructions to process the command.

15. A method, comprising:

identifying a wakeword in a first audio input of a computing device disposed within an audible range of an acoustic signal, the first audio input representing the acoustic signal as detected by the computing device;

determining a first quality value of the first audio input;

receiving one or more data packets, each data packet of the one or more data packets comprising a corresponding remote quality value of one or more remote quality values, each remote quality value of the one or more remote quality values being generated from a corresponding remote audio input of one or more remote audio inputs representing the acoustic signal, each remote audio input of the one or more remote audio inputs obtained by a corresponding remote device of one or more remote devices disposed in the audible range;

determining that the first quality value is greater than each remote quality value of the one or more remote quality values;

transitioning the computing device to a responsive state;

determining that the first audio input further comprises a command; and

processing the command.

16. The method of claim 15, further comprising sending the first quality value to a first remote device of the one or more remote devices.

17. The method of claim 16, wherein sending the first quality value to the first remote device comprises:

establishing a communication channel directly with the first remote device;

generating a local data packet comprising the first quality value; and

sending the local data packet over the communication channel to the first remote device;

the method further comprising receiving a first data packet of the one or more data packets from the first remote device over the communication channel.

18. The method of claim 15, further comprising:

generating a local data packet comprising the first quality value and an instruction for an access point device to send the local data packet to a group of devices comprising the one or more remote devices; and

sending the local data packet over an electronic communication channel to the access point device.

19. The method of claim 15, wherein determining the first quality value of the first audio input comprises computing a signal-to-noise ratio of the first audio input.

20. The method of claim 15, further comprising waiting a predetermined time period after identifying the wakeword in the first audio input, during which predetermined time period the one or more data packets are received.