Mediation of wakeword response for multiple devices

• US 9,996,316 B2
• Filed: 09/28/2015
• Issued: 06/12/2018
• Est. Priority Date: 09/28/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Jorgovanovic, Milos
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/868,170
• Publication Number:
  US 20170090864A1
• Time in Patent Office: 988 Days
• Field of Search: 704275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/0332 involving modification of waveforms
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals
• CPC Class Codes: H04M 11/007 with remote control systems remote control in general G08C
• CPC Class Codes: H04M 2250/74 with voice recognition means

Abstract

Devices, systems, and methods provide synchronization of multiple voice-controlled devices to establish priority of one of the devices to respond to an acoustic signal, preventing other devices from responding to a single user command. Each device is configured to mediate its response to a wakeword by, after detecting the wakeword in an audio input, generating a data packet and attempting to send it to the other devices over a communication channel that prevents data collision. One device will succeed in sending the packet, while the other devices cannot until they receive the first device'"'"'s packet, indicating that the first device has priority. Additionally, the devices may process their audio inputs to determine a signal quality, the signal qualities are shared between all devices, and the device with the best signal quality assumes priority.

Claims

1. A computing device comprising:

a network interface;

a microphone;

a processor that converts an acoustic signal received by the microphone into a first audio input; and

memory storing program instructions, the computing device executing the program instructions to;

determine a first signal strength value representing a first signal-to-noise ratio of the first audio input;

determine that the first audio input comprises a wakeword;

generate a first data packet comprising the first signal strength value;

send the first data packet to an access point device communicatively coupled to the computing device;

receive a second data packet from the access point device at a first time, the second data packet comprising the first signal strength value;

receive a third data packet from the access point device at a second time, the third data packet comprising a second signal strength value representing a second signal-to-noise ratio of a second audio input, wherein the second audio input is generated from the acoustic signal by a second computing device that is configured to process the second audio input in response to detecting the wakeword included in the second audio input;

determine that the first signal strength value is approximately equal to or greater than the second signal strength value;

determine that the first time is prior to the second time;

determine that the first audio input further comprises a command;

responsive to determining that the first time is prior to the second time, perform an action associated with the command.

2. The computing device of claim 1, wherein the computing device further executes the program instructions to, after determining that the first audio input comprises the wakeword:

determine that a predetermined time period has elapsed;

wherein the memory stores an expected packet transmission time, and wherein the predetermined time period has a duration that is at least as long as the expected packet transmission time multiplied by a number of remote devices communicatively coupled to the access point device.

3. The computing device of claim 1, wherein the computing device further executes the program instructions to:

receive a fourth data packet from the access point device, the fourth data packet comprising a third signal strength value representing a third signal-to-noise ratio of a third audio input, wherein the third audio input is generated from the acoustic signal by a third computing device that is configured to process the third audio input in response to detecting the wakeword included in the third audio input; and

before determining that the first audio input comprises the command;

determine that the first signal strength value is less than the third signal strength value;

refrain from executing the program instructions to determine that the first audio input comprises the command; and

refrain from executing the program instructions to perform the action.

4. A computing device, comprising:

a processor that receives a first audio input representing an acoustic signal; and

memory storing program instructions, the processor executing the program instructions to;

determine that the first audio input comprises a wakeword;

send a first data packet comprising a first device identifier to an access point device communicatively coupled to the computing device;

receive, at a first time, a second data packet from the access point device, the second data packet including the first device identifier;

receive, at a second time, a third packet comprising a second device identifier, the third packet representing that a second device had obtained a second audio input representing the acoustic signal and had determined the second audio input comprises the wakeword;

determine that the second data packet includes the first device identifier;

determine that the first time precedes the second time;

determine that the first audio input further comprises a command responsive to determining that the second data packet includes the first device identifier and that the first time precedes the second time; and

process the command.

5. The computing device of claim 4, wherein the processor executes the program instructions to:

maintain, for a predetermined time period after determining that the first audio input comprises the wakeword, a connection to the access point device via a communication channel; and

receive at least the second data packet and the third data packet over the communication channel during the predetermined time period; and

subsequent to the predetermined time period elapsing, process the command.

6. The computing device of claim 4, wherein the processor executes the program instructions to:

determine a first quality value of the first audio input;

send the first data packet comprising the first quality value to the access point device;

receive the third packet comprising a second quality value generated from the second audio input representing the acoustic signal;

determine that the first quality value is approximately equal to or greater than the second quality value; and

determine that the first audio input further comprises a command responsive to determining that the first quality value is approximately equal to or greater than the second quality value.

7. The computing device of claim 6, wherein the first quality value comprises a first signal-to-noise ratio of the first audio input and the second quality value comprises a second signal-to-noise ratio of the second audio input.

8. The computing device of claim 6, wherein the first quality value comprises a first automatic gain control setting used to generate the first audio input, and the second quality value comprises a second automatic gain control setting used to generate the second audio input.

9. The computing device of claim 6, wherein the processor further executes the program instructions to:

receive a fourth data packet comprising a third quality value; and

determine that the third quality value is less than the first quality value.

10. The computing device of claim 6, wherein before the computing device executes the program instructions to process the command, the computing device executes the program instructions to:

determine that the first quality value is greater than a threshold quality value.

11. A method, comprising:

identifying a wakeword in a first audio input of a computing device disposed within an audible range of an acoustic signal, the first audio input representing the acoustic signal as detected by the computing device;

determining a first quality value of the first audio input;

generating a first data packet comprising the first quality value;

sending, at a first time, the first data packet to a first remote device over a first communication channel;

receiving, at a second time, a second data packet from the remote device, the second data packet comprising a second quality value generated from a corresponding remote audio input of one or more remote audio inputs representing the acoustic signal obtained by the first remote device disposed in the audible range;

determining that the first quality value is greater than or equal to second quality value of the one or more remote quality values;

determining that the first time precedes the second time;

determining that the first audio input further comprises a command responsive to determining that the first quality value is greater than or equal to the second quality value and that that the first time precedes the second time; and

processing the command.

12. The method of claim 11, wherein sending the first quality value to the first remote device comprises:

establishing a communication channel directly with the first remote device;

sending the first data packet over the communication channel to the first remote device; and

receiving the second data packet from the first remote device over the communication channel.

13. The method of claim 11, wherein determining the first quality value of the first audio input comprises computing a signal-to-noise ratio of the first audio input.

14. The method of claim 11, further comprising waiting a predetermined time period after identifying the wakeword in the first audio input, during which predetermined time period the second data packet is received.