Shared audio functionality based on device grouping

• US 9,916,839 B1
• Filed: 03/27/2014
• Issued: 03/13/2018
• Est. Priority Date: 03/27/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Scalise, Albert M., David, Tony
• Primary Examiner(s): Ky, Kevin
• Application Number:
  US14/227,227
• Time in Patent Office: 1,447 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

Techniques are described for shared audio functionality between multiple computing devices, based on grouping the multiple computing devices into a device set. The devices may provide audio output, audio input, or both audio output and input. The devices may discover each other via transmitted radio signals, and the devices may be organized into one or more device sets based on location, supported functions, or other criteria. The shared audio functionality may enable a voice command received at one device in the device set to be employed for controlling audio output or other operations of other device(s) in the device set. Shared audio functionality between devices in a device set may also enable synchronized audio output through using multiple devices in the device set.

Claims

1. A computer-implemented method, comprising:

receiving device set information stored in memory, the device set information including a list of computing devices that have shared audio functionality, the list of computing devices that have shared audio functionality including a first computing device and a second computing device, the device set information further including a list of functions supported by computing devices in the list of computing devices, the list of functions supported by the computing devices in the list of computing devices including an audio input function at the first computing device configured to receive a voice command and an audio output function of the second computing device configured to present output audio based at least in part on the voice command;

receiving, at the first computing device, the voice command including a selection, from the list of computing devices in the device set information, of the second computing device to present the output audio;

sending audio data representing the voice command over a network from the first computing device to one or more servers;

analyzing, at the one or more servers, the audio data to determine a description of the output audio to be presented using the audio output function of the second computing device that is described in the list of functions in the device set information;

determining, at the one or more servers and based at least in part on the audio data representing the voice command from the first computing device, that the first computing device and the second computing device are included in the list of computing devices;

based on the device set information, determining at the one or more servers that the voice command received from the first computing device controls the audio output function of the second computing device;

based on the receiving the voice command, generating, at the one or more servers, an instruction to direct the selected second computing device to present the output audio determined from the audio data according to the voice command; and

sending, over the network, the instruction to the selected second computing device to instruct the second computing device to present the output audio without being sent to the first computing device.

2. The method of claim 1, further comprising:

sending the output audio to the second computing device using a wireless peer-to-peer network, to enable the second computing device to present the output audio using the audio output function of the second computing device.

3. The method of claim 1, further comprising:

using a wireless peer-to-peer network to send a network location of the output audio to the second computing device, enabling the second computing device to retrieve the output audio from the network location and to present the output audio using the audio output function of the second computing device.

4. The method of claim 1, further comprising:

presenting the output audio using the audio output function of the second computing device, substantially synchronized with the presenting the output audio using the audio output function of the first computing device based on synchronization of a second clock of the second computing device to a first clock of the first computing device.

5. A system, comprising:

one or more servers configured to;

receive target device set information, stored in memory, that includes a list of computing devices associated with shared audio functionality, the list of computing devices including a receiving computing device and one or more target computing devices, the target device set information further including a list of functions supported by the receiving computing device and the one or more target computing devices, the list of functions supported by the receiving computing device and the one or more target computing devices including an audio input function at the receiving computing device configured to receive a voice command;

receive, at the one or more servers over a network, audio data representing the voice command from the receiving computing device, the voice command providing a selection of the one or more target computing devices from the list of computing devices of the target device set information, and a selection of an audio output function for the selected one or more target computing devices configured to present output audio based at least in part on the voice command;

analyze, by the one or more servers, the audio data to determine the output audio to be presented by a selected one or more target computing devices;

based on the target device set information, determine by the one or more servers that the voice command from the receiving computing device controls the audio output function of the selected one or more target computing devices;

based on the receiving the voice command, generate, by the one or more servers, an instruction to be sent by the one or more servers to the selected one or more target computing devices without being sent to the receiving computing device; and

send, using the network, the instruction to the selected one or more target computing devices without being sent to the receiving computing device, the instruction to instruct the selected one or more target computing devices to present the output audio.

6. The system of claim 5, wherein the sending of the instruction to the selected one or more target computing devices employs one or more wireless peer-to-peer networks established between the one or more servers and the selected one or more target computing devices.

7. The system of claim 5, wherein:

the analyzing the audio data representing the voice command includes performing speech recognition to determine, from the list of computing devices of the target device set information, the identification of the selected one or more target computing devices from the voice command as an utterance included in the voice command;

the device set information includes a section associated with the target device set information, the section delineated by a first set of one or more metadata tags; and

the section associated with the target device set information includes a second set of one or more metadata tags that describes the selected one or more target computing devices.

8. The system of claim 7, wherein:

the second set of one or more metadata tags includes an attribute indicating one or more functions of the selected one or more target computing devices; and

the determining that the selected one or more target computing devices are configured to provide the audio output function includes determining that the attribute indicates the audio output function for the selected one or more target computing devices.

9. The system of claim 5, wherein:

the generating the instruction further comprises;

performing speech recognition to determine an utterance included in the audio data representing the voice command, the utterance including the description of the output audio to be presented; and

determining the instruction to send to the selected one or more target computing devices to present the output audio content, the instruction being associated with the utterance included in the audio data representing the voice command.

10. The system of claim 5, wherein the one or more servers are further configured to:

request that the output audio be sent from a distributed computing device to the selected one or more target computing devices, to enable the selected one or more target computing devices to present the output audio using the audio output function of the selected one or more target computing devices.

11. The system of claim 5, wherein the network is compliant with at least a portion of version of an Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard.

12. The system of claim 5, wherein the one or more servers are further configured to:

send a network location of the output audio to the selected one or more target computing devices using one or more wireless peer-to-peer networks, enabling the selected one or more target computing devices to retrieve the output audio from the network location and to present the output audio using the audio output function of the selected one or more target computing devices.

13. The system of claim 5, wherein the receiving computing device is in the list of computing devices of the target device set information.

14. One or more non-transitory computer-readable media storing instructions which, when executed by at least one processor of one or more servers, instruct the at least one processor to perform actions comprising:

receiving, at the one or more servers, a target device set that includes a list of computing devices that have shared audio functionality, the list of computing devices including a receiving computing device and one or more target computing devices enabling shared audio functionality, the target device set further including a list of functions supported by the receiving computing device and the one or more target computing devices including an audio input function at the receiving computing device configured to receive a voice command;

receiving, at the receiving computing device, the voice command providing a selection of the one or more target computing devices from the list of computing devices of the target device set, and a selection of an audio output function of the selected one or more target computing devices for presenting output audio based at least in part on the voice command;

receiving, at the one or more servers from the receiving computing device, audio data representing the voice command;

analyzing, by the one or more servers, the audio data to determine;

the output audio to be presented at the selected one or more target computing devices according to the voice command using the selected audio output function of the one or more target computing devices in the target device set;

responsive to the audio data representing the voice command, determining the list of computing devices of target device set includes the one or more target computing devices and the receiving computing device, the one or more target computing devices selected from the list of computing devices of the target device set according to the voice command configured to provide the audio output function determined from the audio data;

based on the target device set, determining at the one or more servers that the voice command from receiving computing device controls the audio output function of the selected one or more target computing devices;

based on the receiving the voice command, generating an instruction to direct the selected one or more target computing devices to present the output audio using the audio output function of the selected one or more target computing devices; and

sending, using the network, the instruction to the selected one or more target computing devices without being sent to the receiving computing device, the instruction to instruct the selected one or more target computing devices to present the output audio associated with a description of the output audio determined from the audio data from the receiving computing device using the audio output function of the selected one or more target computing devices.

15. The one or more non-transitory computer-readable media of claim 14, wherein the sending of the instruction to the selected one or more target computing devices employs one or more wireless peer-to-peer networks established between one or more servers, the receiving computing device and the selected one or more target computing devices.

16. The one or more non-transitory computer-readable media of claim 14, wherein:

the selected one or more target computing devices includes a plurality of target computing devices; and

the instruction instructs the plurality of target computing devices to present the output audio in a substantially synchronized playback based on synchronization of a first clock of the selected one or more target computing devices to a second clock of the receiving computing device.

17. The one or more non-transitory computer-readable media of claim 14, wherein:

the selected one or more target computing devices includes a plurality of target computing devices; and

the instruction instructs the plurality of target computing devices to present the output audio serially, such that different subsets of the plurality of target computing devices present the output audio during non-overlapping time periods.

18. The one or more non-transitory computer-readable media of claim 14, the actions further comprising:

requesting that the output audio be sent from a distributed computing device to the receiving computing device; and

sending the output audio from the receiving computing device to the selected one or more target computing devices, to enable the selected one or more target computing devices to present the output audio using the audio output function of the selected one or more target computing devices.

19. The one or more non-transitory computer-readable media of claim 18, wherein the sending of the output audio from the receiving computing device to the selected one or more target computing devices employs one or more wireless peer-to-peer networks established between the receiving computing device and the selected one or more target computing devices.

20. The one or more non-transitory computer-readable media of claim 14, the actions further comprising:

sending a network location of the output audio from the receiving computing device to the selected one or more target computing devices using one or more wireless peer-to-peer networks, enabling the selected one or more target computing devices to retrieve the output audio from the network location and to present the output audio using the audio output function of the selected one or more target computing devices.