Audio-based application architecture

• US 10,382,509 B2
• Filed: 01/28/2011
• Issued: 08/13/2019
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Williams, Hunter, Carr, William D., Gasper, Kevin J., Janes, Cameron, Watts, Andrew J., Wood, James H.
• Primary Examiner(s): To, Jennifer N
• Assistant Examiner(s): Fortino, Ashley M
• Application Number:
  US13/016,009
• Publication Number:
  US 20120198339A1
• Time in Patent Office: 3,119 Days
• Field of Search: 715727
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637

Abstract

An application architecture comprises one or more audio interfaces placed within the premises of users. A cloud-based application engine receives audio information from the interfaces and provides information to cloud-based applications based on the audio within the user premises. The other applications utilize the information to provide or enhance services to the users.

Claims

1. A system comprising:

one or more processors;

one or more network interfaces; and

memory storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to performs acts comprising;

receiving, by a first application hosted by a remote computing device, an indication that a second application is authorized to receive data from the first application;

receiving, by the first application hosted by the remote computing device, and based at least in part on ambient noise levels exceeding a predetermined threshold, audio data corresponding to at least one of a song, a television show, or a movie, the audio data received from at least one on premises audio monitoring device via the one or more network interfaces;

analyzing, by the first application hosted by the remote computing device, the audio data to identify at least one of the song, the television show, or the movie;

providing, via the one or more network interfaces and based at least in part on receiving the indication, first data to the second application, the first data identifying at least one of the song, the television show, or the movie;

receiving, by the first application hosted by the remote computing device via the one or more network interfaces, second data from the second application, the second data corresponding to, but different than, at least one of the song, the television show, or the movie; and

sending, by the first application hosted by the remote computing device via the one or more network interfaces to a second on premises device associated with an on-premises user, the second data, the second on premises device different than the on premises audio monitoring device.

2. The system of claim 1, wherein the second data corresponds to at least one of the song, the television show, or the movie sent to the device corresponds to audio.

3. The system of claim 1, wherein the first data also corresponds to the on-premises user, and wherein the one or more processors further perform acts comprising:

selecting, based at least in part on the on-premises user, the second data corresponding to at least one of the song, the television show, or the movie.

4. The system of claim 1, wherein:

providing the first data includes providing the first data to the first application and the second application; and

receiving the second data includes receiving the second data from at least the first application or the second application.

5. A method comprising:

under control of one or more remote computing devices configured with executable instructions,

receiving, by a first application hosted by one or more remote computing devices, an indication that a second application is authorized to receive input data from the first application;

receiving, by the first application and based at least in part on ambient noise levels exceeding a predetermined threshold, audio data from at least one audio monitoring device located at a first premises;

processing, by the one or more remote computing devices, the audio data to identify a media item presented at the first premises;

reporting, via one or more network interfaces of the one or more remote computing devices and based at least in part on receiving the indication, first data associated with the media item to multiple network-based applications, the multiple network-based applications providing services corresponding to the media item and including the second application, the first data including an identifier of the media item;

receiving, by the first application hosted by the one or more remote computing devices, second data from the second application, the second data corresponding to, but different than, the first data and the audio data; and

sending, by the first application hosted by the one or more remote computing devices, the second data to an additional device, the second device located at the first premises.

6. The method of claim 5, wherein the audio data comprises a plurality of audio streams at the first premises, the method further comprising:

processing the audio streams to identify the first data associated with the media item.

7. The method of claim 5, further comprising processing the audio data to identify current user media consumption.

8. The method of claim 5, further comprising performing voice recognition on the audio data to identify at least one user command.

9. The method of claim 5, further comprising sending, by the first application hosted by the one or more remote computing devices, an audio response to the at least one audio monitoring device located at the first premises.

10. The method of claim 5, wherein the response is provided at the second premises to a personal device associated with the user.

11. The method of claim 5, wherein the media item presented at the first premises is one of a song, a television show, or a movie.

12. The method of claim 5, further comprising prior to reporting data to multiple network-based applications, determining that the user has authorized the multiple network-based applications to receive the data.

13. A network-based system, comprising:

one or more processors; and

memory containing instructions that are executable by the one or more processors to perform acts comprising;

receiving, from a first application, an indication that a second application is authorized to receive data from the first application;

receiving, based at least in part on ambient noise levels exceeding a predetermined threshold, audio data from a first remote network-based service associated with a user, the audio data captured by an audio monitoring device within a first premises;

processing, by one or more remote computing devices, the audio data to identify a current activity associated with the user;

reporting, based at least in part on receiving the indication, first data associated with the media item to the second application, the second application providing services corresponding to the media item, the first data including an identifier of the media item;

receiving, second data from the second application, the second data corresponding to, but different than the audio data; and

sending, based at least in part on data regarding the current activity, the second data to a second device associated with the user, the second device different than the audio monitoring device.

14. The network-based system of claim 13, wherein the data regarding the current activity associated with the user indicates current media consumption associated within the first premises.

15. The network-based system of claim 13, wherein:

the data regarding the current activity associated with the user indicates current media consumption of the user.

16. The network-based system of claim 13, wherein:

the data regarding the current activity associated with the user indicates at least one spoken command within the first premises; and

providing to additional remote network-based services the data regarding the current activity associated with the user to is further based at least in part on the at least one spoken command.

17. The network-based system of claim 13, wherein the current activity associated with the user is at least one of watching a movie, watching a television show, or listening to a song.

18. The network-based system of claim 13, wherein the one or more processors further perform acts comprising:

determining, based at least in part on identifying the current activity, one or more stored user preferences corresponding to the current activity; and

wherein providing the network-based services is further based at least in part on the one or more stored user preferences corresponding to the current activity.

19. A device, comprising:

a microphone configured to capture audio data from a premises associated with a user; and

a network interface configured to;

send an authorization to allow a first application service to communicate with a second application service, the first application service residing on first computing resources remote from the premises and the second application service residing on second computing resources remote from the premises;

determine that an ambient noise level exceeds a threshold;

transmit, based at least in part on determining the ambient noise level exceeds the threshold, the audio data to the first application service;

receive first data from the first application service, the second data related to, but different from, the audio data and generated by the second application service; and

cause one or more application interfaces of a device associated with the user and within the premises to display the second data to the user;

wherein the second data is audio data including unprocessed audio data.

20. The device of claim 19, further comprising:

a speech recognizer,

wherein the second data comprises notifications derived at least in part from user utterances using the speech recognizer.

21. The device of claim 19, further comprising:

a speech recognizer,

wherein the second data comprises media identifications.

22. The device of claim 19, wherein the first data is associated with a request to purchase goods and a preferred delivery schedule.

23. The device of claim 19, wherein the first data is textual directions.