Audio-Based Application Architecture

• US 20120198339A1
• Filed: 01/28/2011
• Published: 08/02/2012
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carr, William D., Wood, James H., Williams, Hunter, Gasper, Kevin J., Janes, Cameron, Watts, Andrew J.
• Granted Patent: US 10,382,509 B2
• Time in Patent Office: Days
• US Class Current: 715/727
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637

Abstract

An application architecture comprises one or more audio interfaces placed within the premises of users. A cloud-based application engine receives audio information from the interfaces and provides information to cloud-based applications based on the audio within the user premises. The other applications utilize the information to provide or enhance services to the users.

Claims

1. A system comprising:

one or more network interfaces;

a cloud-based service that is accessible by on-premises audio monitoring devices to receive audio-based information via the one or more network interfaces from multiple user premises, the user premises being associated respectively with users; and

wherein the cloud-based service exposes one or more application interfaces to multiple cloud-based applications that provide services to the users based at least in part on the audio-based information.

2. The system of claim 1, wherein the audio-based information comprises audio streams.

3. The system of claim 1, wherein the audio-based information comprises user commands derived from vocal utterances of the users.

4. The system of claim 1, wherein the audio-based information comprises environmental information derived at least in part from on-premises audio.

5. The system of claim 1, wherein the one or more application interfaces provide information regarding current user-specific activity based at least in part on the audio-based information.

6. The system of claim 1, wherein the one or more application interfaces provide information regarding current user media consumption based at least in part on the audio-based information.

7. The system of claim 1, wherein the one or more application interfaces provide information regarding current user video consumption based at least in part on the audio-based information.

8. The system of claim 1, wherein the one or more application interfaces provide information regarding current user audio consumption based at least in part on the audio-based information.

9. The system of claim 1, wherein the cloud-based service is responsive via the one or more application interfaces to provide audio to the on-premises audio monitoring devices at least partly in response to the cloud-based applications.

10. A method comprising:

under control of one or more computing systems configured with executable instructions,

receiving audio-based information from on-premises audio monitoring devices located on premises of multiple users;

processing the audio-based information to determine current user-specific information; and

reporting the current user-specific information to multiple network-based applications that provide services to the users based at least in part on the user-specific information.

11. The method of claim 10, wherein the audio-based information comprises continuously-received audio streams from the multiple users within their respective premises.

12. The method of claim 10, wherein the audio-based information comprises audio streams, the method further comprising processing the audio streams to identify the current user-specific information.

13. The method of claim 10, wherein the audio-based information comprises audio streams, the method further comprising processing the audio streams to identify current user media consumption.

14. The method of claim 10, wherein the audio-based information comprises audio streams, the method further comprising performing voice recognition on the audio streams to identify user commands.

15. The method of claim 10, wherein the audio-based information specifies current user media consumption.

16. The method of claim 10, further comprising providing audio to the on-premises audio monitoring devices at least partly in response to the network-based applications.

17. A network-based system, comprising:

one or more processors;

memory containing instructions that are executable by the one or more processors to perform actions comprising;

receiving information from a network-based service regarding current activities of multiple users in multiple user premises, wherein the information is based at least in part on audio monitoring from within the user premises; and

providing network-based services to the users based at least in part on the received information regarding the activities in their respective user premises.

18. The system of claim 17, wherein the received information indicates current media consumption of individual users.

19. The system of claim 17, wherein:

the received information indicates current media consumption of individual users; and

the network-based services supplement the current media consumption.

20. The system of claim 17, wherein:

the received information indicates spoken commands from individual users; and

the network-based services act in response to the spoken commands.

21. An in-home interface device, comprising:

a microphone configured to receive audio from a nearby user and from nearby media devices;

a network interface configured to communicate with a network-based application service;

wherein the in-home interface device is configured to continuously transmit environmental information to one or more network-based applications via the network interface and the network-based application service, and wherein the environmental information is based at least in part on the received audio.

22. The device of claim 21, wherein the environmental information comprises an audio stream.

23. The device of claim 21, further comprising a speech recognizer, and wherein the environmental information comprises notifications derived at least in part from user utterances using the speech recognizer.

24. The device of claim 21, further comprising a speech recognizer, and wherein the environmental information comprises media identifications.