Presentation and management of audio and visual content across devices

• US 10,531,157 B1
• Filed: 09/21/2017
• Issued: 01/07/2020
• Est. Priority Date: 09/21/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Loritsch, Michael Lee, Miller, John Martin, Kotas, Paul Anthony, Tucker, Ross
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US15/710,911
• Time in Patent Office: 838 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand

Abstract

Systems, methods, and computer-readable media are disclosed for systems and methods of presentation and management of audio and visual content across devices. Example methods may include causing presentation of first audio content at a speaker device, causing presentation of a first audio notification indicative of visual content available for presentation, causing presentation of second audio content after the first audio notification, and sending first visual content to a first display device for presentation during presentation of the second audio content.

Claims

1. A method comprising:

detecting, by one or more computer processors coupled to at least one memory device, an utterance from a user using a microphone at a speaker device;

determining that the utterance comprises a trigger word followed by an audible request, wherein the trigger word indicates a type of request;

determining that the audible request is a request for a first audio stream;

causing presentation of the first audio stream at the speaker device in response to the audible request;

determining a content category of the first audio stream;

determining first visual content for presentation to the user using the content category;

determining a first display device for presentation of the first visual content, wherein the first display device is associated with a user account of the user;

pausing playback of the first audio stream;

causing presentation of a first audio notification indicating that the first visual content is available for consumption at the first display device;

resuming playback of the first audio stream;

causing presentation of the first visual content at the first display device;

determining a first user interaction with the first visual content at the first display device;

causing presentation of second visual content at the first display device;

determining that a user location has changed in an ambient environment;

determining a second display device associated with the user account;

causing presentation of the second visual content at the second display device;

ceasing playback of the first audio stream;

determining a second user interaction with the second visual content at the second display device; and

causing third visual content to be presented at the second display device, the third visual content indicating confirmation of an order placed by the user.

2. The method of claim 1, wherein the first audio notification is an audible question, the method further comprising:

determining an audible response to the audible question using the microphone; and

determining that the audible response is an affirmative audible response.

3. The method of claim 1, wherein determining that the user location has changed comprises:

determining, while the second visual content is presented at the first display device, a third user interaction with the second display device.

4. The method of claim 1, wherein determining that the user location has changed comprises determining that the user location has changed using a camera at the second display device, the method further comprising:

determining that the second display device is physically closer to the user location than the first display device.

5. A method comprising:

receiving, by one or more computer processors coupled to at least one memory device, voice input;

causing presentation of first audio content at a speaker device in response to the voice input;

causing analysis of the voice input to determine that a trigger word that indicates a type of request is included in the voice input;

causing presentation of a first audio notification indicative of visual content available for presentation based at least in part on the trigger word;

causing presentation of second audio content after the first audio notification; and

sending first visual content to a first display device for presentation during presentation of the second audio content, wherein the first display device is associated with a user account.

6. The method of claim 5, further comprising:

determining a first user interaction with a second display device; and

sending the first visual content to the second display device.

7. The method of claim 5, further comprising:

determining a first user interaction with the first visual content at the first display device;

sending second visual content to the first display device;

determining a second user interaction with a second display device; and

sending the second visual content to the second display device.

8. The method of claim 5, further comprising:

sending the first visual content to a second display device for presentation during presentation of the second audio content.

9. The method of claim 5, further comprising:

determining that a user is at a first location physically closest to the first display device;

determining that the user has moved to a second location physically closest to a second display device; and

sending the first visual content to the second display device.

10. The method of claim 5, further comprising:

determining a second display device associated with the user account;

determining second visual content that is associated with the first visual content; and

sending the second visual content to the second display device.

11. The method of claim 5, further comprising:

determining that the first display device is in an active state.

12. The method of claim 5, further comprising:

determining that a user is present in an ambient environment of the first display device.

13. The method of claim 5, further comprising:

determining, while the first visual content is presented at the first display device, a first user interaction with a second display device; and

sending the first visual content to the second display device.

14. The method of claim 5, wherein the first audio notification comprises an audible question, the method further comprising:

determining an affirmative audible response to the audible question using one or more microphones.

15. The method of claim 5, wherein a first content category the first visual content is unrelated to a second content category of the first audio content.

16. A system comprising:

a speaker device comprising a microphone;

a first display device;

at least one memory that stores computer-executable instructions; and

at least one processor configured to access the at least one memory and execute the computer-executable instructions to;

receive voice input;

cause presentation of first audio content at a speaker device in response to the voice input;

cause analysis of the voice input to determine that a trigger word that indicates a type of request is included in the voice input;

cause presentation of a first audio notification indicative of visual content available for presentation based at least in part on the trigger word;

cause presentation of second audio content after the first audio notification; and

send first visual content to a first display device for presentation during presentation of the second audio content, wherein the first display device is associated with a user account.

17. The device of claim 16, wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to:

determine a first user interaction with a second display device; and

send the first visual content to the second display device.

18. The device of claim 16, wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to:

determine a first user interaction with the first visual content at the first display device;

send second visual content to the first display device;

determine a second user interaction with a second display device; and

send the second visual content to the second display device.

19. The device of claim 16, wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to:

determine that a user is at a first location physically closest to the first display device;

determine that the user has moved to a second location physically closest to a second display device; and

send the first visual content to the second display device.

20. The device of claim 16, wherein the at least one processor is further configured to access the at least one memory and execute the computer-executable instructions to:

determine a second display device associated with the user account;

determine second visual content that is associated with the first visual content; and

send the second visual content to the second display device.