Message and user profile indications in speech-based systems

• US 10,210,885 B1
• Filed: 05/20/2014
• Issued: 02/19/2019
• Est. Priority Date: 05/20/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Carlson, Ty Loren
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Assistant Examiner(s): Wang, Yi Sheng
• Application Number:
  US14/282,821
• Time in Patent Office: 1,736 Days
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

A speech-based system utilizes a speech interface device located in the home of a user. The system may interact with different users based on different user profiles. The system may include messaging services that generate and/or provide messages to the user through the speech interface device. The speech interface device may have indicators that are capable of being illuminated in different colors. To notify a user regarding the currently active user profile, each user profile is associated with a different color and the color of the active profile is displayed on the speech interface device when the user is interacting with the system. To notify the user regarding awaiting messages, different types of messages are associated with different colors and the colors of the message types of waiting messages are displayed on the speech interface whenever the user is not interacting with the system.

Claims

1. A system, comprising:

one or more processors; and

one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

storing a user profile associated with a user;

receiving a message that is associated with a message type of a plurality message types;

associating a first visual indication of multiple visual indications with the message type, the first visual indication including at least one of a first light sequence, a first light pattern, or a first light color;

associating a second visual indication of the multiple visual indications with the user profile, the second visual indication including at least one of a second light sequence, a second light pattern, or a second light color;

causing, via a network, a speech interface device to display the first visual indication associated with the message type to indicate availability of the message;

receiving, from the speech interface device, audio data representing user speech;

identifying, based at least in part on the audio data, that the user is interacting with the speech interface device;

based at least in part on identifying that the user is interacting with the speech interface device, causing, via the network, the speech interface device to display the second visual indication concurrently with the first visual indication;

receiving, from the speech interface device via the network, a request to output the message;

causing, via the network, the speech interface device to output the message; and

causing, via the network, the speech interface device to cease displaying the first visual indication based at least in part on causing the speech interface device to output the message.

2. The system of claim 1, wherein the system is located remotely from the speech interface device and is configured to communicate with the speech interface device over a wide-area communications network.

3. The system of claim 1, wherein:

causing the speech interface device to display the first visual indication is performed at a time at which the user is not interacting with the system.

4. The system of claim 1, wherein:

the first light color is produced by one or more first lights;

the first light sequence is associated with the one or more first lights;

or

the first light pattern is associated with the one or more first lights; and

the second light color is produced by one or more second lights;

the second light sequence is associated with the one or more second lights;

or

the second light pattern is associated with the one or more second lights.

5. The system of claim 1, wherein the plurality of message types include one or more of:

system messages;

email;

text messages;

error messages;

voice messages;

event messages;

reminders;

service notifications;

social networking messages;

social networking notifications;

social networking status updates;

messages from different senders;

or

messages from different categories of senders.

6. The system of claim 1, wherein the user profile indicates one or more of the following with respect to the user:

preferences of the user;

configurations for use when interacting with the user;

characteristics of the user;

settings for use when interacting with the user;

data sets that have been configured by the user;

data resources of the user;

data resources associated with the user;

activity histories;

account information;

language models;

speech models;

security authorizations;

age of the user;

geographic location of the user;

permissions granted to the user;

user restrictions;

allowed activities;

operating modes;

or

security clearances or other security information.

7. The system of claim 1, wherein causing the speech interface device to display the second visual indication concurrently with the first visual indication comprises:

causing the speech interface device to display the first visual indication using a first segment of a display element; and

causing the speech interface device to display the second visual indication using a second segment of the display element.

8. A method comprising:

associating, by one or more remote computing devices, a first visual indication with a message type of multiple message types, the first visual indication including at least one of a first light sequence, a first light pattern, or a first light color;

receiving, by the one or more remote computing devices, a message having the message type;

causing, by the one or more remote computing devices, via a network, a speech interface device to display the first visual indication associated with the message type of the message to indicate availability of the message;

receiving, by the one or more remote computing devices, audio data representing user speech from the speech interface device;

identifying, by the one or more remote computing devices, and based at least in part on the audio data, that a user associated with a user profile is interacting with the speech interface device;

associating, by the one or more remote computing devices, a second visual indication with the user profile, the second visual indication including at least one of a second light sequence, a second light pattern, or a second light color;

causing, by the one or more remote computing devices, via the network, the speech interface device to display the second visual indication concurrently with the first visual indication;

receiving, by the one or more remote computing devices, and from the speech interface device via the network, a request to play the message;

causing, by the one or more remote computing devices, via the network, the speech interface device to play the message; and

causing, by the one or more remote computing devices, via the network, the speech interface device to cease displaying the first visual indication based at least in part on causing the speech interface device to play the message.

9. The method of claim 8, wherein causing the speech interface device to display the first visual indication comprises controlling individual lights of a multi-light array.

10. The method of claim 8, further comprising:

receiving, by the one or more remote computing devices, an additional message having an additional message type of the multiple message types; and

causing, by the one or more remote computing devices, via the network, the speech interface device to concurrently display the first visual indication associated with the message type of the message and a third visual indication associated with the additional message type of the additional message to indicate availability of the message and the additional message.

11. The method of claim 8, further comprising:

receiving an additional message having an additional message type of the multiple message types;

causing the speech interface device to concurrently display the first visual indication associated with the message type of the message and a third visual indication associated with the additional message type of the additional message to indicate availability of the message and the additional message;

receiving an additional request to play the additional message;

causing the speech interface device to play the additional message; and

causing the speech interface device to cease displaying the third visual indication associated with the additional message type based at least in part on causing the speech interface device to play the additional message.

12. The method of claim 8, wherein the multiple message types include one or more of:

system messages;

email;

text messages;

error messages;

voice messages;

event messages;

reminders;

service notifications;

social networking messages;

social networking notifications;

social networking status updates;

messages from different senders;

or

messages from different categories of senders.

13. The method of claim 8, wherein causing the speech interface device to display the first visual indication comprises at least one of:

illuminating one or more light elements of a plurality of light elements of the speech interface device according to the first light sequence;

illuminating the one or more light elements of the plurality of light elements at a same time;

or

flashing the one or more light elements of the plurality of light elements at one or more rates.

14. The method of claim 8, wherein causing, by the one or more remote computing devices, via a network, the speech interface device to display the second visual indication concurrently with the first visual indication comprises:

causing, by the one or more remote computing devices, via the network, the speech interface device to display the first visual indication using a first segment of a display element; and

causing, by the one or more remote computing devices, via the network, the speech interface device to display the second visual indication using a second segment of the display element.

15. One or more non-transitory computer storage media storing instructions that, when executed by one or more processors, perform operations comprising:

associating a first visual indication with a message type of multiple message types, the first visual indication including at least one of a first light sequence, a first light pattern, or a first light color;

receiving a message having the message type;

causing, via a network, a speech interface device to display the first visual indication associated with the message type of the message to indicate availability of the message;

receiving audio data representing user speech from the speech interface device;

identifying, based at least in part on the audio data, that a user associated with a user profile is interacting with the speech interface device;

associating a second visual indication with the user profile, the second visual indication including at least one of a second light sequence, a second light pattern, or a second light color;

causing, via the network, the speech interface device to display the second visual indication concurrently with the first visual indication;

receiving, from the speech interface device via the network, a request to play the message;

causing, via the network, the speech interface device to play the message; and

causing, via the network, the speech interface device to cease displaying the first visual indication based at least in part on causing the speech interface device to play the message.

16. The one or more non-transitory computer storage media of claim 15, wherein the network is a wide-area communications network, and the one or more non-transitory computer storage media are part of a system that is located remotely from the speech interface device, the system configured to communicate with the speech interface device over the wide-area communications network.

17. The one or more non-transitory computer storage media of claim 15, wherein:

the first light sequence is associated with one or more first lights and the second light sequence is associated with one or more second lights;

the first light pattern is associated with the one or more first lights and the second light pattern is associated with the one or more second lights;

or

the first light color is produced by the one or more first lights and the second light color is produced by the one or more second lights.

18. The one or more non-transitory computer storage media of claim 15, wherein causing, via the network, the speech interface device to display the second visual indication concurrently with the first visual indication comprises:

causing, via the network, the speech interface device to display the first visual indication using a first segment of a display element; and

causing, via the network, the speech interface device to display the second visual indication using a second segment of the display element.

19. The one or more non-transitory computer storage media of claim 15, wherein causing the speech interface device to display the first visual indication comprises controlling individual lights of a multi-light array.

20. The one or more non-transitory computer storage media of claim 15, the operations further comprising:

receiving an additional message having an additional message type of the multiple message types; and

causing, via the network, the speech interface device to concurrently display the first visual indication associated with the message type of the message and a third visual indication associated with the additional message type of the additional message to indicate availability of the message and the additional message.

21. The one or more non-transitory computer storage media of claim 15, wherein causing the speech interface device to display the first visual indication comprises at least one of:

illuminating one or more light elements of a plurality of light elements of the speech interface device according to the first light sequence;

illuminating the one or more light elements of the plurality of light elements at a same time;

or

flashing the one or more light elements of the plurality of light elements at one or more rates.