CONTEXT AWARE, PROACTIVE DIGITAL ASSISTANT

• US 20150162000A1
• Filed: 12/10/2013
• Published: 06/11/2015
• Est. Priority Date: 12/10/2013
• Status: Abandoned Application

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: DI CENSO, Davide, MARTI, Stefan, JUNEJA, Ajay
• Application Number:
  US14/102,033
• Publication Number:
  US 20150162000A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/635 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication

Abstract

A digital assistant system may be used to proactively retrieve and provide information to a user based on an identified context of the user'"'"'s environment and/or activities. By monitoring a user'"'"'s environment and/or activities, an inference can be made about what supplemental information the user may find helpful and/or relevant. The supplemental information can be proactively retrieved and presented to the user. The system can include one or more sensors to gather information about the user'"'"'s environment and/or activities. The system can also include one or more output modules for proactively providing information to the user, such as a speaker for providing information in an auditory format and a display for providing information in a visual format. The system can be wearable and always on so that it can quickly and unobtrusively provide information to the user.

Claims

1. A system for proactively providing information, comprising:

a sensor, wearable by a user, configured to detect at least one of visual data and auditory data associated with an environment of the user;

an output module configured to present information to the user; and

control logic configured to perform an operation comprising;

identifying a context of the environment of the user from the detected at least one of visual data and auditory data associated with the environment of the user;

retrieving information relevant to the identified context; and

proactively presenting the retrieved information to the user through the output module.

2. The system of claim 1, wherein the output module comprises an speaker configured to be positioned in or proximate to an ear canal of the user, and wherein proactively presenting the retrieved information to the user comprises presenting the retrieved information in an audio format to the user through the speaker.

3. The system of claim 1, wherein the output module comprises a visual display, wearable by the user, configured to be positioned in a line of sight of the user, and wherein proactively presenting the retrieved information to the user comprises presenting the retrieved information in a video format to the user through the visual display.

4. The system of claim 1, wherein the wearable sensor comprises a microphone configured to detect the voice of the user, and wherein identifying a context of the environment of the user comprises:

detecting the spoken words of the user; and

identifying a context related to at least one of the spoken words.

5. The system of claim 4, wherein the system proactively presents the retrieved information when the microphone does not detect the voice of the user for a predetermined period of time.

6. The system of claim 1, wherein the sensor comprises a microphone configured to detect sounds from the environment of the user, and wherein identifying a context of the environment of the user comprises:

detecting at least one of sounds and voices in the environment of the user; and

identifying a context related to at least one of the sounds and voices.

7. The system of claim 6, wherein the system proactively presents the retrieved information when the microphone does not detect the at least one of sounds and voices in the user'"'"'s environment for a predetermined period of time.

8. The system of claim 1, wherein the wearable sensor comprises an imaging sensor configured to capture images of the environment of the user, and wherein identifying a context of the environment of the user comprises:

capturing at least one image in the environment of the user; and

identifying a context related to at least one image in the environment of the user.

9. The system of claim 1, wherein the sensor, output module, and control logic are integrated into a common housing, the housing being wearable by the user.

10. A method for proactively providing information, comprising:

detecting at least one of visual data and auditory data associated with an environment of a user using environmental sensors associated with the user;

identifying a context of the environment of the user from the detected at least one of visual data and auditory data associated with the environment of the user based on a relavancy between the detected aspect and an attribute associated with the user;

retrieving information relevant to the identified context; and

proactively presenting the retrieved information to the user.

11. The method of claim 10, wherein proactively presenting the retrieved information to the user comprises presenting the retrieved information to the user in an audio format.

12. The method of claim 10, wherein proactively presenting the retrieved information to the user comprises presenting the retrieved information to the user in a visual format.

13. The method of claim 10, wherein detecting at least one of the visual data and auditory data associated with the environment of the user comprises detecting spoken words, and wherein identifying a context of the environment of the user comprises identifying a context related to at least one of the spoken words.

14. The method of claim 13, wherein proactively presenting the retrieved information to the user comprises presenting the retrieved information to the user after spoken words are not detected for a predetermined period of time.

15. The method of claim 10, wherein detecting at least one of visual data and auditory data associated with the environment of the user comprises capturing at least one image of the environment of the user, and wherein identifying a context of the environment of the user comprises identifying a context related to at least of the captured images.

16. The method of claim 15, wherein proactively presenting the retrieved information to the person comprises presenting the retrieved information to the person after a captured image is relatively unchanged over a sequence of captured images.

17. A computer program product for proactively providing information, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising;

computer-readable program code configured to capture environmental context data associated with a user;

computer-readable program code configured to identify an activity of a user within the captured context data;

computer-readable program code configured to evaluate the activity to determine whether to provide supplemental information regarding the activity to the user;

computer-readable program code configured to, upon determining to provide the supplemental data, retrieve supplemental information associated with the activity; and

computer-readable program code configured to proactively present the retrieved information to the user.

18. The computer program product of claim 17, wherein the computer-readable program code is configured to present the retrieved information to the user as an audio presentation.

19. The computer program product of claim 17, wherein evaluating the activity to determine whether to provide supplemental information comprises:

characterizing the activity based on a temporal time value or an activity type associated with the activity; and

comparing the characterization of the activity to a relevance threshold, wherein if the characterization satisfies the relevance threshold, indicates that the supplemental information should be retrieved, otherwise, the supplemental information should not be retrieved.

20. The computer program product of claim 17, wherein the computer-readable program code configured is configured to detect spoken words from the captured environmental data, and identify the activity based on the spoken words.