PERCEPTUAL COMPUTING WITH CONVERSATIONAL AGENT

• US 20130212501A1
• Filed: 12/21/2012
• Published: 08/15/2013
• Est. Priority Date: 02/10/2012
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Barak Hurwitz
• Inventors: Anderson, Glen J., Kamhi, Gila, Mongia, Rajiv K., Govezensky, Yosi, Hurwitz, Barak, Moran, Amit, Ferens, Ron
• Application Number:
  US13/724,992
• Publication Number:
  US 20130212501A1
• Time in Patent Office: Days
• US Class Current: 715/764
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Perceptual computing with a conversational agent is described. In one example, a method includes receiving a statement from a user, observing user behavior, determining a user context based on the behavior, processing the user statement and user context to generate a reply to the user, and presenting the reply to the user on a user interface.

Claims

1. A method comprising:

receiving a statement from a user;

observing one or more physical contextual cues;

determining a user context based on the observed physical contextual cues;

processing the user statement and user context to generate a reply to the user; and

presenting the reply to the user on a user interface.

2. The method of claim 1, wherein observing physical contextual cues comprises at least one of observing facial expressions, observing eye movements, observing gestures, measuring biometric data, and measuring tone or volume of speech.

3. The method of claim 1, further comprising receiving user history activity information determined based on at least one of e-mail content, messaging content, browsing history, location information, and personal data and wherein processing comprises processing the user statement and user mood with the user history activity information.

4. The method of claim 1 wherein receiving a statement from a user comprises receiving a statement on a user device and sending the statement and the additional information to a remote server.

5. The method of claim 1, wherein receiving a statement from a user comprises receiving a spoken statement through a microphone and converting the statement to text.

6. The method of claim 1, further comprising receiving additional information by determining a location of a user using a location system of a user terminal and wherein processing includes using the determined location.

7. The method of claim 1, wherein processing comprises weighing the statement based on the determined user mood.

8. The method of claim 7, wherein determining a user context comprises measuring user properties using biometric sensors.

9. The method of claim 7, wherein determining a user context comprises analyzing facial expressions received in a camera.

10. The method of claim 1, wherein determining a user context comprises determining a user attention to the user interface and wherein processing comprises weighing the statement based on the determined user attention.

11. The method of claim 10, further comprising determining whether a statement is directed to the user interface using the determined user attention and, if not, then not generating a reply to the statement.

12. The method of claim 11, wherein if the statement is not directed to the user interface, then recording the statement to provide background information for subsequent user statements.

13. The method of claim 1, further comprising receiving the statement and additional information at a server from a user terminal and wherein processing comprises generating a conversational reply to the user and sending the reply from the server to the user terminal.

14. The method of claim 13, further comprising selecting a database to use in generating a reply based on the content of the user statement.

15. The method of claim 14, wherein the selected database is one of a conversational database and a navigational database.

16. The method of claim 1, wherein presenting the reply comprises presenting the reply using an avatar as a conversational agent on a user terminal.

17. A machine-readable medium comprising instructions that when executed by a machine cause the machine to perform operations comprising:

receiving a statement from a user;

observing one or more physical contextual cues;

determining a user context based on the observed physical contextual cues;

processing the user statement and user context to generate a reply to the user; and

presenting the reply to the user on a user interface.

18. The medium of claim 17, wherein processing comprises comparing the user statement to the determined user context to determine if they are consistent and, if not, then presenting an inquiry to the user to explain the inconsistency.

19. The medium of claim 17, further comprising:

observing a user facial expression at a time of receiving a user statement;

associating the user facial expression with a user mood; and

presenting an inquiry to the user regarding the associated user mood.

20. An apparatus comprising:

a user input subsystem to receive a statement from a user and to observe user behavior;

a user input interpreter to determine a user context based on the behavior;

a conversation subsystem to process the user statement and user context to generate a reply to the user; and

a system output module to present the reply to the user on a user interface.

21. The apparatus of claim 20, further comprising

a cross modality module to combine information received from other user input from the user input subsystem with the statement and the observed user behavior and provide the combined information to the conversation subsystem.

22. The apparatus of claim 21, further comprising a system data summarizer to summarize user stored data about the user and provide a summary of the stored data to the cross modality module.