Modeling a user's emotion and personality in a computer user interface

• US 5,987,415 A
• Filed: 06/30/1998
• Issued: 11/16/1999
• Est. Priority Date: 03/23/1998
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Breese, John S., Ball, John Eugene
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): SAX, ROBERT L
• Application Number:
  US09/109,233
• Time in Patent Office: 504 Days
• Field of Search: 704/270, 395/10
• US Class Current: 704/270
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: H04N 21/466 Learning process for intelligent management, e.g. learning user preferences for recommending movies services using the results of monitoring in broadcast systems H04H60/61
• CPC Class Codes: H04N 21/4663 involving probabilistic networks, e.g. Bayesian networks

Abstract

The invention is embodied in a computer user interface including an observer capable of observing user behavior, an agent capable of conveying emotion and personality by exhibiting corresponding behavior to a user, and a network linking user behavior observed by said observer and emotion and personality conveyed by said agent. The network can include an observing network facilitating inferencing user emotional and personality states from the behavior observed by the observer as well as an agent network facilitating inferencing of agent behavior from emotion and personality states to be conveyed by the agent. In addition, a policy module can dictate to the agent network desired emotion and personality states to be conveyed by the agent based upon user emotion and personality states inferred by the observing network. Typically, each network is a stochastic model. Each stochastic model is preferably a Bayesian network, so that the observing network is a first Bayesian network while the agent network is a second Bayesian network. Generally, the first and second Bayesian networks are similar copies of one another. Each of the two Bayesian networks include a first layer of multi-state nodes representing respective emotional and personality variables, and a second layer of multi-state nodes representing respective behavioral variables. Each one of the nodes includes probabilities linking each state in the one node with states of others of the nodes. More specifically, each one of the nodes in the first layer includes probabilities linking the states of the one first layer node to the states of nodes in the second layer. Similarly, each one of the nodes in the second layer include probabilities linking the states of the one second layer node to states of nodes in the first layer.

Claims

1. A computer user interface capable of sensing user emotional and personality states and conveying them to a computer application for processing, comprising:

an observer capable of observing user behavior;

an observing network facilitating inferencing user emotional and personality states from said behavior observed by said observer; and

said observing network capable of conveying to said computer application the user emotional and personality states inferred by said observing network for processing by said application.

2. The user interface of claim 1 wherein said network comprises a stochastic model.

3. The user interface of claim 2 wherein said stochastic model comprises a Bayesian network.

4. The user interface of claim 3 wherein said Bayesian network comprises:

a first layer of multi-state nodes representing respective emotional and personality variables having outputs connected to said policy module; and

a second layer of multi-state nodes representing respective behavioral variables having inputs connected to said observer, wherein each one of said nodes comprises probabilities linking each state in the one node with states of others of said nodes.

5. The user interface of claim 4 wherein:

the probabilities in each one of said nodes in said first layer links the states of said one first layer node to the states of nodes in said second layer; and

the probabilities in each one of said nodes in said second layer link the states of said one second layer node to states of nodes in said first layer.

6. The user interface of claim 4 wherein said multi-state nodes representing emotional and personality variables comprise at least one of:

(a) a valence node, (b) an arousal node, (c) a friendliness node and (d) a dominance node.

7. The user interface of claim 4 wherein said multi-state nodes representing behavior variables comprise at least one of:

(a) a speech attribute node, (b) a facial expression node, (c) a word attribute node.

8. The user interface of claim 4 wherein said multi-state nodes representing behavioral variables comprise a set of word attribute nodes having probabilities relating states of said emotional and personality variables to a set of corresponding word attributes, said interface further comprising:

a language submodel coupled to said word attribute nodes; and

a word node coupled to said language submodel.

9. The user interface of claim 8 wherein said set of word attributes comprise at least plural ones of:

(a) terseness, (b) positiveness, (c) activeness, (d) strength and (e) formality.

10. The user interface of claim 9 wherein said language submodel comprises:

a word attribute layer relating individual word expressions to probabilities of conveying particular ones of said set of word attributes;

a scoring layer relating states of said word attribute layer for a particular word expression to states of corresponding ones of said set of word attribute nodes of said network.

11. The user interface of claim 10 wherein said language submodel further comprises a match module selecting a word expression having a winning score computed by said scoring layer.

12. A method of operating a computer user interface in support of a computer application, comprising:

providing a user model stochastic network;

inferring, in said user model stochastic network, user emotion and personality states from observed user behavioral states;

transmitting, to said application, said user emotional and personality states for processing by said application.

13. Computer-readable media storing instructions for carrying out the steps of claim 12.

14. The method of claim 12 wherein the step of providing a user model stochastic network comprises:

providing a first layer of multi-state nodes representing respective emotional and personality variables; and

providing a second layer of multi-state nodes representing respective behavioral variables.

15. The method of claim 14 wherein:

the step of providing said first layer of nodes comprises providing probabilities linking the states of said one first layer node to the states of nodes in said second layer; and

the step of providing said second layer of nodes comprises providing probabilities linking the states of said one second layer node to states of nodes in said first layer.

16. The method of claim 14 wherein the step of providing multi-state nodes representing emotional and personality variables comprise providing at least one of:

(a) a valence node, (b) an arousal node, (c) a friendliness node and (d) a dominance node.

17. The method of claim 16 wherein the step of providing multi-state nodes representing behavior variables comprises providing at least one of:

(a) a speech attribute node, (b) a facial expression node, (c) a word attribute node.

18. The method of claim 14 wherein the step of providing multi-state nodes representing behavioral variables comprises providing a set of word attribute nodes having probabilities relating states of said emotional and personality variables to a set of corresponding word attributes, said method further comprising:

providing a language submodel coupled to said word attribute nodes; and

providing a word node coupled to said language submodel.

19. The method of claim 18 wherein the step of providing a set of word attributes comprises providing at least plural ones of:

(a) terseness, (b) positiveness, (c) activeness, (d) strength and (e) formality.

20. The method of claim 18 wherein the step of providing said language submodel comprises:

providing a word attribute layer relating individual word expressions to probabilities of conveying particular ones of said set of word attributes; and

providing a scoring layer relating states of said word attribute layer for a particular word expression to states of corresponding ones of said set of word attribute nodes of said network.

21. The method of claim 20 wherein the step of providing said language submodel further comprises providing a match module capable of selecting a word expression having a winning score computed by said scoring layer.