System and method for providing interaction between a user and an embodied conversational agent

• US 10,346,749 B2
• Filed: 02/17/2016
• Issued: 07/09/2019
• Est. Priority Date: 12/21/2015
• Status: Active Grant

• Current Assignee: Palo Alto Networks Incorporated
• Original Assignee: Wipro Limited
• Inventors: Kumar, Amit, Raj, Sheeba Santhosh, Ramanna, Ramprasad Kanakatte, Mannopantar, Raghottam
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US15/045,587
• Publication Number:
  US 20170178005A1
• Time in Patent Office: 1,238 Days
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: H04M 2203/1025 of avatars
• CPC Class Codes: H04M 3/5133 Operator terminal details
• CPC Class Codes: H04M 3/5166 in combination with interactive voice response systems or voice portals, e.g. as front-ends

Abstract

This disclosure relates generally to human-machine interaction. In one embodiment, an interaction device for providing the interaction between the user and the ECA is disclosed. The interaction device comprises a processor and a memory communicatively coupled to the processor. The memory stores processor instructions, which, on execution, causes the processor to receive conversation data of a user interacting with the ECA, wherein the ECA is presented on an interface of the interaction device. The processor further determines an emotional state of the user based on one or more behavioral parameters associated with the conversation data of the user. The processor identifies a response state for the ECA corresponding to the emotional state of the user, wherein the response state is identified from a plurality of response states based on a pre-defined probability for each response state. The processor further transitions behavior of the ECA based on the response state.

Claims

1. A method for providing an interaction between a user and an embodied conversational agent (ECA), the method comprising:

receiving, by an interaction device, conversation data of a user interacting with the ECA, wherein the ECA is presented on an interface of the interaction device;

computing, by the interaction device, a user behavior score based on a weightage assigned to each behavioral parameter of one or more behavioral parameters associated with the conversation data of the user;

determining, by the interaction device, an emotional state of the user based on the user behavior score;

identifying, by the interaction device, a response state for the ECA corresponding to the emotional state of the user, wherein the response state is identified from a plurality of response states based on a pre-defined probability for each response state; and

transitioning, by the interaction device, behavior of the ECA based on the response state.

2. The method of claim 1, wherein the one or more behavioral parameters comprises facial expression of the user, conversation sentiment, audio sentiment, and historical user behavior data.

3. The method of claim 1, wherein determining the emotional state of the user further comprises:

comparing the user behavior score with a pre-defined user behavior score.

4. The method of claim 1, wherein identifying the response state for the ECA corresponding to the emotional state of the user comprises:

determining the plurality of response states corresponding to the emotional state based on a pre-defined response matrix, wherein each response state of the plurality of response states is assigned the pre-defined probability; and

identifying the response state from the plurality of response states by performing a Monte Carlo sampling on pre-defined probabilities of the plurality of response states.

5. The method of claim 1, wherein transitioning the behavior of the ECA comprises transitioning visual appearance of the ECA.

6. The method of claim 5, wherein transitioning the visual appearance of the ECA comprises modifying at least one of facial expression of the ECA and one or more gestures of the ECA.

7. The method of claim 5, wherein transitioning the visual appearance of the ECA comprises:

determining an appearance transition sequence for the response state by mapping the response state with a pre-defined appearance transition matrix, wherein the appearance transition sequence comprises one or more appearance transitions; and

applying the appearance transition sequence on the ECA to transition the behavior of the ECA.

8. The method of claim 1, wherein transitioning the behavior of the ECA comprises modulating voice of the ECA by modifying at least one of pitch, volume of speech, speed of the speech, tone of the speech, and pause between words or statements in the speech.

9. An interaction device comprising:

a processor; and

a memory communicatively coupled to the processor, wherein the memory stores processor instructions, which, on execution, causes the processor to;

receive conversation data of a user interacting with the ECA, wherein the ECA is presented on an interface of the interaction device;

compute a user behavior score based on a weightage assigned to each behavioral parameter of one or more behavioral parameters associated with the conversation data of the user;

determine an emotional state of the user based on the user behavior score;

identify a response state for the ECA corresponding to the emotional state of the user, wherein the response state is identified from a plurality of response states based on a pre-defined probability for each response state; and

transition behavior of the ECA based on the response state.

10. The interaction device of claim 9, wherein the one or more behavioral parameters comprises facial expression of the user, conversation sentiment, audio sentiment, and historical user behavior data.

11. The interaction device of claim 9, wherein the processor is configured to:

determine the emotional state of the user by comparing the user behavior score with a pre-defined user behavior score.

12. The interaction device of claim 9, wherein the processor is configured to:

determine the plurality of response states corresponding to the emotional state based on a pre-defined response matrix, wherein each response state of the plurality of response states is assigned the pre-defined probability; and

identify the response state from the plurality of response states by performing a Monte Carlo sampling on pre-defined probabilities of the plurality of response states.

13. The interaction device of claim 9, wherein the processor is configured to transition the behavior of the ECA by transitioning visual appearance of the ECA.

14. The interaction device of claim 13, wherein the processor is configured to transition the visual appearance of the ECA by modifying at least one of facial expression of the ECA and one or more gestures of the ECA.

15. The interaction device of claim 14, wherein the processor is configured to:

determine an appearance transition sequence for the response state by mapping the response state with a pre-defined appearance transition matrix, wherein the appearance transition sequence comprises one or more appearance transitions; and

apply the appearance transition sequence on the ECA to transition the behavior of the ECA.

16. The interaction device of claim 9, wherein transitioning the behavior of the ECA comprises modulating voice of the ECA by modifying at least one of pitch, volume of speech, speed of the speech, tone of the speech, and pause between words or statements in the speech.

17. A non-transitory computer-readable medium storing computer-executable instructions for:

receiving conversation data of a user interacting with the ECA, wherein the ECA is presented on an interface of the interaction device;

computing a user behavior score based on a weightage assigned to each behavioral parameter of one or more behavioral parameters associated with the conversation data of the user;

determining an emotional state of the user based on the user behavior score;

identifying a response state for the ECA corresponding to the emotional state of the user, wherein the response state is identified from a plurality of response states based on a pre-defined probability for each response state; and

transitioning behavior of the ECA based on the response state.

18. The non-transitory computer-readable medium of claim 17, wherein the one or more behavioral parameters comprises facial expression of the user, conversation sentiment, audio sentiment, and historical user behavior data.

19. The non-transitory computer-readable medium of claim 17, wherein the instructions for determining the emotional state of the user further comprise:

comparing the user behavior score with a pre-defined user behavior score.

20. The non-transitory computer-readable medium of claim 17, wherein the instructions for identifying the response state for the ECA corresponding to the emotional state of the user comprises:

determining the plurality of response states corresponding to the emotional state based on a pre-defined response matrix, wherein each response state of the plurality of response states is assigned the pre-defined probability; and

identifying the response state from the plurality of response states by performing a Monte Carlo sampling on pre-defined probabilities of the plurality of response states.