Human-to-human conversation analysis

• US 10,346,542 B2
• Filed: 02/27/2013
• Issued: 07/09/2019
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Original Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Inventors: Wooters, Charles C
• Primary Examiner(s): Leland, III, Edwin S
• Application Number:
  US13/778,824
• Publication Number:
  US 20140067375A1
• Time in Patent Office: 2,323 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

Customer support, and other types of activities in which there is a dialog between two humans can generate large volumes of conversation records. Automated analysis of these records can provide information about high-level features of, for example, the workings of a customer service department. Analysis of these conversations between a customer and a customer-support agent may also allow identification of customer support activities that can be provided by virtual agents instead of actual human agents. The analysis may evaluate conversations in terms of complexity, duration, and sentiment of the participants. Additionally, the conversations may also be analyzed to identify the existence of selected concepts or keywords. Workflow characteristics, the extent to which the conversation represents a multi-step process intended to accomplish a task, may also be determined for the conversations. Characteristics of individual conversations may be combined to obtain generalized or representative features for a set of a conversation records.

Claims

1. A method comprising:

accessing one or more log files of a corpus of human-to-human conversations;

performing textual analysis on the one or more log files, the textual analysis including identifying carriage returns, tabs, changes in font, and/or changes in text color;

based at least in part on the textual analysis, converting each of the one or more log files into a different file format that indicates dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

identifying, from the dialogue turns of a first conversation in the corpus, a particular dialog turn that includes a question from a particular individual;

representing the dialog turns of the particular individual as a series of indicia, positions in the series including either a first indicia indicating that a corresponding dialogue turn includes a question or a second indicia indicating that a corresponding dialog turn does not include a question;

comparing the series of indicia to a predefined sequence of indicia;

determining, based at least in part on comparing the series of indicia to the predefined sequence of indicia, that the first conversation has workflow-like features;

determining, using one or more processing units, a first metric and a second metric for the first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have the workflow-like features, the work-flow like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining, using the one or more processing units, a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in creating a virtual agent to replace a human associated with the corpus; and

displaying, via a graphical user interface, information that is based at least in part on the representative value.

2. The method of claim 1, wherein, the corpus comprises over 1,000 individual conversations.

3. The method of claim 1, wherein at least one of the first conversation or the second conversation includes a conversation between a customer who asks one or more questions and a human agent who answers the one or more questions.

4. The method of claim 3, further comprising providing a suggestion to replace the human agent with the virtual agent.

5. The method of claim 1, wherein at least one of the first conversation or the second conversation is stored in the corpus as a live text chat between two humans.

6. The method of claim 1, wherein at least one of the first conversation or the second conversation is stored in the corpus as a text record generated from speech-to-text analysis of spoken communication between two humans.

7. The method of claim 1, wherein the determining the first metric for the first conversation includes determining at least one of:

a number of separate questions contained in a dialogue turn of a customer for the first conversation;

a length of a communication in the dialogue turn of the customer for the first conversation;

a number of dialogue turns in the first conversation;

a number of dialogue turns of an agent for the first conversation that contain at least one question;

or

a semantic distance between dialogue turns for the first conversation.

8. The method of claim 1, wherein the determining the second metric for the first conversation includes determining at least one of:

a number and order of dialogue turns of an agent for the first conversation that contain at least one question;

or

a number and order of dialogue turns of the agent for the first conversation that do not include a question.

9. The method of claim 1, further comprising

providing, via one or more graphical user interfaces, a first metric indicating a first measure of the corpus of human-to-human conversations for which a human could be replaced with a virtual agent;

and

providing, via the one or more graphical user interfaces, a second metric indicating a second measure of the corpus of human-to-human conversations for which a human could not be replaced with a virtual agent.

10. One or more non-transitory computer-readable storage media, configured to store computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

obtaining a recording of a spoken communication between a first individual and a second individual;

performing speaker recognition on the recording to identify first audio characteristics of the first individual and second audio characteristics of the second individual;

identifying, based at least in part on the first audio characteristics, first dialogue turns representing first input from the first individual;

identifying, based at least in part on the second audio characteristics, second dialog turns representing second input from the second individual;

generating one or more log files including indications of the first dialog turns and the second dialog turns;

storing the one or more log files as a first conversation in a corpus of human-to-human conversations;

determining a first metric and a second metric for the first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human associated with the corpus with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

11. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the first metric for the first conversation includes determining a number of separate questions contained in a dialogue turn of a customer for the first conversation based on at least one of a number of question marks in the dialogue turn or natural language processing.

12. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the first metric for the first conversation includes determining a length of a dialogue turn of a customer for the first conversation based on at least one of a number of words of the dialogue turn, a number of characters in the dialogue turn, or a length of time of the dialogue turn.

13. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the first metric for the first conversation includes determining a number of dialogue turns in the first conversation.

14. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the first metric for the first conversation includes determining a number of dialogue turns of an agent for the first conversation that contain at least one question.

15. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the first metric for the first conversation includes determining a semantic distance between dialogue turns for the first conversation.

16. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the second metric for the first conversation includes determining a number and order of dialogue turns of an agent for the first conversation that contain at least one question.

17. The one or more non-transitory computer-readable storage media of claim 10, wherein the determining the second metric for the first conversation includes determining a number and order of dialogue turns of an agent for the first conversation that do not include a question.

18. A system comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising;

obtaining first data representing a first conversation, the first data being in a first data format;

obtaining second data representing a second conversation, the second data being in a second data format that is different than the first format;

converting the first data from the first data format to a third data format;

converting the second data from the second data format to the third data format, wherein the computer-executable instructions are configured to perform textual analysis on data in the third data format;

performing textual analysis on the first data of the first conversation and the second data of the second conversation;

based at least in part on the textual analysis, identifying dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining a first metric and a second metric for the first conversation, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for the second conversation, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human associated with the first conversation or the second conversation with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

19. The system of claim 18, wherein the determining the first metric for the first conversation is based at least in part on a number of separate questions contained in a dialogue turn of a participant for the first conversation.

20. The system of claim 18, wherein the determining the first metric for the first conversation is based at least in part on a length of a dialogue turn of a participant for the first conversation.

21. The system of claim 18, wherein the determining the first metric for the first conversation is based at least in part on a number of dialogue turns in the first conversation.

22. The system of claim 18, wherein the determining the first metric for the first conversation is based at least in part on a number of dialogue turns of a participant for the first conversation that contain at least one question.

23. The system of claim 18, wherein the determining the first metric for the first conversation is based at least in part on a semantic distance between dialogue turns for the first conversation.

24. The system of claim 18, wherein the determining the second metric for the first conversation is based at least in part on a number and order of dialogue turns of a participant for the first conversation that contain at least one question.

25. The system of claim 18, wherein the determining the second metric for the first conversation is based at least in part on a number and order of dialogue turns of a participant for the first conversation that do not include a question.

26. A system comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising;

performing textual analysis on data of a first conversation and data of a second conversation;

based at least in part on the textual analysis, identifying dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining a first metric and a second metric for the first conversation, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for the second conversation, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

27. The system of claim 26, wherein the determining the first metric for the first conversation is based on at least one of:

a number of separate questions contained in a dialogue turn for the first conversation;

a length of a communication in the dialogue turn for the first conversation;

a number of dialogue turns in the first conversation;

a number of dialogue turns for the first conversation that contain at least one question;

or

a semantic distance between dialogue turns for the first conversation.

28. The system of claim 26, wherein the determining the second metric for the first conversation includes determining at least one of:

a number and order of dialogue turns for the first conversation that contain at least one question;

or

a number and order of dialogue turns for the first conversation that do not include a question.

29. The system of claim 26, wherein the operations further comprise:

based at least in part on the representative value, providing a suggestion to replace the human with the virtual agent.

30. A method comprising:

accessing one or more log files of a corpus of human-to-human conversations;

performing textual analysis on the one or more log files, the textual analysis including identifying carriage returns, tabs, changes in font, and/or changes in text color;

based at least in part on the textual analysis, converting each of the one or more log files into a different file format that indicates dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining, using one or more processing units, a first metric and a second metric for a first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the work-flow like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining, using the one or more processing units, a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in creating a virtual agent to replace a human associated with the corpus; and

displaying, via a graphical user interface, information that is based at least in part on the representative value;

based at least in part on the representative value, providing a suggestion to replace the human with the virtual agent.