Human-to-human conversation analysis

• US 10,515,156 B2
• Filed: 07/08/2019
• Issued: 12/24/2019
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Original Assignee: Verint Americas Incorporated (Verint Systems Incorporated)
• Inventors: Wooters, Charles C.
• Primary Examiner(s): Leland, III, Edwin S
• Application Number:
  US16/504,773
• Publication Number:
  US 20190332676A1
• Time in Patent Office: 169 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

Customer support, and other types of activities in which there is a dialogue between two humans can generate large volumes of conversation records. Automated analysis of these records can provide information about high-level features of, for example, the workings of a customer service department. Analysis of these conversations between a customer and a customer-support agent may also allow identification of customer support activities that can be provided by virtual agents instead of actual human agents. The analysis may evaluate conversations in terms of complexity, duration, and sentiment of the participants. Additionally, the conversations may also be analyzed to identify the existence of selected concepts or keywords. Workflow characteristics, the extent to which the conversation represents a multi-step process intended to accomplish a task, may also be determined for the conversations. Characteristics of individual conversations may be combined to obtain generalized or representative features for a set of a conversation records.

Claims

1. Non-transitory computer-readable storage media, configured to store computer-executable instructions that, when executed on one or more processing units, cause the processing units to perform acts comprising:

obtaining a record of a human-to-human conversation between a customer and an agent;

replacing personally identifiable information and financial information with placeholder data that corresponds to the type of information replaced;

identifying, from the record, a customer dialogue turn containing a communication from the customer and an agent dialogue turn containing a communication from the agent;

identifying a beginning time and an ending time of the conversation;

calculating a duration of the conversation based on the beginning time and the ending time;

counting, for multiple different types of placeholder data, a number of instances that one or more of the multiple different types of placeholder data was added to the record;

determining a complexity scores for the conversation;

determining a workflow score for the conversation based at least in part on content in the agent dialogue turn, wherein the workflow score for a conversation indicates a probability that a series of dialogue turns in the conversation are communications in which the agent engages in multiple dialogue turns in order to accomplish a task for the customer; and

preparing data representing the duration, the number of instances that one or more of the multiple different types of place holder data was added to the record, the complexity score, and the workflow score of the conversation for display on a graphical user interface (GUI).

2. The non-transitory computer-readable storage media of claim 1, wherein the complexity score is based at least in part on one or more of (i) a number of separate questions contained in a first dialogue turn of the customer, (ii) a length of a communication in the first dialogue turn of the customer, (iii) a number of dialogue turns in the conversation, or (iv) a number of dialogue turns of the agent that contain at least one question.

3. The non-transitory computer-readable storage media of claim 1, wherein the workflow score is based at least in part on a number and order of dialogue turns of the agent that contain at least one question and a number and order of dialogue turns of the agent that do not include a question.

4. The non-transitory computer-readable storage media of claim 1, further comprising calculating a sentiment score for the conversation representing an inference about a degree of positive or negative feelings the customer has for the conversation.

5. The non-transitory computer-readable storage media of claim 4, further comprising calculating sentiment scores for two points in the conversation and determining a change in sentiment between the two points in the conversation.

6. A method comprising:

identifying a first participant in a human-to-human conversation that is seeking information or assistance from a second participant in the human-to-human conversation;

identifying a series of dialogue turns in the human-to-human conversation representing portions of the conversation generated by either the first participant or the second participant, at least a portion of the dialogue turns alternating between the first participant and the second participant;

determining which of the dialogue turns of the second participant include at least one question;

representing the dialogue turns of the second participant as a series of indicia, positions in the series including either a first indicia indicating that a corresponding dialogue turn includes at least one question or a second indicia indicating that a corresponding dialogue turn does not include a question;

comparing an ordering of the first indicia and the second indicia in the series of indicia to one or more predefined sequences of indicia; and

when at least one of the one or more predefined sequences of indicia matches at least a portion of the series of indicia, characterizing the human-to-human conversation as representing a workflow.

7. The method of claim 6, wherein determining which of the dialogue turns include at least one question comprises identifying dialogue turns that include one or more question marks.

8. The method of claim 6, wherein at least one of the dialogue turns of the second participant comprises a pre-written response.

9. The method of claim 6, wherein the predefined sequence of indicia is shorter than the series of indicia and the predefined sequence of indicia is compared to multiple portions of the series of indicia.

10. The method of claim 6, wherein a workflow is an algorithmic series of communications between the first participant and the second participant to accomplish a task.

11. The method of claim 6, further comprising calculating a workflow score based at least in part on a number of the one or more predefined sequences of indicia that match at least a portion of the series of indicia.

12. A method for calculating a complexity score for a human-to-human conversation, the method comprising:

identifying a first participant in the human-to-human conversation that is seeking information or assistance from a second participant in the human-to-human conversation;

identifying a series of dialogue turns in the human-to-human conversation, the dialog turns representing portions of the conversation generated by either the first participant or the second participant, at least a portion of the dialogue turns alternating between the first participant and the second participant;

measuring a length of a first dialogue turn of the first participant;

determining a number of separate questions included in the first dialogue turn of the first participant;

determining a number of dialogue turns of the second participant that include one or more questions; and

calculating, using one or more processing units, the complexity score based at least in part on the length of the first dialogue turn of the first participant, the number of separate questions included in the first dialogue turn of the first participant, and the number of dialogue turns of the second participant that include one or more questions.

13. The method of claim 12, wherein the length of the first dialogue turn of the first participant is measured by at least one of a number of words, a number of characters, or a length of time.

14. The method of claim 12, wherein the number of separate questions included in the first dialogue turn of the first participant is determined by at least one of counting a number of question marks or by natural language processing.

15. The method of claim 12, wherein determining the number of dialogue turns of the second participant that include one or more questions comprises identifying dialogue turns that include at least one question mark.

16. The method of claim 12, further comprising determining a total number of dialogue turns in the conversation, and wherein calculating the complexity score is based at least in part on the total number of dialogue turns in the conversation.

17. The method of claim 12, further comprising determining a linguistic content score of the of dialogue turns of the first participant, the linguistic content score based at least in part on grammatical accuracy, spelling accuracy, or readability, and wherein calculating the complexity score is based at least in part on the linguistic content score.

18. The method of claim 12, further comprising determining a semantic distance between dialogue turns of the first participant, and wherein calculating the complexity score is based at least in part on the semantic distance.