Automatic pattern recognition in conversations

• US 10,110,743 B2
• Filed: 11/20/2017
• Issued: 10/23/2018
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Zoominfo Converse LLC
• Original Assignee: AffectLayer, Inc. (ZoomInfo Technologies, Inc.)
• Inventors: Raanani, Roy, Levy, Russell, Facher, Dominik, Breakstone, Micha Yochanan
• Primary Examiner(s): Addy, Thjuan K
• Application Number:
  US15/817,490
• Publication Number:
  US 20180077286A1
• Time in Patent Office: 337 Days
• Field of Search: 37926507, 37926505, 37926506, 3792661, 37926503, 37926511, 37926512
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0281 Customer communication at a business location, e.g. providing product or service information, consulting
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 2203/357 Autocues for dialog assistance
• CPC Class Codes: H04M 2203/556 Statistical analysis and interpretation
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/5175 Call or contact centers supervision arrangements
• CPC Class Codes: H04M 3/5232 Call distribution algorithms

Abstract

A pattern recognition system (“system”) automatically determines conversation patterns that distinguish a first set of participants from a second set of participants. For example, a first set of participants can be top performing representatives and the second set of participants can be low performing representatives. The system analyzes a first set of recordings of the top performing representatives to extract a first set of features associated with the first set of recordings, and analyzes the first set of features to generate first pattern data that is indicative of a pattern of the conversation of the top performing representatives. Similarly, the system also generates second pattern data that is indicative of a pattern of the conversation of the low performing representatives. The system analyzes the first pattern data and the second pattern data to generate distinctive features that distinguish the first pattern from the second pattern.

Claims

1. A computer-implemented method, comprising:

receiving identification information regarding a first set of representatives;

retrieving a first set of recordings of conversations associated with the first set of representatives, wherein each conversation includes at least one of multiple customers and at least one representative from the first set of representatives;

retrieving a second set of recordings of conversations associated with a second set of representatives, wherein each conversation includes at least one of the multiple customers and at least one representative from the second set of representatives;

extracting a first set of features from the first set of recordings and a second set of features from the second set of recordings to generate multiple features, wherein the multiple features indicate characteristics of any of (a) a customer of multiple customers in the corresponding conversation, (b) a representative of multiple representatives in the corresponding conversation, (c) the corresponding conversation;

generating a first pattern data by analyzing the first set of features, the first pattern data indicative a pattern of the conversation of the first set of representatives;

generating a second pattern data by analyzing the second set of features, the second pattern data indicative of a pattern of the conversation of the second set of representatives; and

generating multiple distinctive features that are distinctive between the first pattern data and the second pattern data by analyzing the first set of features and the second set of features.

2. The computer-implemented method of claim 1, wherein

receiving the identification information of the first set of representatives includes:

determining those of the multiple representatives that satisfy a specified criterion as the first set of representatives.

3. The computer-implemented method of claim 2, wherein the specified criterion includes those of the multiple representatives having a performance indicator whose value satisfies the specified criterion.

4. The computer-implemented method of claim 2, wherein the second set of representatives includes those of the multiple representatives that do not satisfy the specified criterion.

5. The computer-implemented method of claim 1, wherein extracting the first set of features includes:

generating features that include a transcription, vocabulary and a language model of the conversations as a first output.

6. The computer-implemented method of claim 5, wherein extracting the first set of features includes:

generating, using the first output, features that include semantic information from the conversations.

7. The computer-implemented method of claim 1, wherein extracting the first set of features includes:

generating a set of low-level features that indicate information associated with a voice signal in the multiple recordings, and a set of high-level features that include personality traits and personal attributes of the multiple representatives and emotion data that indicates emotions of the multiple representatives.

8. The computer-implemented method of claim 1, wherein extracting the first set of features includes generating features that include data regarding conversation flow.

9. The computer-implemented method of claim 1, wherein extracting the first set of features includes generating features related a representative-customer pair in a conversation of the conversations.

10. The computer-implemented method of claim 1, wherein extracting the first set of features includes:

generating a speaker engagement metric that includes information regarding a degree of engagement of a specified customer of the multiple customers in a conversation of the multiple conversations.

11. The computer-implemented method of claim 1, wherein extracting the first set of features includes extracting a visual feature associated with a conversation of the conversations.

12. The computer-implemented method of claim 1, wherein generating the first pattern data includes generating the first pattern data based on a usage of vocabulary by the first set of representatives, the first pattern data including usage of a specified word or a phrase.

13. The computer-implemented method of claim 1, wherein generating the first pattern data includes generating the first pattern data based on a usage of a specified subset of vocabulary by the first set of representatives, the specified subset including one or more words or phrases that are determined by a weighting scheme.

14. The computer-implemented method of claim 13, wherein the weighting scheme is term frequency-inverse document frequency (TF-IDF).

15. The computer-implemented method of claim 1, wherein generating the first pattern data includes generating the first pattern data based on at least one of a frequency of setting an action item or a timing of setting the action in a conversation by the first set of representatives.

16. The computer-implemented method of claim 1, wherein generating the first pattern data includes generating the first pattern data based on a length of utterances by the first set of representatives.

17. The computer-implemented method of claim 1, wherein generating the first pattern data includes generating the first pattern data based on a talk-listen ratio of the first set of representatives.

18. The computer-implemented method of claim 1, wherein generating the distinctive features includes:

determining a difference between a first value associated with a specified feature in the first pattern data and a second value associated with the specified feature in the second pattern data, and

determining the specified feature as a distinctive feature of the distinctive features between the first pattern data and the second pattern data if the difference exceeds a specified threshold.

19. The computer-implemented method of claim 1, wherein generating the distinctive features includes determining those of the multiple features that occur in the first pattern data but not in the second pattern data.

20. The computer-implemented method of claim 1, wherein generating the distinctive features includes determining those of the multiple features that occur in the second pattern data but not in the first pattern data.

21. The computer-implemented method of claim 1, wherein extracting the first set of features includes extracting the multiple features using any of an artificial intelligence, a machine learning, or natural language processing technique.

22. The computer-implemented method of claim 1, wherein at least one of the first set of recordings includes a recording of a video call between one of the customers and one of the first set of representatives.

23. The computer-implemented method of claim 1, wherein at least one of the first set of recordings includes an online meeting between one of the customers and one of the first set of representatives.

24. The computer-implemented method of claim 1, wherein at least one of the multiple recordings includes a recording of a virtual reality-based conversation between one of the customers and one of the multiple representatives.

25. The computer-implemented method of claim 1, wherein at least one of the multiple recordings includes a recording of an augmented reality-based conversation between one of the customers and one of the multiple representatives.

26. The computer-implemented method of claim 1, wherein at least one of the multiple recordings includes an e-mail conversation between one of the customers and one of the multiple representatives.

27. A non-transitory computer-readable storage medium storing computer-readable instructions, comprising:

instructions for extracting a first set of features from a first set of recordings and a second set of features from a second set of recordings to generate multiple features, wherein the first set of recordings include conversations of a first set of representatives, wherein the second set of recordings include conversations of a second set of representatives, wherein the multiple features indicate characteristics of any of (a) a customer of multiple customers in the corresponding conversation, (b) a representative of multiple representatives in the corresponding conversation, (c) the corresponding conversation;

instructions for generating;

first pattern data by analyzing the first set of features, the first pattern data indicative a pattern of the conversation of the first set of representatives with a first set of customers, and

second pattern data by analyzing the second set of features, the second pattern data indicative of a pattern of the conversation of the second set of representatives with a second set of customers; and

instructions for determining a correlation of features between the first pattern data and the second pattern data, wherein the correlation is indicative of a difference between a specified feature of the multiple features in the first pattern data and the second pattern data.

28. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for extracting the first set of features includes instructions for extracting a visual feature associated with a conversation of the conversations.

29. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for generating the first pattern data include:

instructions for generating the first pattern data based on a usage of vocabulary by the first set of representatives, the first pattern data including usage of a specified word or a phrase.

30. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for generating the first pattern data include:

instructions for generating the first pattern data based on a usage of a specified subset of the vocabulary by the first set of representatives, the specified subset including one or more words or phrases that are determined by a weighting scheme.

31. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for generating the first pattern data include:

instructions for generating the first pattern data based on at least one of a frequency of setting an action item or a timing of setting the action in a conversation by the first set of representatives.

32. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for generating the first pattern data include:

instructions for generating the first pattern data based on a length of utterances by the first set of representatives.

33. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for generating the first pattern data include:

instructions for generating the first pattern data based on a talk-listen ratio of the first set of representatives.

34. The non-transitory computer-readable storage medium of claim 27, wherein the instructions for determining the correlation of features include:

instructions for determining the difference between a first value of the specified feature in the first pattern data and a second value of the specified feature in the second pattern data, and

instructions for determining the specified feature as a distinctive feature between the first pattern data and the second pattern data if the difference exceeds a specified threshold.

35. A system, comprising:

a first component that is configured to extract a first set of features from a first set of recordings and a second set of features from a second set of recordings to generate multiple features, wherein the first set of recordings include conversations of a first set of representatives, wherein the second set of recordings include conversations of a second set of representatives, wherein the multiple features indicate characteristics of any of (a) a customer of multiple customers in the corresponding conversation, (b) a representative of multiple representatives in the corresponding conversation, (c) the corresponding conversation;

a second component that is configured to generate;

first pattern data by analyzing the first set of features, the first pattern data indicative a pattern of the conversation of the first set of representatives with a first set of customers, and

second pattern data by analyzing the second set of features, the second pattern data indicative of a pattern of the conversation of the second set of representatives with a second set of customers; and

a third component that is configured to determine a correlation of features between the first pattern data and the second pattern data, wherein the correlation is indicative of a difference between a specified feature of the multiple features in the first pattern data and the second pattern data.