Diarization using linguistic labeling

• US 10,134,401 B2
• Filed: 11/20/2013
• Issued: 11/20/2018
• Est. Priority Date: 11/21/2012
• Status: Active Grant

• Current Assignee: Verint Systems Incorporated
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Ziv, Omer, Achituv, Ran, Shapira, Ido, Dreyfuss, Jeremie
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/084,976
• Publication Number:
  US 20140142940A1
• Time in Patent Office: 1,826 Days
• Field of Search: 704235
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction

Abstract

Systems and methods of diarization using linguistic labeling include receiving a set of diarized textual transcripts. A least one heuristic is automatedly applied to the diarized textual transcripts to select transcripts likely to be associated with an identified group of speakers. The selected transcripts are analyzed to create at least one linguistic model. The linguistic model is applied to transcripted audio data to label a portion of the transcripted audio data as having been spoken by the identified group of speakers. Still further embodiments of diarization using linguistic labeling may serve to label agent speech and customer speech in a recorded and transcripted customer service interaction.

Claims

1. A method of diarization, the method comprising:

receiving a set of textual transcripts from a transcription server and a set of audio files associated with the set of textual transcripts from an audio database server;

performing a blind diarization on the set of textual transcripts and the set of audio files to segment and cluster the textual transcripts into a plurality of textual speaker clusters, wherein the number of textual speaker clusters is at least equal to a number of speakers in the textual transcript;

automatedly applying at least one heuristic to the textual speaker clusters with a processor to select textual speaker clusters likely to be associated with an identified group of speakers;

analyzing the selected textual speaker clusters with the processor to create at least one linguistic model;

applying the linguistic model to transcripted audio data with the processor to label a portion of the transcripted audio data as having been spoken by the identified group of speakers;

determining word use frequencies for words in the selected transcripts with the processor, wherein the word use frequencies are used to create the at least one linguistic model;

determining word use frequencies for words in the diarized portions of non-selected transcripts with the processor; and

comparing the word use frequencies for words in the selected transcripts to the word use frequencies for words in the non-selected transcripts with the processor to identify a plurality of discriminating words for use in the at least one linguistic model,

wherein the diarized textual transcripts are associated in groups of at least two, wherein the group of at least two includes a textual transcript originating from the identified group of speakers and at least one textual transcript originating from an other speaker, and wherein the non-selected transcripts are assumed to have originated from an other speaker,

further wherein the at least one heuristic is a detection of a script associated with the identified group of speakers,

further wherein a plurality of scripts associated with the identified group of speakers is compared to each of the diarized transcripts and a correlation score between each of the diarized transcripts and the plurality of scripts is calculated and further wherein the diarized transcript in each group with the greatest correlation score is selected as being the transcript likely to be associated with the identified group of speakers; and

saving the at least one linguistic model to a linguistic database server and associating it with the labeled speaker;

with the processor, applying the saved at least one linguistic model from the linguistic database server to a new audio file transcript from an audio source to perform diarization of the new audio file by blind diarizing the new audio file, comparing each new textual speaker cluster to the at least one linguistic model, and labeling each textual speaker cluster as belonging to a customer service agent or belonging to a customer.

2. The method of claim 1, wherein the identified group of speakers are customer service agents and the audio data is audio data of a customer service interaction between at least one customer service agent and at least one customer.

3. The method of claim 1, further comprising:

receiving a set of recorded audio data; and

transcribing the set of recorded audio data to produce the set of textual transcripts.

4. The method of claim 1, wherein the at least one heuristic is detection of a script associated with the identified group of speakers.

5. The method of claim 1, further comprising:

calculating a difference between the word use frequencies for each word in the selected transcripts and the non-selected transcripts; and

comparing the difference to a predetermined selection threshold, wherein if the difference is greater than the predetermined selection threshold, the word is identified as a discriminating word.

6. The method of claim 1, wherein the audio data is streaming audio data.

7. The method of claim 1, the linguistic model further comprises at least one script and at least one word cloud.

8. A method of diarization of audio data from a customer service interaction between at least an agent and a customer, the method comprising:

receiving a set of diarized textual transcripts of customer service interactions between at least an agent and a customer from a transcription server, wherein the diarized textual transcripts are grouped in pluralities comprising at least a transcript associated to the agent and a transcript associated to the customer, wherein the transcript associated to the agent and the transcript associated to the customer are from a singular customer service interaction;

automatedly applying at least one heuristic to the diarized textual transcripts with a processor to select at least one of the transcripts in each plurality as being associated to the agent;

analyzing the selected transcripts with the processor to create at least one linguistic model;

utilizing the linguistic model to further analyze the transcripts by selecting a set, determining word use frequencies for words in the selected set of diarized transcripts, determining word use frequencies for words in the non-selected diarized transcripts, and using the determined word use frequencies in creating the at least one linguistic model;

saving the at least one linguistic model to a linguistic database server; and

applying the linguistic model to new transcripted audio data with the processor to label a portion of the transcripted audio data as having been spoken by the agent, where in the new transcripted audio data is not diarized and a known speaker has not yet been associated with the new transcripted audio data.

9. The method of claim 8, wherein the at least one heuristic is detection of at least one script associated with an agent in the diarized textual transcripts.

10. The method of claim 8, wherein the at least one linguistic model is an agent linguistic model and a customer linguistic model and the determined word use frequencies in the selected diarized transcripts are used to create the agent linguistic model and the determined word use frequencies of the non-selected diarized transcripts are used to create the customer linguistic model.

11. The method of claim 8, further comprising:

calculating a difference between the word use frequencies in the selected diarized transcripts to the word use frequencies in the non-selected diarized transcripts;

comparing the differences to at least one predetermined selection threshold; and

including a word use frequency in the linguistic model if the difference exceeds the at least one predetermined selection threshold.

12. The method of claim 9, further comprising:

comparing a plurality of scripts associated with the agent to each of the diarized transcripts; and

calculating a correlation score for each diarized transcript based upon the comparison;

wherein the diarized transcript in each group with the greatest correlation score is selected as being the transcript associated to the agent.

13. A system for diarization and labeling of audio data, the system comprising:

a database server comprising a plurality of audio files;

a transcription server that transcribes and diarizes the audio files of the plurality of audio files into a plurality of groups comprising at least two diarized textual transcripts;

a processor that automatedly applies at least one heuristic to the diarized textual transcripts to select at least one of the transcripts in each group as being associated to an identified group of speakers and analyze the selected transcripts to create at least one linguistic model indicative of the identified group of speakers by determining word use frequencies for words in the selected diarized transcripts, determining word use frequencies for words in the non-selected diarized transcripts, and using the determined word use frequencies in creating the at least one linguistic model;

a linguistic database server that stores the at least one linguistic model; and

an audio source that provides new transcripted audio data to the processor;

wherein the processor applies the linguistic model to new transcripted audio data to label a portion of the transcripted audio data as being associated with the identified group of speakers, wherein the new transcripted audio data has not been diarized and has not been associated with a group of speakers,

further wherein the audio data is of a customer service interaction between at least one customer service agent and at least one customer and the at least one heuristic is an agent script wherein the identified group of speakers are customer service agents, and

further wherein the diarized textual transcripts are grouped in pluralities comprising at least a transcript associated to the agent and a transcript associated to the customer, wherein the transcript associated to the agent and the transcript associated to the customer are from a singular customer service interaction.