Method and apparatus for interaction or discourse analytics

• US 8,676,586 B2
• Filed: 09/16/2008
• Issued: 03/18/2014
• Est. Priority Date: 09/16/2008
• Status: Active Grant

• Current Assignee: Nice Ltd
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Wasserblat, Moshe, Pereg, Oren, Lubowich, Yuval
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US12/211,112
• Publication Number:
  US 20100070276A1
• Time in Patent Office: 2,009 Days
• Field of Search: None
• US Class Current: 704/270
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04M 3/5232 Call distribution algorithms

Abstract

A method and apparatus for analyzing and segmenting a vocal interaction captured in a test audio source, the test audio source captured within an environment. The method and apparatus first use text and acoustic features extracted from the interaction with tagging information, for constructing a model. Then, at production time, text and acoustic features are extracted from the interactions, and by applying the model, tagging information is retrieved for the interaction, enabling analysis, flow visualization or further processing of the interaction.

Claims

1. A method for analyzing a vocal interaction captured in an audio source, the method comprising:

a first receiving step for receiving a test audio source from a device by a computing platform via an input port, the test audio source comprising an interaction between an agent and a customer of the organization;

a text extraction step for extracting an at least one test word from the test audio source;

an acoustic feature extraction step for extracting at least one acoustic feature from the test audio source;

combining the at least one test word and the at least one acoustic feature in a feature vector; and

an interaction classification step based on applying a model that was previously trained on a training audio corpus, the model incorporating an at least one textual feature and an at least one acoustic feature that are constructed in a training feature vector combining the at least one textual feature and the at least one acoustic feature, and tagged data that indicate how to section the interaction to the at least one test word and the at least one acoustic feature extracted from the test audio source,

and determining a segment in the test audio source that is further classified to a class, based on the feature vector and the training feature vector, thereby obtaining a score that reflects the probability that the feature vector belongs to the class.

2. The method of claim 1 further comprising a step of performing agent quality management using the segment.

3. The method of claim 1 further comprising a step of using the segment for at least one item selected from the group consisting of:

improving text extraction;

performing statistical analysis;

visualization;

category assignment;

parts of speech extraction enhancement;

interaction analysis; and

new search descriptors identification.

4. The method of claim 1 further comprising a step of using the segment for at least one item selected from the group consisting of:

interaction analysis;

visualization;

statistics;

natural language processing;

categorization; and

classification.

5. The method of claim 1 wherein the segment is selected from the group consisting of:

a question;

an answer;

a statement;

a non-verbal segment; and

a street-noise segment.

6. The method of claim 1 further comprising a natural language processing step for processing the at least one test word.

7. The method of claim 6 wherein the natural language processing step comprises part of speech analysis for tagging the at least one test word into a part of speech, or a stemming step for stemming the at least one test word.

8. The method of claim 1 further comprising a textual feature extraction step for extracting a textual feature from the at least one test word.

9. The method of claim 1 further comprising a speaker dizarization step for segmenting the test audio source according to speaker.

10. The method of claim 1 wherein at least one result obtained by the interaction classification step is used by at least one item selected from the group consisting of:

interaction flow visualization;

category assignment;

text extraction improvement;

new search descriptors identification;

part of speech extraction enhancement step; and

statistics or diagrams steps, wherein at least one or both of text extraction improvement or part of speech extraction enhancement step comprise updating settings or parameters related to speech recognition.

11. The method of claim 1 further comprising the steps of:

receiving a training interaction corpus comprising at least one training audio source;

a second text extraction step for extracting an at least one training word from the at least one training audio source;

an acoustic feature extraction step for extracting at least one acoustic feature from the training audio source;

receiving tagging information related to an incomplete part of the at least one training audio source wherein the tagging information is based on analyzed definition and context of the at least one training word;

combining the at least one training word and the at least one acoustic feature in a construct of a feature vector; and

a model training step for constructing the model, based on the feature vector of the at least one training word and the at least one acoustic feature, and on the tagging information.

12. The method of claim 11 further comprising a second natural language processing step for processing the at least one training word, a second textual feature extraction step for extracting a textual feature from the at least one training word, or a second speaker dizarization step for segmenting the at least one training audio source according to speaker.

13. The method of claim 1 wherein the text extraction step comprises speech to text, word spotting, or phonetic search.

14. The method of claim 1 wherein the at least one acoustic feature is selected from the group consisting of:

pitch;

energy;

shimmer;

silence;

talk-over;

emotion level;

speech speed;

speaker gender; and

speaker age.

15. An apparatus for analyzing a vocal interaction between an agent and a customer of an organization captured in a test audio source, the test audio source electronically captured within an environment as a test audio electronic signal, the apparatus comprising:

an extraction engine for electronically extracting at least one test textual word and at least one test acoustic feature from the test audio electronic signal and combining the least one test textual word and the at least one test acoustic feature in a construct of a test feature vector,

or for electronically extracting at least one training textual word and at least one training acoustic feature from a training audio electronic signal and combining the at least one training textual word and the at least one training acoustic feature in a construct of a training feature vector;

a classification training engine for receiving tagging information respective to any part-of-speech (POS) in the training audio electronic signal independently of a sentence boundary that indicate how to section the vocal interaction and generating a model, the model relating between the at least one training textual word and the at least one training acoustic feature according to the training feature vector and the tagging information; and

a classification engine for applying the model having the tagging information that indicate how to section the vocal interaction to the at least one test textual word and the at least one test acoustic feature according to the test feature vector, thus obtaining a segment in the test audio electronic signal, and for further classifying the segment to a class, based on the test feature vector and the training feature vector, to obtain a score that reflects the probability that the test feature vector belongs to the class.

16. The apparatus of claim 15 wherein the extraction engine comprises a speech to text engine, a word spotting engine, or a phonetic search engine.

17. The apparatus of claim 15 further comprising a natural language processing engine for processing the at least one test textual word or the at least one training textual word.

18. The apparatus of claim 17 wherein the natural language processing engine comprises a part of speech analysis engine for tagging the at least one test textual word or the at least one training textual word into a part of speech, or a stemming engine for stemming the at least one test textual word or the at least one training textual word.

19. The apparatus of claim 15 wherein the at least one test acoustic feature or the at least one training acoustic feature is selected from the group consisting of:

pitch;

energy;

shimmer;

silence;

talk-over;

emotion level;

speech speed;

speaker gender; and

speaker age.

20. The apparatus of claim 15 further comprising a textual feature definition component for defining an at least one textual feature, and a textual feature extraction component for electronically extracting an at least one textual feature from the test audio electronic signal or training audio electronic signal.

21. The apparatus of claim 15 further comprising a capturing or logging component for capturing the audio electronic signal, and a storage component for storing the audio electronic signal or the at least one test textual word or the at least one training word or the model.

22. The apparatus of claim 15 further comprising a quality management component for evaluating a quality parameter of the agent or an activity associated with the organization.

23. The apparatus of claim 15 further comprising an at least one component selected from the group consisting of:

an interaction analysis component;

a visualization component;

a statistics component;

a natural language processing component;

a categorization component; and

a classification component.

24. An apparatus for analyzing a vocal interaction captured in a test audio source, the test audio source electronically captured within an environment as a test audio electronic signal, the apparatus comprising:

a text extraction engine for electronically extracting at least one textual test word from the test audio electronic signal or at least one training textual word from a training audio electronic signal;

an acoustic feature extraction engine for electronically extracting at least one test acoustic feature from the test audio electronic signal or at least one training acoustic feature from the training audio electronic signal;

a feature vector combining component for combining the at least one textual test word and the at least one test acoustic feature in a construct of a test feature vector, or for combining the at least one training textual word and the at least one training acoustic feature in a construct of a training feature vector;

a classification training engine for receiving tagging information respective to any part-of-speech (POS) in the training audio electronic signal independently of a sentence boundary that indicate how to section the vocal interaction and generating a model, the model relating between the at least one training textual word and the at least one training acoustic feature according to the training feature vector and the tagging information;

a classification engine for applying the model having the tagging information that indicate how to section the vocal interaction to the at least one test textual word and the at least one test acoustic feature according to the test feature vector, thus obtaining a segment in the test audio electronic signal, and for further classifying the segment to a class, based on the test feature vector and the training feature vector, to obtain a score that reflects the probability that the test feature vector belongs to the class;

a capturing or logging component for capturing the test audio or the training audio electronic signal;

a storage component for storing the audio electronic signal or the at least one test textual word or the at least one training textual word or the model; and

a quality management component for evaluating a quality parameter of the agent or an activity associated with the organization.

25. A computer readable storage medium containing a set of instructions for a general purpose computer, the set of instructions comprising:

receiving a test audio source having a vocal interaction therein;

extracting an at least one test textual word from the test audio source;

extracting at least one test acoustic feature from the test audio source; and

combining the at least one test word and the at least one acoustic feature in a feature vector;

applying a model, the model trained on a training audio corpus incorporating an at least one textual entity and an at least one acoustic feature that are constructed in a training feature vector combining the at least one textual feature and the at least one acoustic feature, and tagged data indicating sectioning of the interaction to the at least one test textual word and the at least one test acoustic feature extracted from the test audio source, and determining a segment in the test audio source and further classifying the test audio source to a class, based on the feature vector and the training feature vector, thereby obtaining a score that reflects the probability that the feature vector belongs to the class.