Script compliance in spoken documents based on number of words between key terms

• US 9,947,320 B2
• Filed: 11/12/2014
• Issued: 04/17/2018
• Est. Priority Date: 11/12/2014
• Status: Active Grant

• Current Assignee: Nice Ltd
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Lembersky, Gennadi, Nissan, Maor, Bretter, Ronny
• Primary Examiner(s): Kazeminezhad, Farzad
• Application Number:
  US14/538,833
• Publication Number:
  US 20160133256A1
• Time in Patent Office: 1,252 Days
• Field of Search: 704270, 704257, 704235
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/5175 Call or contact centers supervision arrangements

Abstract

A method, computerized apparatus and computer program product for determining script compliance in interactions, the method comprising: receiving one or more indexed audio interaction; receiving a text representing a script; automatically extracting two or more key terms from the script; automatically generating a query representing the script, comprising: receiving one or more constraint associated with the at least two key terms; and determining spotted key terms of the key terms that appear in the indexed audio interactions; determining complied constraints based on a number of words difference between two key terms of the at least two key terms; and determining a relevance score for each of the indexed audio interactions, based on the spotted key terms and the complied constraints.

Claims

1. A computerized method for determining compliance of audio interactions with a script, the method comprising:

using a processor, receiving at least one audio interaction;

decoding speech within the at least one audio interaction;

indexing the decoded speech so as to obtain at least one indexed audio interaction;

receiving a text representing a script;

automatically extracting at least two key terms from the script, comprising;

discarding irrelevant terms;

determining a recognizability score for a word in the script based on a recognizability of a phoneme combination in the word;

determining a uniqueness score for the word in the script;

combining the recognizability score and the uniqueness score into a word score; and

determining that the word is a key term, subject to the word score exceeding a threshold;

automatically generating a query representing the script, comprising;

receiving at least one constraint related to a number of words difference between two key terms of the at least two key terms;

determining spotted key terms of die at least two key terms that appear in the at least one indexed audio interaction;

determining at least one complied constraint from the at least one received constraint; and

determining a relevance score for the at least one indexed audio interaction, based on the spotted key terms and the at least one complied constraint; and

using the relevance score to determine compliance of the at least one audio interaction with the script.

2. The method of claim 1, wherein the at least one indexed audio interaction comprises a multiplicity of indexed audio interactions, and further comprising filtering indexed audio interactions based on the relevance score.

3. The method of claim 2 further comprising:

sorting the multiplicity of indexed audio interactions based on the relevance score, and filtering from the multiplicity of indexed audio interactions those interactions having the highest or lowest relevance scores.

4. The method of claim 1, wherein the query comprises a weight associated with at least one of the at least two key terms.

5. The method of claim 4, wherein determining the relevance score comprises:

determining at least one group of key terms, wherein a key term in the at least one group appears in the at least one interaction, and the key term is connected to another key term in the at least one group if the key term and the other key term appear in one constraint complied with by the interaction;

determining a score for the at least one group as a function of weights associated with key terms that belong to the at least one group; and

determining the relevance score as a maximal relevance score associated with any group of the at least one group.

6. The method of claim 1, wherein the at least one constraint refers to order and distance between the at least two key terms, wherein the distance relates to time difference between the at least two key terms.

7. The method of claim 1, wherein the uniqueness score is determined using a Term Frequency-Inverse Document Frequency (TF-IDF) algorithm.

8. The method of claim 1, wherein

determining that the word is a key term comprises determining that the recognizability score exceeds a first threshold, or the uniqueness score exceeds a second threshold.

9. The method of claim 1, wherein combining the recognizability score and the uniqueness score into a word score is determined as the algebraic average or geometric average of the scores.

10. The method according to claim 1, further comprising sending a list of the at least one audio interactions that do not comply with the script to a user interface.

11. A computerized apparatus having a processor, the processor being adapted to perform steps comprising:

receiving at least one audio interaction;

decoding speech within the at least one audio interaction;

indexing the decoded speech so as to obtain at least one indexed audio interaction;

receiving a text representing a script;

automatically extracting at least two key terms from the script, comprising;

discarding irrelevant terms;

determining a recognizability score for a word in the script based on a recognizability of a phoneme combination in the word;

determining a uniqueness score for the word in the script;

combining the recognizability score and the uniqueness score into a word score; and

determining that the word is a key term, subject to the word score exceeding a threshold;

automatically generating a query representing the script, comprising;

receiving at least one constraint related to a number of words difference between two key terms of the at least two key terms;

determining spotted key terms of the at least two key terms that appear in the at least one indexed audio interaction;

determining at least one complied constraint from the at least one received constraint; and

determining a relevance score for the at least one indexed audio interaction, based on the spotted key terms and the at least one complied constraint; and

using the relevance score to determine compliance of the at least one audio interaction with the script.

12. The computerized apparatus of claim 11, wherein the at least one indexed audio interaction comprises a multiplicity of indexed audio interactions, and further comprising filtering indexed audio interactions based on the relevance score.

13. The computerized apparatus of claim 12, wherein the processor is further adapted to perform the steps of:

sorting the multiplicity of indexed audio interactions based on the relevance score, and

filtering from the multiplicity of indexed audio interactions those interactions having the highest or lowest relevance scores.

14. The computerized apparatus of claim 11, wherein the at least one constraint refers to order and distance between the at least two key terms, wherein the distance relates to time difference between the at least two key terms.

15. The computerized apparatus of claim 11, the relevance score is determined by:

determining at least one group of key terms, wherein a key term in the at least one group appears in the at least one interaction, and the key term is connected to another key term in the at least one group if the key term and the other key term appear in one constraint complied with by the interaction;

determining a score for the at least one group as a function of weights associated with key terms that belong to the at least one group; and

determining the relevance score as a maximal relevance score associated with any group of the at least one group.

16. The computerized apparatus of claim 11, wherein

determining that the word is a key term comprises determining that the recognizability score exceeds a first threshold, or the uniqueness score exceeds a second threshold.

17. The computerized apparatus of claim 11, wherein the processor is further adapted to perform a step comprising:

filtering the at least one indexed audio interactions for which the relevance score is above or below a predetermined value.

18. The computerized apparatus of claim 11, wherein combining the recognizability score and the uniqueness score into a word score is determined as the algebraic average or geometric average of the scores.

19. The computerized apparatus of claim 11, wherein the processor is further adapted to perform steps comprising sending a list of the at least one audio interactions that do not comply with the script to a user interface.

20. A computer program product comprising a non-transitory computer readable storage medium retaining program instructions, which program instructions when read by a processor, cause the processor to perform a method comprising:

receiving at least one audio interaction;

decoding speech within the at least one audio interaction;

indexing the decoded speech so as to obtain at least one indexed audio interaction;

receiving a text representing a script;

automatically extracting at least two key terms from the script, comprising;

discarding irrelevant terms;

determining a recognizability score for a word in the script based on a recognizability of a phoneme combination in the word;

determining a uniqueness score for the word in the script;

combining the recognizability score and the uniqueness score into a word score; and

determining that the word is a key term, subject to the word score exceeding a threshold;

automatically generating a query representing the script, comprising;

receiving at least one constraint related to a number of words difference between two key terms of the at least two key terms;

determining spotted key terms of the at least two key terms that appear in the at least one indexed audio interaction;

determining at least one complied constraint from the at least one received constraint; and

determining a relevance score for the at least one indexed audio interaction, based on the spotted key terms and the at least one complied constraint; and

using the relevance score to determine compliance of the at least one audio interaction with the script.