System and method for analysis and adjustment of speech-enabled systems

• US 8,117,030 B2
• Filed: 09/13/2006
• Issued: 02/14/2012
• Est. Priority Date: 09/13/2004
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Bushey, Robert R., Knott, Benjamin A., Martin, John M.
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US11/531,509
• Publication Number:
  US 20070027694A1
• Time in Patent Office: 1,980 Days
• Field of Search: 704/243, 704/244, 704/270, 704/270.1, 704/275
• US Class Current: 704/243
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 2015/0636 Threshold criteria for the updating
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 2203/2011 Service processing based on information specified by a party before or during a call, e.g. information, tone or routing selection
• CPC Class Codes: H04M 3/5166 in combination with interactive voice response systems or voice portals, e.g. as front-ends

Abstract

A method for analyzing and adjusting the performance of a speech-enabled application includes selecting a number of user utterances that were previously received by the speech-enabled application. The speech-enabled application receives such user utterances and associates each user utterance with an action-object based on one or more salient terms in the user utterance that are associated with the action-object. The method further includes associating one of a number of action-objects with each of the selected user utterances. Furthermore, for each action-object, the percentage of the utterances associated with the action-object that include at least one of the salient terms associated with the action-object is determined. If the percentage does not exceed a selected threshold, the method also includes adjusting the one or more salient terms associated with the action-object.

Claims

1. A computer implemented method for adjusting a call center application operable to associate user utterances with action-objects based on existing sets of salient terms, the method for adjusting comprising:

automatically adding, by the computer, a new salient term to an existing set of salient terms associated with an action-object when a percentage of user utterances failing to associate with the action-object, based upon the existing set of salient terms, exceeds a specified threshold; and

automatically removing, by the computer, a current association of a salient term with the action-object from the existing set of salient terms when a percentage of user utterances associating with the action-object, based upon the existing set of salient terms, exceeds a specified threshold,

wherein a size of the existing set of salient terms is maintained near an optimum value with regard to a match rate and a performance rate of the call center application.

2. The method of claim 1, wherein adding the new salient term includes selecting the new salient term based on a count indicative of terms used in a selected set of user utterances.

3. The method of claim 2, wherein adding the new salient term further includes removing an existing salient term.

4. The method of claim 3, wherein removing the current association of the salient term is based on the count.

5. The method of claim 1, wherein removing the current association of the salient term includes selecting the salient term based on a count indicative of terms used in a selected set of user utterances.

6. The method of claim 5, wherein removing the current association of the salient term further includes adding a new salient term.

7. The method of claim 6, wherein adding the new salient term is based on the count.

8. A non-transitory computer readable memory medium, including computer executable instructions for adjusting a call center application operable to associate a user utterance with an action-object based on a salient term in the user utterance, the instructions including instructions executable to:

determine that at least one of a match rate for associating user utterances with the action-object and a performance rate for the associating user utterances with the action-object is below a corresponding threshold; and

responsive to executing the instructions to determine, modify salient terms associated with the action-object;

wherein the instructions executable to modify comprise instructions executable to remove an association of the salient term with the action-object based on a frequency with which the salient term occurs in a selected set of user utterances, and

wherein a size of a set of salient terms matching the action-object is maintained near an optimum value with regard to the match rate and the performance rate.

9. The memory medium of claim 8, wherein the instructions executable to modify include instructions executable to add a new salient term associated with the action-object.

10. The memory medium of claim 9, wherein the instructions executable to add comprise instructions executable to select the new salient term based on a word count indicative of a frequency with which terms occur in a selected set of user utterances.

11. The memory medium of claim 8, wherein the instructions executable to remove comprise instructions executable to remove the association of the salient term based on a word count indicative of the frequency.

12. The memory medium of claim 11, wherein a word count corresponding to the new salient term is greater than a word count corresponding to the salient term.

13. A service center system comprising a computing apparatus, including a processor having access to a computer readable storage medium storing processor executable instructions, the computing apparatus being enabled to execute the instructions to:

associate a user utterance with an action-object based on a salient term in the user utterance; and

adjust at least one of a set of salient action terms and salient object terms associated with the action-object when a specified percentage of user utterances including the salient term associate with the action-object, and wherein the instructions to adjust include instructions to remove an association of the salient term with the action-object,

wherein a size of a set of salient terms associated with the action-object is maintained near an optimum value with regard to a match rate and a performance rate of the service center system.

14. The service center system of claim 13, wherein the instructions to adjust include instructions to add a new salient term associated with the action-object, wherein the new salient term includes at least one of a new salient action term and a new salient object term.

15. The service center system of claim 14, wherein the instructions to add include instructions to select the new salient term based on a word count indicative of a frequency with which terms occur in a selected set of user utterances.

16. The service center system of claim 15, wherein the instructions to remove include instructions to remove the association of the salient term based on the word count.

17. The service center system of claim 16, wherein a word count corresponding to the new salient term is greater than a word count corresponding to the salient term.