Analyzing Speech Application Performance

• US 8,666,040 B2
• Filed: 09/22/2006
• Issued: 03/04/2014
• Est. Priority Date: 09/22/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dhanakshirur, Girish
• Primary Examiner(s): SING, SIMON P
• Application Number:
  US11/534,309
• Publication Number:
  US 20080084971A1
• Time in Patent Office: 2,720 Days
• Field of Search: 379/265.03, 379/88.18
• US Class Current: 379/88.18
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: H04M 3/2227 Quality of service monitoring
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Abstract

A method of analyzing speech application performance can include determining a call path for each of a plurality of calls from a log of an interactive voice response system having a speech application. Each call path can be defined by an ordered set of dialog nodes of the speech application. A number of occurrences of at least one selected type of event for the dialog nodes of the plurality of calls can be counted. At least one call path from the plurality of call paths can be indicated according, at least in part, to the number of occurrences of the selected type of event within the at least one indicated call path.

Claims

1. A method of analyzing speech application performance comprising:

determining a call path for each of a plurality of calls from a log of an interactive voice response system comprising a speech application, wherein each call path is defined by an ordered set of dialog nodes of the speech application;

counting a number of occurrences of at least one selected type of event for the dialog nodes of the plurality of call paths; and

indicating at least one call path from the plurality of call paths according, at least in part, to the number of occurrences of the selected type of event within the at least one indicated call path.

2. The method of claim 1, further comprising presenting a graphic illustration of the at least one indicated call path.

3. The method of claim 1, wherein, for each node of a call path, determining a number of occurrences further comprises:

determining whether the selected type of event occurred at a current dialog node;

if no counter for the event type exists at the current dialog node, creating a counter associated with the current dialog node and selected type of event and initializing the counter to a default value; and

if a counter is associated with the selected type of event for the current dialog node, incrementing the counter.

4. The method of claim 1, wherein indicating at least one call path from the plurality of call paths further comprises selecting at least one call path from the plurality of call paths according to the number of occurrences of the at least one type of event for a particular dialog node.

5. The method of claim 1, further comprising grouping calls having matching call paths.

6. The method of claim 5, wherein indicating at least one call path further comprises indicating a group of calls according to a measure of the number of occurrences of the selected type of event within the call paths of the group of calls.

7. The method of claim 5, further comprising determining the number of calls within each group of calls.

8. The method of claim 7, wherein indicating at least one call path further comprises indicating call paths of a group of calls according, at least in part, to a measure of the number of calls within that group of calls.

9. The method of claim 1, further comprising selecting the event type to be a no match event.

10. The method of claim 1, further comprising selecting the event type to be a confidence score below a predetermined threshold.

11. The method of claim 1, further comprising selecting the event type to be a misrecognition event.

12. The method of claim 1, further comprising selecting the event type from the group consisting of a re-prompt event, a hang-up event, and an agent transfer event.

13. A method of analyzing speech application performance comprising:

determining a call path for each of a plurality of calls listed within a log of an interactive voice response system having a speech application, wherein each call path is defined by an ordered set of dialog nodes of the speech application;

determining a number of occurrences of a selected type of event at each dialog node of the call path for each of the plurality of calls;

identifying call paths of the plurality of calls that correspond to search criteria specifying, at least in part, a number of occurrences of the selected type of event; and

presenting a graphic illustration of an identified call path.

14. The method of claim 13, wherein identifying call paths of the plurality of calls further comprises selecting at least one call path according to the number of occurrences of the selected type of event for a particular dialog node.

15. The method of claim 13, further comprising grouping calls having matching call paths.

16. The method of claim 15, wherein identifying call paths of the plurality of calls further comprises selecting a group of calls according to a measure of the number of occurrences of the selected type of event within the call paths of the group of calls.

17. A computer usable storage device having stored therein a computer usable program code for analyzing speech application performance, the computer usable program code, which when executed by a computer hardware system, causes the computer hardware system to perform:

determining a call path for each of a plurality of calls from a log of an interactive voice response system comprising a speech application, wherein each call path is defined by an ordered set of dialog nodes of the speech application;

determining a number of occurrences of at least one selected type of event for the dialog nodes of the plurality of call paths; and

indicating at least one call path from the plurality of call paths according, at least in part, to the number of occurrences of the selected type of event within the at least one indicated call path, wherein

the computer usable storage device is not a transitory, propagating signal per se.

18. The computer usable storage device of claim 17, wherein

the indicating includes selecting at least one call path from the plurality of call paths according to a number of occurrences of the at least one type of event for a particular dialog node.

19. The computer usable storage device of claim 17, wherein the computer usable program code further causes the computer hardware system to perform

grouping calls having matching call paths.

20. The computer usable storage device of claim 19, wherein

the indicating includes indicating a group of calls according to a measure of a number of occurrences of the selected type of event within the call paths of the group of calls.