Analyzing speech application performance

• US 8,929,519 B2
• Filed: 12/23/2013
• Issued: 01/06/2015
• Est. Priority Date: 09/22/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dhanakshirur, Girish
• Primary Examiner(s): SING, SIMON P
• Application Number:
  US14/138,512
• Publication Number:
  US 20140112461A1
• Time in Patent Office: 379 Days
• Field of Search: 379/72, 379/88.18
• US Class Current: 379/88.18
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: H04M 3/2227 Quality of service monitoring
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Abstract

A method of analyzing speech application performance can include determining a call path for each of a plurality of calls from a log of an interactive voice response system having a speech application. Each call path can be defined by an ordered set of dialog nodes of the speech application. A number of occurrences of at least one selected type of event for the dialog nodes of the plurality of calls can be counted. At least one call path from the plurality of call paths can be indicated according, at least in part, to the number of occurrences of the selected type of event within the at least one indicated call path.

Claims

1. A computer hardware system configured to analyze speech application performance, comprising:

at least one processor, wherein the at least one processor is configured to initiate and/or perform;

identifying, from a log of an interactive voice response system including a speech application, a call path for each of a plurality of calls, each call path is defined by an ordered set of dialog nodes of the speech application;

counting a number of occurrences of at least one selected type of event for the dialog nodes of the plurality of call paths; and

indicating at least one call path from the plurality of call paths according to the number of occurrences of the selected type of event within the at least one indicated call path.

2. The system of claim 1, wherein the at least one processor is configured to initiate and/or perform

presenting a graphic illustration of the at least one indicated call path.

3. The system of claim 1, wherein

for each node of a call path, the at least one processor is configured to initiate and/or perform:

determining whether the selected type of event occurred at a current dialog node;

creating and initializing a counter, associated with the current dialog node and the selected type of event, to a default value upon no counter for the selected type of event existing at the current dialog node; and

incrementing the counter upon the counter being associated with the selected type of event for the current dialog node.

4. The system of claim 1, wherein

the indicating includes selecting at least one call path from the plurality of call paths based upon the number of occurrences of the at least one type of event for a particular dialog node.

5. The system of claim 1, wherein the at least one processor is configured to initiate and/or perform

grouping calls having matching call paths.

6. The system of claim 5, wherein

a group of calls are indicated according to a measure of the number of occurrences of the selected type of event within the call paths of the group of calls.

7. The system of claim 5, wherein the at least one processor is configured to initiate and/or perform

determining the number of calls within each group of calls.

8. The system of claim 7, wherein

call paths of a group of calls are indicated based upon a measure of the number of calls within that group of calls.

9. The system of claim 1, wherein

the event type is selected to be a no match event.

10. The system of claim 1, wherein

the event type is selected to be a confidence score below a predetermined threshold.

11. The system of claim 1, wherein

the event type is selected to be a misrecognition event.

12. The system of claim 1, wherein

the event type is selected from the group consisting of:

a re-prompt event,

a hang-up event, and

an agent transfer event.

13. A computer hardware system configured to analyze speech application performance, comprising:

at least one processor, wherein the at least one processor is configured to initiate and/or perform;

identifying a call path for each of a plurality of calls listed within a log of an interactive voice response system having a speech application, each call path is defined by an ordered set of dialog nodes of the speech application;

determining a number of occurrences of a selected type of event at each dialog node of the call path for each of the plurality of calls;

identifying call paths of the plurality of calls that correspond to search criteria specifying a number of occurrences of the selected type of event; and

presenting a graphic illustration of an identified call path.

14. The system of claim 13, wherein

the identifying includes selecting at least one call path according to the number of occurrences of the selected type of event for a particular dialog node.

15. The system of claim 13, wherein the at least one processor is configured to initiate and/or perform

grouping calls having matching call paths.

16. The system of claim 15, wherein

the identifying includes selecting a group of calls according to a measure of the number of occurrences of the selected type of event within the call paths of the group of calls.