System and method for the automated analysis of performance data

• US 6,842,504 B2
• Filed: 08/13/2002
• Issued: 01/11/2005
• Est. Priority Date: 07/02/2002
• Status: Active Grant

• Current Assignee: SBC Properties, L.P.
• Original Assignee: SBC Properties, L.P.
• Inventors: Bushey, Robert R., Martin, John M., Knott, Benjamin A., Mills, Scott H., Joseph, Kurt M., Pasquale, Theodore B.
• Primary Examiner(s): WEAVER, SCOTT LOUIS
• Application Number:
  US10/217,863
• Publication Number:
  US 20040032935A1
• Time in Patent Office: 882 Days
• Field of Search: 379/1.02, 379/10.01, 379/10.02, 379/10.03, 379/27.03, 379/29.02, 379/29.1, 379/67.1, 379/88.11, 379/88.24
• US Class Current: 379/67.1
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: H04M 15/00 Arrangements for metering, time-control or time indication ; Metering, charging or billing arrangements for voice wireline or wireless communications, e.g. VoIP

Abstract

A method and system for automating the analysis of performance data includes an analysis system automatically analyzing a plurality of performance data sets, a task engine, and a score engine. The task engine allows for the matching of each performance data set with an assigned task for each performance data set and the retrieval of a correct key sequence for each performance data set. The score engine compares the correct key sequence with a recorded key sequence of the performance data set to determine if the assigned task is successfully accomplished. The score engine further calculates one or more response times for each performance data set. The automated analysis of performance data allows for a cost savings, more efficient use of time, and more reliable and consistent performance data analysis results.

Claims

1. A method for the automated analysis of interactive voice response system performance data, the method comprising:

determining a task for a performance data set;

retrieving a correct key sequence for the task;

searching for the correct key sequence at the end of the performance data set;

automatically comparing the correct key sequence for the task with a recorded key sequence of the performance data set;

determining if the task was successfully completed based on the comparison of the correct key sequence and the recorded key sequence based on the recorded key sequence corresponding with the correct key sequence;

scoring the performance data set as successful or unsuccessful;

obtaining one or more menu duration times;

automatically calculating one or more response times for one or more menus by subtracting the menu duration time from a listening time for each menu;

calculating a total response time for the performance data set; and

generating an output file sorted by a participant number and by a task number.

2. The method of claim 1 wherein determining a task for the performance data set comprises matching a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

3. A method for the automated analysis of performance data, the method comprising:

determining a task for a performance data set;

retrieving a correct key sequence for the task;

automatically comparing the correct key sequence for the task with a recorded key sequence of the performance data set; and

automatically calculating one or more response times associated with performance of the correct key sequence.

4. The method of claim 3 wherein calculating one or more response times comprises calculating a response time for an individual menu.

5. The method of claim 3 wherein calculating one or more response times comprises calculating a total response time for the performance data set.

6. The method of claim 3 wherein calculating the response time comprises obtaining one or more menu duration times.

7. The method of claim 3 wherein determining a task for the performance data set comprises matching a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

8. The method of claim 3 wherein comparing the correct key sequence with a recorded key sequence comprises determining the task is successfully completed if the recorded key sequence corresponds with the correct key sequence.

9. The method of claim 3 wherein comparing the correct key sequence with a recorded key sequence comprises searching for the correct key sequence at the end of the performance data set.

10. The method of claim 3 further comprising generating an output file.

11. The method of claim 10 wherein generating the output file comprises sorting the output file by a participant number and by the order in which a participant performed the task.

12. Software for the automated analysis of performance data, the software embodied in a computer-readable medium and operable to:

determine a task for a performance data set;

retrieve a correct key sequence for the task;

automatically compare the correct key sequence for the task with a recorded key sequence of the performance data set; and

automatically calculate one or more response times associated with performance of the correct key sequence.

13. The software of claim 12 wherein calculating one or more response times comprises calculating a response time for one or more menus.

14. The software of claim 12 wherein calculating one or more response times comprises calculating a total response time for the performance data set.

15. The software of claim 12 wherein calculating the response time comprises obtaining one or more menu duration times.

16. The software of claim 12 wherein determining a task for the performance data set comprises matching a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

17. The software of claim 12 wherein comparing the correct key sequence with a recorded key sequence comprises determining the task is successfully completed if the recorded key sequence corresponds with the correct key sequence.

18. The software of claim 12 wherein comparing the correct key sequence with a recorded key sequence comprises searching for the correct key sequence at the end of the performance data set.

19. The software of claim 12 further operable to generate an output file sorted by a participant number and by the order in which a participant performed the task.

20. A system for the automated analysis of performance data, the system comprising:

a plurality of performance data sets;

a task engine operable to determine a task for each performance data set and retrieve a correct key sequence for each task; and

a score engine associated with the task engine, the score engine operable to compare the correct key sequence for the task with a recorded key sequence of the performance data set and calculate one or more response times associated with performance of the correct key sequence.

21. The system of claim 20 wherein the score engine calculates a response time for one or more menus.

22. The system of claim 20 wherein the score engine calculates a total response time for each performance data set.

23. The system of claim 20 wherein the score engine obtains one or more menu duration times.

24. The system of claim 20 wherein the score engine subtracts a menu duration time from a listening time in order to calculate the response time.

25. The system of claim 20 wherein the task engine matches a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

26. The system of claim 20 wherein the score engine is further operable to determine if the task was successfully completed based on the recorded key sequence corresponding with the correct key sequence.

27. The system of claim 20 wherein the score engine is further operable to generate an output file.

28. The system of claim 20 wherein the tasks comprise a plurality of simulated interactions with an interactive voice response system.

29. The system of claim 20 further comprising a graphical user interface associated with the task engine and the score engine, the graphical user interface having a summary window and operable to allow for the selection of the performance data set to be analyzed.

30. A method for the automated analysis of performance data, the method comprising;

determining a task for a performance data set;

retrieving a correct key sequence for the task;

automatically comparing the correct key sequence for the task with a recorded key sequence of the performance data set; and

automatically calculating one or more response times, wherein calculating the response time comprises obtaining one or more menu duration times and subtracting the menu duration time from a listening time.

33. The method of claim 30 wherein calculating one or more response times comprises calculating a response time for an individual menu.

34. The method of claim 30 wherein calculating one or more response times comprises calculating a total response time for the performance data set.

35. The method of claim 30 wherein determining a task for the performance data set comprises matching a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

31. Software for the automated analysis of performance data, the software embodied in a computer-readable medium and operable to:

determine a task for a performance data set;

retrieve a correct key sequence for the task;

automatically compare the correct key sequence for the task with a recorded key sequence of the performance data set; and

automatically calculate one or more response times, wherein calculating the response time comprises obtaining one or more menu duration times and subtracting the menu duration time from a listening time.

36. The software of claim 31 wherein calculating one or more response times comprises calculating a response time for an individual menu.

37. The software of claim 31 wherein calculating one or more response times comprises calculating a total response time for the performance data set.

38. The software of claim 31 wherein determining a task for the performance data set comprises matching a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.

32. A system for the automated analysis of performance data, the system comprising:

a plurality of performance data sets;

a task engine operable to determine a task for each performance data set and retrieve a correct key sequence for each task; and

a score engine associated with the task engine, the score engine operable to compare the correct key sequence for the task with a recorded key sequence of the performance data set and calculate one or more response times, wherein the score engine obtains one or more menu duration times and subtracts the one or more menu duration times from a listening time in order to calculate the response time.

39. The system of claim 32 wherein the score engine is operable to calculate a response time for an individual menu.

40. The system of claim 32 wherein the score engine is operable to calculate a total response time for respective performance data sets.

41. The system of claim 32 wherein the task engine is operable to match a participant number of the performance data set and a sequence number of the performance data set to one of a plurality of tasks in a database.