System and method for analyzing automatic speech recognition performance data

• US 7,383,170 B2
• Filed: 10/10/2003
• Issued: 06/03/2008
• Est. Priority Date: 10/10/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Knowledge Ventures, L.P. (AT&T, Inc.)
• Inventors: Mills, Scott H., Martin, John M.
• Primary Examiner(s): Hudspeth, David
• Assistant Examiner(s): Neway, Samuel G
• Application Number:
  US10/683,648
• Publication Number:
  US 20050080630A1
• Time in Patent Office: 1,698 Days
• Field of Search: None
• US Class Current: 704/9
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems

Abstract

In a disclosed method for interpreting automatic speech recognition (ASR) performance data, a data processing system may receive user input that selects a log file to be processed. The log file may contain log records produced by an ASR system as a result of verbal interaction between an individual and the ASR system. In response to receiving the user input, the data processing system may automatically interpret data in the log records and generate interpretation results. The interpretation results may include a duration for a system prompt communicated to the individual by the ASR system, a user response to the system prompt, and a duration for the user response. The user response may include a textual representation of a verbal response from the individual, obtained through ASR. The interpretation results may also include an overall duration for the telephone call.

Claims

1. A method for interpreting automatic speech recognition (ASR) performance data, the method comprising:

receiving user input that selects a log file to be processed, wherein the log file contains log records produced by an ASR system as a result of verbal interaction between an individual and the ASR system during a telephone call; and

in response to receiving the user input, automatically interpreting data in the log records and generating interpretation results;

wherein the interpretation results include general analysis results and exchange analysis results;

wherein the general analysis results include;

an identifier for the individual, a date for the telephone call, a start time for the telephone call, an overall duration for the telephone call, a cumulative duration for multiple system prompt portions of the telephone call, and a cumulative duration for multiple user reply portions of the telephone call; and

wherein the exchange analysis results include;

an identifier for a system prompt communicated to the individual by the ASR system, a duration for the system prompt, a user response to the system prompt, wherein the user response includes a textual representation of a verbal response from the individual, obtained through ASR, and a duration for the user response; and

wherein the exchange analysis results explicitly tie together the identifier for the system prompt, the duration for the system prompt, the user response, and the duration for the user response by grouping the identifier for the system prompt, the duration for the system prompt, the user response, and the duration for the user response together in a row.

2. A method for interpreting automatic speech recognition (ASR) performance data, the method comprising:

receiving user input that selects a log file to be processed, wherein the log file contains log records produced by an ASR system as a result of verbal interaction between an individual and the ASR system; and

in response to receiving the user input, automatically interpreting data in the log records and generating interpretation results;

wherein the interpretation results include;

a duration for a system prompt communicated to the individual by the ASR system, a user response to the system prompt, wherein the user response includes a textual representation of a verbal response from the individual, obtained through ASR, and a duration for the user response wherein the interpretation results explicitly tie together the duration for the system prompt, the user response, and the duration for the user response by grouping the duration for the system prompt, the user response, and the duration for the user response together in a row.

3. A method according to claim 2, wherein the verbal interaction between an individual and the ASR system occurred during a telephone call and wherein the interpretation results further comprise an overall duration for the telephone call.

4. A method according to claim 2, wherein the operation of automatically interpreting data in the log records and generating interpretation results comprises:

automatically identifying a start time for the system prompt within the log file;

automatically identifying an end time for the system prompt within the log file;

computing the duration for the system prompt, based on the start time and the end time.

5. A method according to claim 2, wherein the operation of automatically interpreting data in the log records and generating interpretation results comprises:

automatically identifying a start time for the user response within the log file;

automatically identifying an end time for the user response within the log file;

computing the duration for the user response, based on the start time and the end time.

6. A method according to claim 2, wherein the operation of automatically generating interpretation results comprises:

generating interpretation results that reproduce less than seventy-five percent of the data from the log file.

7. A method according to claim 2, wherein the operation of automatically generating interpretation results comprises:

generating interpretation results that reproduce less than half of the data from the log file.

8. A method according to claim 2, wherein the verbal interaction between an individual and the ASR system occurred during a telephone call and wherein:

the operation of automatically generating interpretation results comprises generating general analysis results and exchange analysis results;

the general analysis results include the overall duration for the telephone call; and

the exchange analysis results include the duration for the system prompt, the user response to the system prompt, and

the duration for the user response.

9. A program product for analyzing automatic speech recognition (ASR) performance data, the program product comprising:

a computer-usable storage medium;

an ASR log analysis engine encoded in the computer-usable storage medium, wherein the ASR log analysis engine performs operations comprising;

receiving user input that selects a log file to be processed, wherein the log file contains log records produced by an ASR system as a result of verbal interaction between an individual and the ASR system; and

in response to receiving the user input, automatically interpreting data in the log records and generating interpretation results;

wherein the interpretation results include;

a duration for a system prompt communicated to the individual by the ASR system, a user response to the system prompt, wherein the user response includes a textual representation of a verbal response from the individual, obtained through ASR, and a duration for the user response wherein the interpretation results explicitly tie together the duration for the system prompt, the user response, and the duration for the user response by grouping the duration for the system prompt, the user response, and the duration for the user response together in a row.

10. The program product of claim 9, wherein the verbal interaction between an individual and the ASR system occurred during a telephone call and wherein the interpretation results generated by the ASR log analysis engine further comprise an overall duration for the telephone call.

11. The program product of claim 9, wherein the operation of automatically interpreting data in the log records and generating interpretation results comprises:

automatically identifying a start time for the system prompt within the log file;

automatically identifying an end time for the system prompt within the log file;

computing the duration for the system prompt, based on the start time and the end time.

12. The program product of claim 9, wherein the operation of automatically interpreting data in the log records and generating interpretation results comprises:

automatically identifying a start time for the user response within the log file;

automatically identifying an end time for the user response within the log file;

computing the duration for the user response, based on the start time and the end time.

13. The program product of claim 9, wherein the operation of automatically generating interpretation results comprises:

generating interpretation results that reproduce less than seventy-five percent of the data from the log file.

14. The program product of claim 9, wherein the operation of automatically generating interpretation results comprises:

generating interpretation results that reproduce less than half of the data from the log file.

15. The program product of claim 9, wherein the verbal interaction between an individual and the ASR system occurred during a telephone call and wherein:

the operation of automatically generating interpretation results comprises generating general analysis results and exchange analysis results;

the general analysis results include the overall duration for the telephone call; and

the exchange analysis results include the duration for the system prompt, the user response to the system prompt, and

the duration for the user response.

16. The program product of claim 15, wherein the operation of automatically generating interpretation results comprises grouping the general analysis results together and grouping the exchange analysis results together.