Performance gauge for a distributed speech recognition system

• US 20030105630A1
• Filed: 11/30/2001
• Published: 06/05/2003
• Est. Priority Date: 11/30/2001
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dictaphone Corporation (Microsoft Corporation)
• Inventors: Greene, Channell, Kuhnen, Regina, MacGinitie, Andrew, Cyr, James, Hold, Martin
• Granted Patent: US 6,766,294 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems

Abstract

A performance gauge for use in conjunction with a transcription system including a speech processor linked to at least one speech recognition engine and at least one transcriptionist. The speech processor includes an input for receiving speech files and storage means for storing the received speech files until such a time that they are forwarded to a selected speech recognition engine or transcriptionist for processing. The system includes a transcriptionist text file database in which manually transcribed transcriptionist text files are stored, each stored transcriptionist text file including time stamped data indicative of position within an original speech file. The system further includes a recognition engine text file database in which recognition engine text files transcribed via the at least one speech recognition engine are stored, each stored recognition engine text file including time stamped data indicative of position within an original speech file. The system further includes a comparator comparing the time stamped recognition engine text files with time stamped transcriptionist text files based upon the same speech file so as to determine differences between the recognition engine text file and the transcriptionist text file and a report generator compiling the identified differences so as to issue a report for evaluation by administrators of the transcription system.

Claims

1. A performance gauge for use in conjunction with a transcription system including a speech processor linked to at least one speech recognition engine and at least one transcriptionist, the speech processor includes an input for receiving speech files and storage means for storing the received speech files until such a time that they are forwarded to a selected speech recognition engine or transcriptionist for processing, the system comprising;

a transcriptionist text file database in which manually transcribed transcriptionist text files are stored, each stored transcriptionist text file including time stamped data indicative of position within an original speech file;

a recognition engine text file database in which recognition engine text files transcribed via the at least one speech recognition engine are stored, each stored recognition engine text file including time stamped data indicative of position within an original speech file;

a comparator comparing the time stamped recognition engine text files with time stamped transcriptionist text files based upon the same speech file so as to determine differences between the recognition engine text file and the transcriptionist text file;

a report generator compiling the identified differences so as to issue a report for evaluation by administrators of the transcription system.

2. The performance gauge according to claim 1,wherein the system includes a speech processor linked to a plurality of speech recognition engines and a plurality of transcriptionists.

3. The performance gauge according to claim 2,wherein recognition engine text files and transcriptionist text files are linked within a database of the speech processor.

4. The performance gauge according to claim 1,wherein differences determined by the comparator include additions, deletions and mistranscriptions.

5. The performance gauge according to claim 4,wherein the differences are ascertained with reference to the transcriptionist text file.

6. The performance gauge according to claim 1,wherein the comparator is limited to comparisons of identical time stamped segments.

7. A method for gauging performance within a transcription system including a speech processor linked to at least one speech recognition engine and at least one transcriptionist, the speech processor includes an input for receiving speech files and storage means for storing the received speech files until such a time that they are forwarded to a selected speech recognition engine or transcriptionist for processing, the method including the following steps;

preparing a recognition engine text file based upon automated speech recognition of a speech file via the at least one speech recognition engine, the recognition engine text file including time stamped data indicative of position within the speech file;

storing the recognition engine text file;

preparing a transcriptionist text file based upon manual transcription of the speech file, the stored transcriptionist text file including time stamped data indicative of position within the speech file;

storing the transcriptionist text file;

comparing the time stamped recognition engine text file with time stamped transcriptionist text file so as to determine differences between the recognition engine text file and the transcriptionist text file;

compiling and reporting the differences for evaluation by administrators of the transcription system.

8. The method according to claim 7,wherein the step of preparing a transcriptionist text file includes manually reviewing the recognition engine text file.

9. The method according to claim 8,wherein the step of manually reviewing the recognition engine text file includes listening to the speech file while reviewing the recognition engine text file and preparing the transcriptionist text file.

10. The method according to claim 7,wherein the system includes a speech processor linked to a plurality of speech recognition engines and a plurality of transcriptionists.

11. The method according to claim 10,wherein recognition engine text files and transcriptionist text files are linked within a database of the speech processor.

12. The method according to claim 7,wherein differences determined by the comparator include additions, deletions and mistranscriptions.

13. The method according to claim 12,wherein the differences are ascertained with reference to the transcriptionist text file.

14. The method according to claim 7,wherein the comparator is limited to comparisons of identical time stamped segments.