Method and system for generating semi-literal transcripts for speech recognition systems

• US 6,535,849 B1
• Filed: 01/18/2000
• Issued: 03/18/2003
• Est. Priority Date: 01/18/2000
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: ScanSoft, Inc. n/k/a Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Schonwetter, Michael J., Pakhomov, Sergey S.
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/487,398
• Time in Patent Office: 1,155 Days
• Field of Search: 704/255, 704/257, 704/260, 704/270, 704/235, 386/46
• US Class Current: 704/235
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A method for generating a semi-literal transcript from a partial transcript of recorded speech. The method includes augmenting the partial transcript with words from one of a filled pause model and a background model to build an augmented probabilistic finite state model for the partial transcript, inputting the recorded speech and the augmented probabilistic finite state model to a speech recognition system, and generating a hypothesized output for the recorded speech using the speech recognition system, whereby the hypothesized output may be used as the semi-literal transcript.

Claims

1. A method for generating a semi-literal transcript from a partial transcript of recorded speech, comprising:

(a) augmenting the partial transcript with words from one of a filled pause model and a background model to build an augmented probabilistic finite state model for the partial transcript;

(b) inputting the recorded speech and the augmented probabilistic finite state model to a speech recognition system; and

(c) generating a hypothesized output for the recorded speech using the speech recognition system, whereby the hypothesized output may be used as the semi-literal transcript.

2. The method of claim 1 further comprising integrating the hypothesized output with the partial transcript to generate the semi-literal transcript for the recorded speech.

3. The method of claim 1 wherein the act of augmenting comprises augmenting the partial transcript with words from the filled pause model.

4. The method of claim 1 wherein the act of augmenting comprises augmenting the partial transcript with words from the background model.

5. The method of claim 1 wherein the act of augmenting comprises assigning probabilities to the occurrence of words in the partial transcript.

6. The method of claim 2 wherein the act of integrating comprises aligning the words of the hypothesized output with the words of the partial transcript and determining which word to use in the semi-literal transcript if the words of the hypothesized output and the words of the partial transcript do not match.

7. The method of claim 6 wherein the act of aligning further comprises developing aligned pairs of words and labeling each aligned pair as one of a match, insertion, deletion, and substitution.

8. The method of claim 7 wherein the act of determining further comprises using a word in one of the hypothesized output and the partial transcript based on an intended use of the semi-literal transcript.

9. The method of claim 8 wherein the act of determining further comprises using a word from the hypothesized output for each aligned pair labeled as insertion if the intended use of the semi-literal transcript is for language modeling.

10. The method of claim 8 wherein the act of determining further comprises using a word from the hypothesized output for each aligned pair labeled as insertion if the intended use of the semi-literal transcript is for acoustic modeling and if the word in the hypothesized output is from one of the filled pause model and background model.

11. The method of claim 8 wherein the act of determining further comprises using a word from the partial transcript for each aligned pair labeled as insertion if the intended use of the semi-literal transcript is for acoustic modeling and if the word in the hypothesized output is not from one of the filled pause model and background model.

12. The method of claim 7 wherein the act of determining further comprises using a word from the partial transcript for each aligned pair labeled as one of match and deletion.

13. The method of claim 2 further comprising developing a pronunciation dictionary from the partial transcript for use in the speech recognition system.

14. The method of claim 2 further comprising building the filled pause model from a training corpus with filled pauses and a partial transcript without filled pauses.

15. The method of claim 2 further comprising building the background model using a literal transcript corpus and a partial transcript corpus.

16. A method for generating a semi-literal transcript from a partial transcript of recorded speech, comprising:

(a) augmenting the partial transcript with words from a filled pause model and a background model to build an augmented probabilistic finite state model for the partial transcript;

(b) inputting the recorded speech and the augmented probabilistic finite state model to a speech recognition system;

(c) generating a hypothesized output for the recorded speech using the speech recognition system; and

(d) integrating the hypothesized output with the partial transcript to generate a semi-literal transcript for the recorded speech.

17. The method of claim 16 wherein the act of integrating comprises aligning at least some of the words of the hypothesized output with the words of the partial transcript and determining which word to use in the semi-literal transcript if the words of the hypothesized output and the words of the partial transcript do not match.

18. The method of claim 16 further comprising using the semi-literal transcript as a substitute for a literal transcript of the recorded speech if an actual accuracy level of the semi-literal transcript is greater than a set accuracy level.

19. The method of claim 16 further comprising using only portions of the semi-literal transcript as a substitute for a literal transcript of the recorded speech depending upon whether the intended use of the semi-literal transcript is for one of language modeling and acoustic modeling.

20. A method for using a partial transcript in a speech recognition system, comprising:

(a) augmenting the partial transcript of recorded speech with words from a filled pause model and a background model to build an augmented probabilistic finite state model of the partial transcript;

(b) inputting the recorded speech and the augmented probabilistic finite state model to a speech recognition system;

(c) generating a hypothesized output for the recorded speech using the speech recognition system;

(d) integrating the hypothesized output with the partial transcript to generate a semi-literal transcript for the recorded speech; and

(e) using the semi-literal transcript as a substitute for a literal transcript of the recorded speech.

21. The method of claim 20 wherein the act of augmenting a partial transcript comprises assigning probabilities to the occurrence of filled pause words, background words, and tokenized words after a given tokenized word in the partial transcript.

22. The method of claim 21 wherein the act of generating a hypothesized output comprises using the speech recognition system to compare sounds of words in the recorded speech to the probabilities of the occurrence of words in the augmented probabilistic finite state model to build the hypothesized output.

23. The method of claim 20 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for language modeling.

24. The method of claim 20 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for acoustic modeling.

25. The method of claim 20 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for facilitated literal transcript generation.

26. The method of claim 20 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for speaker evaluation and diagnosis.

27. The method of claim 20 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for generation of pronunciation variants and probabilities.

28. A method for using a partial transcript of recorded speech, comprising:

(a) producing a semi-literal transcript from a partial transcript augmented with words from one of a filled pause model and a background model so as to form an augmented probabilistic finite state model of the partial transcript; and

(b) using the semi-literal transcript as a substitute for a literal transcript of the recorded speech.

29. The method of claim 28 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for language modeling.

30. The method of claim 28 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for acoustic modeling.

31. The method of claim 28 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for facilitated literal transcript generation.

32. The method of claim 28 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for speaker evaluation and diagnosis.

33. The method of claim 28 wherein the act of using the semi-literal transcript comprises using the semi-literal transcript for generation of pronunciation variants and probabilities.

34. An apparatus for generating a semi-literal transcript from a partial transcript of recorded speech, the apparatus comprising:

(a) an interpolator containing programs for augmenting the partial transcript with words from one of a filled pause model and a background model to build an augmented probabilistic finite state model of the partial transcript;

(b) a speech recognizer containing programs for generating a hypothesized output for the recorded speech using the augmented probabilistic finite state model and the recorded speech as inputs; and

(c) an integrator containing instructions for integrating the hypothesized output with the partial transcript to generate a semi-literal transcript of the recorded speech.