Discriminative training of document transcription system
First Claim
1. In a computer system including a first document tangibly stored in a first computer-readable medium and containing at least some information in common with a spoken audio stream, a method performed by at least one computer processor executing computer program instructions tangibly stored in a second computer-readable medium, the method comprising steps of:
- (A) identifying first text tangibly stored in the first document on a third computer-readable medium, wherein the first text represents a first instance of a concept;
(B) identifying, based on the identified first text, a first plurality of at least three spoken forms of the first instance of the concept, including at least one spoken form not contained in the first document;
(C) replacing the identified first text with a first context-free grammar specifying the first plurality of spoken forms of the first instance of the concept to produce a second document tangibly stored in a fourth computer-readable medium;
(D) identifying second text tangibly stored in the first document on the third computer-readable medium, wherein the second text represents a second instance of the concept;
(E) identifying, based on the identified second text, a second plurality of at least three spoken forms of the second instance of the concept, wherein the first plurality of spoken forms differs from the second plurality of spoken forms;
(F) replacing the identified second text with a second context-free grammar specifying the second plurality of spoken forms of the second instance of the concept within the second document;
(G) generating a first language model, tangibly stored in a fifth computer-readable medium, based on the second document;
(H) using the first language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a third document tangibly stored in a sixth computer-readable medium;
(I) filtering text from the third document by reference to the second document to produce a filtered document, tangibly stored in a seventh computer-readable medium, in which text filtered from the third document is marked as unreliable; and
(J) using the filtered document and the spoken audio stream to train an acoustic model, tangibly stored in an eighth computer-readable medium, by performing steps of;
(J)(1) applying a first speech recognition process to the spoken audio stream using a set of base acoustic models and a grammar network based on the filtered document to produce a first set of recognition structures tangibly stored in a ninth computer-readable medium;
(J)(2) applying a second speech recognition process to the spoken audio stream using the set of base acoustic models and a second language model to produce a second set of recognition structures tangibly stored in a tenth computer-readable medium; and
(J)(3) performing discriminative training of the acoustic model using the first set of recognition structures, the second set of recognition structures, the filtered document, and only those portions of the spoken audio stream corresponding to text not marked as unreliable in the filtered document.
14 Assignments
0 Petitions
Accused Products
Abstract
A system is provided for training an acoustic model for use in speech recognition. In particular, such a system may be used to perform training based on a spoken audio stream and a non-literal transcript of the spoken audio stream. Such a system may identify text in the non-literal transcript which represents concepts having multiple spoken forms. The system may attempt to identify the actual spoken form in the audio stream which produced the corresponding text in the non-literal transcript, and thereby produce a revised transcript which more accurately represents the spoken audio stream. The revised, and more accurate, transcript may be used to train the acoustic model using discriminative training techniques, thereby producing a better acoustic model than that which would be produced using conventional techniques, which perform training based directly on the original non-literal transcript.
74 Citations
18 Claims
-
1. In a computer system including a first document tangibly stored in a first computer-readable medium and containing at least some information in common with a spoken audio stream, a method performed by at least one computer processor executing computer program instructions tangibly stored in a second computer-readable medium, the method comprising steps of:
-
(A) identifying first text tangibly stored in the first document on a third computer-readable medium, wherein the first text represents a first instance of a concept; (B) identifying, based on the identified first text, a first plurality of at least three spoken forms of the first instance of the concept, including at least one spoken form not contained in the first document; (C) replacing the identified first text with a first context-free grammar specifying the first plurality of spoken forms of the first instance of the concept to produce a second document tangibly stored in a fourth computer-readable medium; (D) identifying second text tangibly stored in the first document on the third computer-readable medium, wherein the second text represents a second instance of the concept; (E) identifying, based on the identified second text, a second plurality of at least three spoken forms of the second instance of the concept, wherein the first plurality of spoken forms differs from the second plurality of spoken forms; (F) replacing the identified second text with a second context-free grammar specifying the second plurality of spoken forms of the second instance of the concept within the second document; (G) generating a first language model, tangibly stored in a fifth computer-readable medium, based on the second document; (H) using the first language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a third document tangibly stored in a sixth computer-readable medium; (I) filtering text from the third document by reference to the second document to produce a filtered document, tangibly stored in a seventh computer-readable medium, in which text filtered from the third document is marked as unreliable; and (J) using the filtered document and the spoken audio stream to train an acoustic model, tangibly stored in an eighth computer-readable medium, by performing steps of; (J)(1) applying a first speech recognition process to the spoken audio stream using a set of base acoustic models and a grammar network based on the filtered document to produce a first set of recognition structures tangibly stored in a ninth computer-readable medium; (J)(2) applying a second speech recognition process to the spoken audio stream using the set of base acoustic models and a second language model to produce a second set of recognition structures tangibly stored in a tenth computer-readable medium; and (J)(3) performing discriminative training of the acoustic model using the first set of recognition structures, the second set of recognition structures, the filtered document, and only those portions of the spoken audio stream corresponding to text not marked as unreliable in the filtered document. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A computer-implemented system comprising:
-
a first document, tangibly stored in a first non-transitory computer-readable medium, containing at least some information in common with a spoken audio stream; and computer-readable program instructions tangibly stored in a second non-transitory computer-readable medium, said computer-readable program instructions adapted to be executed by a computer processor to perform a method comprising; identifying first text tangibly stored in the first document on a third non-transitory computer-readable medium, wherein the first text represents a first instance of a concept; identifying, based on the identified first text, a first plurality of at least three spoken forms of the first instance of the concept, including at least one spoken form not contained in the first document; replacing the identified first text with a first context-free grammar specifying the first plurality of spoken forms of the first instance of the concept to produce a second document tangibly stored in a fourth non-transitory computer-readable medium; identifying second text tangibly stored in the first document on the third non-transitory computer-readable medium, wherein the second text represents a second instance of the concept; identifying, based on the identified second text, a second plurality of at least three spoken forms of the second instance of the concept, wherein the first plurality of spoken forms differs from the second plurality of spoken forms; replacing the identified second text with a second context-free grammar specifying the second plurality of spoken forms of the second instance of the concept within the second document; generating a first language model, tangibly stored in a fifth non-transitory computer-readable medium, based on the second document; using the first language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a third document tangibly stored in a sixth non-transitory computer-readable medium; filtering text from the third document by reference to the second document to produce a filtered document, tangibly stored in a seventh non-transitory computer-readable medium, in which text filtered from the third document is marked as unreliable; and using the filtered document and the spoken audio stream to train an acoustic model, tangibly stored in an eighth non-transitory computer-readable medium, by; applying a first speech recognition process to the spoken audio stream using a set of base acoustic models and a grammar network based on the filtered document to produce a first set of recognition structures tangibly stored in a ninth non-transitory computer-readable medium; applying a second speech recognition process to the spoken audio stream using the set of base acoustic models and a second language model to produce a second set of recognition structures tangibly stored in a tenth non-transitory computer-readable medium; and performing discriminative training of the acoustic model using the first set of recognition structures, the second set of recognition structures, the filtered document, and only those portions of the spoken audio stream corresponding to text not marked as unreliable in the filtered document. - View Dependent Claims (8, 9, 10)
-
-
11. A method performed by at least one computer processor executing computer program instructions tangibly stored in a first computer-readable medium, the method comprising steps of:
-
(A) identifying a normalized document tangibly stored in a second computer-readable medium and containing at least some information in common with a spoken audio stream, the normalized document including;
(1) a first context-free grammar specifying a first plurality of at least three spoken forms of a first instance of a concept; and
(2) a second context-free grammar specifying a second plurality of at least three spoken forms of a second instance of the concept, wherein the first plurality of spoken forms differs from the second plurality of spoken forms;(B) identifying a language model, tangibly stored in a third computer-readable medium, based on the normalized document; (C) using the language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a second document tangibly stored in a fourth computer-readable medium; (D) filtering text from the second document by reference to the normalized document to produce a filtered document tangibly stored in a fifth computer-readable medium and in which text filtered from the second document is marked as unreliable; and (E) using the filtered document and the spoken audio stream to train an acoustic model, tangibly stored in a sixth computer-readable medium, by performing steps of; (E)(1) applying a first speech recognition process to the spoken audio stream using a set of base acoustic models and a grammar network based on the filtered document to produce a first set of recognition structures, tangibly stored in a seventh computer-readable medium; (E)(2) applying a second speech recognition process to the spoken audio stream using the set of base acoustic models and a second language model to produce a second set of recognition structures tangibly stored in an eighth computer-readable medium; and (E)(3) performing discriminative training of the acoustic model using the first set of recognition structures, the second set of recognition structures, the filtered document, and only those portions of the spoken audio stream corresponding to text not marked as unreliable in the filtered document. - View Dependent Claims (12, 13, 14)
-
-
15. A computer program product comprising computer-readable computer program instructions, tangibly stored in a first non-transitory computer-readable medium, said computer-readable program instructions adapted to be executed by a computer processor to perform a method comprising:
-
identifying a normalized document, tangibly stored in a second non-transitory computer-readable medium, containing at least some information in common with a spoken audio stream, the normalized document including;
(1) a first context-free grammar specifying a first plurality of at least three spoken forms of a first instance of a concept; and
(2) a second context-free grammar specifying a second plurality of at least three spoken forms of a second instance of the concept, wherein the first plurality of spoken forms differs from the second plurality of spoken forms;identifying a language model, tangibly stored in a third non-transitory computer-readable medium, based on the normalized document; using the language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a second document tangibly stored in a fourth non-transitory computer-readable medium; filtering text from the second document by reference to the normalized document to produce a filtered document tangibly stored in a fifth non-transitory computer-readable medium and in which text filtered from the second document is marked as unreliable; and using the filtered document and the spoken audio stream to train an acoustic model, tangibly stored in a sixth non-transitory computer-readable medium, comprising; applying a first speech recognition process to the spoken audio stream using a set of base acoustic models and a grammar network based on the filtered document to produce a first set of recognition structures, tangibly stored in a seventh non-transitory computer-readable medium; applying a second speech recognition process to the spoken audio stream using the set of base acoustic models and a second language model to produce a second set of recognition structures tangibly stored in an eighth non-transitory computer-readable medium; and performing discriminative training of the acoustic model using the first set of recognition structures, the second set of recognition structures, the filtered document, and only those portions of the spoken audio stream corresponding to text not marked as unreliable in the filtered document. - View Dependent Claims (16, 17, 18)
-
Specification