SPEECH TRANSCRIPTION INCLUDING WRITTEN TEXT

• US 20140149119A1
• Filed: 03/14/2013
• Published: 05/29/2014
• Est. Priority Date: 11/28/2012
• Status: Active Grant

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Beaufays, Francoise, Sak, Hasim
• Granted Patent: US 9,594,744 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for transcribing utterances into written text are disclosed. The methods, systems, and apparatus include actions of obtaining a lexicon model that maps phones to spoken text and obtaining a language model that assigns probabilities to written text. Further includes generating a transducer that maps the written text to the spoken text, the transducer mapping multiple items of the written text to an item of the spoken text. Additionally, the actions include constructing a decoding network for transcribing utterances into written text, by composing the lexicon model, the inverse of the transducer, and the language model.

Claims

1. A computer-implemented method comprising:

obtaining a lexicon model that maps phones to spoken text;

obtaining a language model that assigns probabilities to written text;

generating a transducer that maps the written text to the spoken text, the transducer mapping multiple items of written text to an item of the spoken text; and

constructing a decoding network for transcribing utterances into written text, by composing the lexicon model, the inverse of the transducer, and the language model.

2. The method of claim 1, further comprising:

obtaining an utterance of a user; and

transcribing the utterance to an item of the written text using the decoding network.

3. The method of claim 1, wherein the spoken text comprises a textual representation of a spoken language.

4. The method of claim 1, wherein the written text comprises a textual representation of a written language.

5. The method of claim 1, wherein generating the transducer comprises:

obtaining grammar rules for expanding the written text into the spoken text;

generating expansions of the written text into spoken text using the grammar rules; and

adding the expansions to the transducer.

6. The method of claim 5, wherein obtaining the grammar rules comprises:

obtaining the grammar rules from a text-to-speech system.

7. The method of claim 1, wherein obtaining the language model comprises

obtaining spoken text of the language model;

generating written text from the spoken text of the language model using the inverse of the transducer; and

adding the generated written text to the language model.

8. The method of claim 1, where obtaining the lexicon model comprises

obtaining a context dependency model mapping context dependent phones to context independent phones;

obtaining a pronunciation lexicon mapping sequences of the context independent phones to the spoken text; and

constructing the lexicon model by composing the context dependency model and the pronunciation lexicon.

9. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

obtaining a lexicon model that maps phones to spoken text;

obtaining a language model that assigns probabilities to written text;

generating a transducer that maps the written text to the spoken text, the transducer mapping multiple items of written text to an item of the spoken text; and

constructing a decoding network for transcribing utterances into written text, by composing the lexicon model, the inverse of the transducer, and the language model.

10. The system of claim 9, wherein the operations further comprise:

obtaining an utterance of a user; and

transcribing the utterance to an item of the written text using the decoding network.

11. The system of claim 9, wherein the spoken text comprises a textual representation of a spoken language.

12. The system of claim 9, wherein the written text comprises a textual representation of a written language.

13. The system of claim 9, wherein generating the transducer comprises:

obtaining grammar rules for expanding the written text into the spoken text;

generating expansions of the written text into spoken text using the grammar rules; and

adding the expansions to the transducer.

14. The system of claim 13, wherein obtaining the grammar rules comprises:

obtaining the grammar rules from a text-to-speech system.

15. The system of claim 9, wherein obtaining the language model comprises

obtaining spoken text of the language model;

generating written text from the spoken text of the language model using the inverse of the transducer; and

adding the generated written text to the language model.

16. The system of claim 9, where obtaining the lexicon model comprises

obtaining a context dependency model mapping context dependent phones to context independent phones;

obtaining a pronunciation lexicon mapping sequences of the context independent phones to the spoken text; and

constructing the lexicon model by composing the context dependency model and the pronunciation lexicon.

17. A computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining a lexicon model that maps phones to spoken text;

obtaining a language model that assigns probabilities to written text;

generating a transducer that maps the written text to the spoken text, the transducer mapping multiple items of written text to an item of the spoken text; and

constructing a decoding network, for transcribing utterances into written text, by composing the lexicon model, the inverse of the transducer, and the language model.

18. The system of claim 17, wherein the operations further comprise:

obtaining an utterance of a user; and

transcribing the utterance to an item of the written text using the decoding network.

19. The system of claim 17, wherein the spoken text comprises a textual representation of a spoken language.

20. The system of claim 17, wherein the written text comprises a textual representation of a written language.