Translating Between Spoken and Written Language

• US 20120290299A1
• Filed: 05/13/2011
• Published: 11/15/2012
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Basson, Sara H., Hamilton, Rick, Jiang, Dan Ning, Kanevsky, Dimitri, Nahamoo, David, Picheny, Michael, Ramabhadran, Bhuvana, Sainath, Tara N.
• Granted Patent: US 8,856,004 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Techniques for converting spoken speech into written speech are provided. The techniques include transcribing input speech via speech recognition, mapping each spoken utterance from input speech into a corresponding formal utterance, and mapping each formal utterance into a stylistically formatted written utterance.

Claims

1. A method for converting spoken speech into written speech, wherein the method comprises:

transcribing input speech via speech recognition;

mapping each spoken utterance from input speech into a corresponding formal utterance; and

mapping each formal utterance into a stylistically formatted written utterance.

2. The method of claim 1, wherein mapping each spoken utterance from input speech into a corresponding formal utterance comprises generating a set of alternative and equivalent formal representations of each spoken utterance.

3. The method of claim 1, further comprising editing any speech recognition error to create a modified transcript of speech input.

4. The method of claim 3, wherein creating a modified transcript comprises modifying the transcript to more closely align the transcript with written grammar.

5. The method of claim 3, wherein creating a modified transcript comprises modifying the transcript to more closely align with a preferred language style.

6. The method of claim 1, further comprising using a statistical equation to train a spoken to written speech translator.

7. The method of claim 6, wherein a statistical equation comprises Prob_theta(F|A)=Prob_theta(F|T) Prob(T|A)/Prob(A,F|T), where F is written speech, T is spoken speech, A is speech spoken by an individual.

8. The method of claim 1, further comprising representing one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

9. The method of claim 1, further comprising estimating one or more parameters with an expectation maximization algorithm that optimizes one or more functions via association with an auxiliary function.

10. The method of claim 9, wherein a function for estimation of one or more parameters comprises Prob_theta(F|A)=f(theta)=Prob_theta(F|T) Prob_theta(T|A)/Prob(A,F|T).

11. The method of claim 10, further comprising associating the function Prob_theta(F|T) with a summation of all possible choices of associating a written utterance with one or more spoken word utterances.

12. The method of claim 11, wherein each alignment of a spoken word utterance with one or more possible formal written utterances has a likelihood measure, and wherein total probability Prob_theta(F|T) is a summation of all likelihoods of alignments.

13. The method of claim 1, further comprising providing a system, wherein the system comprises one or more distinct software modules, each of the one or more distinct software modules being embodied on a tangible computer-readable recordable storage medium, and wherein the one or more distinct software modules comprise a speech recognition module, a formal mapping module and a stylistic formatting module executing on a hardware processor.

14. A translation system comprising:

a first module comprising a database of spoken speech categories (SSC) representing units of basic spoken speech characteristics;

a second module comprising a database of written speech transformations (WST); and

a third module comprising a training translation module that creates a spoken speech category and written speech transformation from two parallel corpora of spoken and written speech via alignment and semantic identification processes of equivalent pieces of text.

15. The system of claim 14, wherein the first module includes one or more sample utterances, wherein the one or more sample utterances comprise examples of at least one of repetitions, informal expressions, omissions, incorrect grammar order, run-on sentences, and undesirable vocabulary choices.

16. The system of claim 14, wherein the second module includes a list of one or more rules for one or more operations performed on a spoken speech category to make spoken speech more similar to a written language style.

17. The system of claim 14, wherein the two parallel corpora are created manually by at least one of having a user read a written corpus and re-phrase it orally and having a user speak a spoken corpus and write down text that corresponds to what was said.

18. The system of claim 14, wherein the third module matches one or more terms in the spoken and written corpora, aligns the matched corpora with each other using one or more coinciding word markers and creates a database that indicates how one or more short phrases in one corpus map with one or more short phrases in another corpus.

19. The system of claim 18, further comprising a machine learning system for mapping phrases in one corpus that are aligned with phrases to another corpus automatically using statistical inference.

20. A computer program product comprising a tangible computer readable recordable storage medium including computer useable program code for converting spoken speech into written speech, the computer program product including:

computer useable program code for transcribing input speech via speech recognition;

computer useable program code for mapping each spoken utterance from input speech into a corresponding formal utterance; and

computer useable program code for mapping each formal utterance into a stylistically formatted written utterance.

21. The computer program product of claim 20, wherein the computer useable program code for mapping each spoken utterance from input speech into a corresponding formal utterance comprises computer useable program code for generating a set of alternative and equivalent formal representations of each spoken utterance.

22. The computer program product of claim 20, further comprising computer useable program code for representing one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

23. A system for converting spoken speech into written speech, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

transcribe input speech via speech recognition;

map each spoken utterance from input speech into a corresponding formal utterance; and

map each formal utterance into a stylistically formatted written utterance.

24. The system of claim 23, wherein the at least one processor coupled to the memory operative to map each spoken utterance from input speech into a corresponding formal utterance is further operative to generate a set of alternative and equivalent formal representations of each spoken utterance.

25. The system of claim 23, wherein the at least one processor coupled to the memory is further operative to represent one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.