Text processing using natural language understanding

• US 8,856,004 B2
• Filed: 05/13/2011
• Issued: 10/07/2014
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Basson, Sara H., Hamilton, Rick, Jiang, Dan Ning, Kanevsky, Dimitri, Nahamoo, David, Picheny, Michael, Ramabhadran, Bhuvana, Sainath, Tara N.
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US13/107,001
• Publication Number:
  US 20120290299A1
• Time in Patent Office: 1,243 Days
• Field of Search: 704/235, 704/240, 704/251, 704/255, 704/257, 704 1- 10, 704/270, 704/270.1, 704/278
• US Class Current: 704/235
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Techniques for converting spoken speech into written speech are provided. The techniques include transcribing input speech via speech recognition, mapping each spoken utterance from input speech into a corresponding formal utterance, and mapping each formal utterance into a stylistically formatted written utterance.

Claims

1. A method for converting spoken speech into written speech, wherein the method comprises:

using at least one processor to transcribe input speech via speech recognition;

using the at least one processor to map each spoken utterance from input speech into a corresponding formal utterance;

using the at least one processor to map each formal utterance into a stylistically formatted written utterance; and

using the at least one processor to train a spoken to written speech translator using a statistical equation, wherein the statistical equation comprises Prob_theta(F|A)=Prob_theta(F|T) Prob(T|A/Prob(A,F|T), where F is written speech, T is spoken speech, A is speech spoken by an individual.

2. The method of claim 1, wherein mapping each spoken utterance from input speech into a corresponding formal utterance comprises generating a set of alternative and equivalent formal representations of each spoken utterance.

3. The method of claim 1, further comprising editing any speech recognition error to create a modified transcript of speech input.

4. The method of claim 3, wherein creating a modified transcript comprises modifying the transcript to more closely align the transcript with written grammar.

5. The method of claim 3, wherein creating a modified transcript comprises modifying the transcript to more closely align with a preferred language style.

6. The method of claim 1, further comprising representing one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

7. The method of claim 1, further comprising estimating one or more parameters with an expectation maximization algorithm that optimizes one or more functions via association with an auxiliary function.

8. The method of claim 7, wherein a function for estimation of one or more parameters comprises Prob_theta(F|A)=f(theta)=Prob_theta(F|T) Prob_theta(T|A)/Prob (A,F|T), where F is written speech, T is spoken speech, and A is speech spoken by an individual.

9. The method of claim 8, further comprising associating the function Prob_theta(F|T) with a summation of all possible choices of associating a written utterance with one or more spoken word utterances.

10. The method of claim 9, wherein each alignment of a spoken word utterance with one or more possible formal written utterances has a likelihood measure, and wherein total probability Prob_theta(F|T) is a summation of all likelihoods of alignments.

11. The method of claim 1, further comprising providing a system, wherein the system comprises one or more distinct software modules, each of the one or more distinct software modules being embodied on a tangible computer recordable storage device, and wherein the one or more distinct software modules comprise a speech recognition module, a formal mapping module and a stylistic formatting module executing on a hardware processor.

12. A computer program product comprising a tangible computer recordable storage device including computer useable program code for converting spoken speech into written speech, the computer program product including:

computer useable program code for transcribing input speech via speech recognition;

computer useable program code for mapping each spoken utterance from input speech into a corresponding formal utterance;

computer useable program code for mapping each formal utterance into a stylistically formatted written utterance; and

computer useable program code for representing one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

13. The computer program product of claim 12, wherein the computer useable program code for mapping each spoken utterance from input speech into a corresponding formal utterance comprises computer useable program code for generating a set of alternative and equivalent formal representations of each spoken utterance.

14. A system for converting spoken speech into written speech, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

transcribe input speech via speech recognition;

map each spoken utterance from input speech into a corresponding formal utterance;

map each formal utterance into a stylistically formatted written utterance; and

represent one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

15. The system of claim 14, wherein the at least one processor coupled to the memory operative to map each spoken utterance from input speech into a corresponding formal utterance is further operative to generate a set of alternative and equivalent formal representations of each spoken utterance.

16. A method for converting spoken speech into written speech, wherein the method comprises:

using at least one processor to transcribe input speech via speech recognition;

using the at least one processor to map each spoken utterance from input speech into a corresponding formal utterance;

using the at least one processor to map each formal utterance into a stylistically formatted written utterance; and

using the at least one processor to estimate one or more parameters with an expectation maximization algorithm that optimizes one or more functions via association with an auxiliary function, wherein a function for estimation of one or more parameters comprises Prob_theta(F|A)=f (theta)=Prob_theta(F|T) Prob_theta(T|A)/Prob(A,F|T), where F is written speech, T is spoken speech, and A is speech spoken by an individual.

17. The method of claim 16, wherein mapping each spoken utterance from input speech into a corresponding formal utterance comprises generating a set of alternative and equivalent formal representations of each spoken utterance.

18. The method of claim 16, further comprising editing any speech recognition error to create a modified transcript of speech input.

19. The method of claim 18, wherein creating a modified transcript comprises modifying the transcript to more closely align the transcript with written grammar.

20. The method of claim 18, wherein creating a modified transcript comprises modifying the transcript to more closely align with a preferred language style.

21. The method of claim 16, further comprising using a statistical equation to train a spoken to written speech translator.

22. The method of claim 21, wherein the statistical equation comprises Prob_theta(F|A)=Prob_theta(F|T) Prob(T|A)/Prob(A,F|T), where F is written speech, T is spoken speech, A is speech spoken by an individual.

23. The method of claim 16, further comprising representing one or more metrics of proximity from one language style to another, using an entropy-based measure that incorporates understandability over a specified number of words.

24. The method of claim 16, further comprising associating the function Prob_theta(F|T) with a summation of all possible choices of associating a written utterance with one or more spoken word utterances.

25. The method of claim 24, wherein each alignment of a spoken word utterance with one or more possible formal written utterances has a likelihood measure, and wherein total probability Prob_theta(F|T) is a summation of all likelihoods of alignments.