System and method for optimizing speech recognition and natural language parameters with user feedback

• US 8,738,375 B2
• Filed: 05/09/2011
• Issued: 05/27/2014
• Est. Priority Date: 05/09/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Ljolje, Andrej, CASEIRO, Diamantino Antonio, Gilbert, Mazin, Goffin, Vincent, MISHRA, Taniya
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US13/103,665
• Publication Number:
  US 20120290298A1
• Time in Patent Office: 1,114 Days
• Field of Search: 704/9, 704/7, 704/251, 704/252, 704/243, 704/257, 704/270, 704/231, 704/244, 704/235, 704/250, 704/255
• US Class Current: 704/235
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for assigning saliency weights to words of an ASR model. The saliency values assigned to words within an ASR model are based on human perception judgments of previous transcripts. These saliency values are applied as weights to modify an ASR model such that the results of the weighted ASR model in converting a spoken document to a transcript provide a more accurate and useful transcription to the user.

Claims

1. A method comprising:

receiving, from a sender, a speech document;

capturing, via a processor, a context of the speech document;

weighting a first automatic speech recognition model based on the context of the speech document, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model based on the context of the speech document, to yield a weighted second automatic speech recognition model;

converting, via the processor, the speech document to text using the weighted first automatic speech recognition model, to yield a first transcript;

converting, via the processor, the speech document to text using the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of the first transcript and the second transcript; and

updating, via the processor, the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

2. The method of claim 1, wherein the context of the speech document comprises one of a name of the sender, a location of the sender, a time sent, and a subject.

3. The method of claim 1, wherein weighting of the first automatic speech recognition model and weighting of the second automatic speech recognition model is further based on a user profile.

4. The method of claim 3, wherein the user profile comprises one of a previous communication history and a list of contexts.

5. The method of claim 4, wherein each context in the list of contexts comprises an importance ranking.

6. The method of claim 3, wherein the transcripts receive a score based on predicted errors in conversion, the user profile, and the context of the speech document.

7. The method of claim 1, wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model assign a saliency weight to a set of words, based on a frequency of the set of words, the context of the speech document, and a user profile.

8. The method of claim 7, wherein a high saliency weight indicates high predicted importance to the user.

9. The method of claim 7, wherein converting the speech document to text is based on the saliency weight of a portion of the text.

10. The method of claim 9, wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model direct the processor to spend more effort converting to text high saliency text.

11. A system, comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving, from a sender, a speech document;

capturing a context of the speech document;

weighting a first automatic speech recognition model based on the context of the speech document and, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model based on the context of the speech document, to yield a weighted second automatic speech recognition model;

converting the speech document to text by applying the weighted first automatic speech recognition model, to yield a first transcript;

converting the speech document to text by applying the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of the first transcript and the second transcript; and

updating the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

12. The system of claim 11, wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model assign a saliency weight to a set of words, based on a frequency of the set of words, the context of the speech document, and a user profile.

13. The system of claim 12, wherein a high frequency of the set of words yields a low saliency weight and a low frequency of the set of words yields a high saliency weight.

14. The system of claim 11, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is further based on a likelihood that a set of words was erroneously recognized.

15. The system of claim 14, wherein the likelihood that the set of words was erroneously recognized is determined based on a word insertion error rate, a word deletion error rate, and a word substitution error rate.

16. The system of claim 11, wherein the judgment is received from one of a keyboard, a vocal response, a pointing device, and a touch screen.

17. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving from a sender a speech document;

capturing a context of a speech document;

weighting a first automatic speech recognition model based on the context of the speech document, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model based on the context of the speech document, to yield a weighted second automatic speech recognition model;

converting the speech document to text using the weighted first automatic speech recognition model, to yield a first transcript;

converting the speech document to text using the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of the first transcript and the second transcript; and

updating, via the processor, the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

18. The computer-readable storage device of claim 17, wherein the weighting of the first automatic speech model and the weighting of the second automatic speech model is further based on a user profile, the frequency within the speech document of a set of words, and a geographical location associated with the speech document.

19. The computer-readable storage device of claim 17 having additional instructions stored which result in the operations further comprising:

storing the judgment in a database, yielding a stored judgment;

updating the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the stored judgment.

20. The computer-readable storage device of claim 19 having additional instructions stored which result in the operations further comprising:

providing the stored judgment to a manufacturer of the first automatic speech recognition model and the second automatic speech recognition model.