System and method for optimizing speech recognition and natural language parameters with user feedback

• US 9,984,679 B2
• Filed: 07/18/2016
• Issued: 05/29/2018
• Est. Priority Date: 05/09/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ljolje, Andrej, Caseiro, Diamantino Antonio, Gilbert, Mazin, Goffin, Vincent, Mishra, Taniya
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Chavez, Rodrigo
• Application Number:
  US15/212,908
• Publication Number:
  US 20160329045A1
• Time in Patent Office: 680 Days
• Field of Search: 704235
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for assigning saliency weights to words of an ASR model. The saliency values assigned to words within an ASR model are based on human perception judgments of previous transcripts. These saliency values are applied as weights to modify an ASR model such that the results of the weighted ASR model in converting a spoken document to a transcript provide a more accurate and useful transcription to the user.

Claims

1. A method comprising:

weighting a first automatic speech recognition model, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model, to yield a weighted second automatic speech recognition model;

converting, via a processor, speech to text using the weighted first automatic speech recognition model, to yield a first transcript;

converting, via the processor, the speech to text using the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of one of the first transcript and the second transcript; and

updating, via the processor, one of the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

2. The method of claim 1, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a context of the speech.

3. The method of claim 2, wherein the judgment is received for the first transcript of the second transcript with a highest score.

4. The method of claim 1, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a user profile.

5. The method of claim 4, wherein the user profile comprises a list of contexts.

6. The method of claim 4, wherein the user profile comprises a previous communication history.

7. The method of claim 1, wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model each contain saliency weights to words in the speech.

8. The method of claim 7, wherein a high saliency weight indicates a high predicted importance to the user.

9. The method of claim 8, wherein the processor spends additional effort converting high saliency text.

10. A system comprising:

a processor; and

a computer-readable storage device having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

weighting a first automatic speech recognition model, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model, to yield a weighted second automatic speech recognition model;

converting speech to text using the weighted first automatic speech recognition model, to yield a first transcript;

converting the speech to text using the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of one of the first transcript and the second transcript; and

updating one of the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

11. The system of claim 10, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a context of the speech.

12. The system of claim 11, wherein the judgment is received for the first transcript or the second transcript with a highest score.

13. The system of claim 10, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a user profile.

14. The system of claim 13, wherein the user profile comprises a list of contexts.

15. The system of claim 13, wherein the user profile comprises a previous communication history.

16. The system of claim 10, wherein the weighted first automatic speech recognition model and the weighted second automatic speech recognition model each contain saliency weights to words in the speech.

17. The system of claim 16, wherein a high saliency weight indicates a high predicted importance to the user.

18. The system of claim 17, wherein the processor spends additional effort converting high saliency words.

19. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

weighting a first automatic speech recognition model, to yield a weighted first automatic speech recognition model;

weighting a second automatic speech recognition model, to yield a weighted second automatic speech recognition model;

converting speech to text using the weighted first automatic speech recognition model, to yield a first transcript;

converting the speech to text using the weighted second automatic speech recognition model, to yield a second transcript;

receiving, from a user, a judgment of perceived accuracy of one of the first transcript and the second transcript; and

updating one of the weighted first automatic speech recognition model and the weighted second automatic speech recognition model based on the judgment.

20. The computer-readable storage device of claim 19, wherein the weighting of the first automatic speech recognition model and the weighting of the second automatic speech recognition model is based on a context of the speech.