Methods and apparatus for correcting recognition errors

• US 10,522,133 B2
• Filed: 05/23/2012
• Issued: 12/31/2019
• Est. Priority Date: 05/23/2011
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Labsky, Martin, Kleindienst, Jan, Macek, Tomas, Nahamoo, David, Curin, Jan, Ganong, III, William F.
• Primary Examiner(s): Paula, Cesar B
• Assistant Examiner(s): Nguyen, Luu-Phuong T
• Application Number:
  US13/479,010
• Publication Number:
  US 20120304057A1
• Time in Patent Office: 2,778 Days
• Field of Search: 715256
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

Techniques for error correction using a history list comprising at least one misrecognition and correction information associated with each of the at least one misrecognitions indicating how a user corrected the associated misrecognition. The techniques include converting data input from a user to generate a text segment, determining whether at least a portion of the text segment appears in the history list as one of the at least one misrecognitions, if the at least a portion of the text segment appears in the history list as one of the at least one misrecognitions, obtaining the correction information associated with the at least one misrecognition, and correcting the at least a portion of the text segment based, at least in part, on the correction information.

Claims

1. A method of correcting speech recognition errors, the method comprising:

converting, using at least one automatic speech recognition engine, speech data input by a first user to generate a text segment;

determining whether at least a portion of the text segment appears as a misrecognition in a history list, the history list comprising;

a plurality of misrecognitions that resulted from errors in automated speech recognition previously performed on speech input received from the first user; and

correction information associated with each of the plurality of misrecognitions indicating how the first user corrected the associated misrecognition;

when the at least a portion of the text segment appears in the history list as one of the plurality of misrecognitions,

obtaining, from the history list, the correction information associated with the one misrecognition;

determining whether to automatically correct the text segment based on the correction information obtained from the history list;

when it is determined not to automatically correct the text segment based on the correction information obtained from the history list, outputting the text segment without correction for presentation to the first user; and

when it is determined to automatically correct the text segment based on the correction information obtained from the history list, automatically correcting the at least a portion of the text segment using the correction information obtained from the history list;

when it is determined that the first user edited the text segment to produce corrected text,

determining whether to insert the text segment and the corrected text into the history list at least in part by determining whether the first user is replacing the text segment which was correctly recognized as presented; and

inserting the text segment and the corrected text to the history list when it is determined that the text segment and the corrected text is to be inserted into the history list.

2. The method of claim 1, wherein when it is determined to automatically correct the text segment based on the correction information obtained from the history list, presenting the text segment with the corrected at least a portion of the text segment to the first user.

3. The method of claim 1, wherein outputting the text segment without correction for presentation to the first user comprises outputting the text segment to the first user along with the correction information as a suggested correction for the at least a portion of the text segment.

4. The method of claim 1, wherein determining whether to insert the text segment and the corrected text into the history list is performed using a confidence value generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.

5. The method of claim 1, wherein determining whether to insert the text segment and the corrected text into the history list comprises determining whether the corrected text is in an N-best list generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.

6. The method of claim 1, wherein the history list comprises a plurality of previous recognized text segments, and wherein at least one of the plurality of previously recognized text segments includes an expiration after which the at least one of the plurality of previously recognized text segments is removed from the history list, the expiration being associated with at least one duration and/or at least one event.

7. The method of claim 1, wherein determining whether to insert the text segment and the corrected text into the history list comprises determining whether the first user is correcting the misrecognition in the text segment or is replacing the text segment which was correctly recognized as presented.

8. The method of claim 7, wherein determining whether the first user is correcting the misrecognition in the text segment or is replacing the text segment which was correctly recognized as presented is based, at least in part, on:

at least one alternative recognition result for the speech data, a confidence score for the text segment generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment, or both the at least one alternative recognition result and the confidence score.

9. The method of claim 7, further comprising:

in response to a determination that the first user is replacing the text segment which was correctly recognized as presented, refraining from inserting the text segment and the corrected text to the history list.

10. The method of claim 7, further comprising:

in response to a determination that the first user is correcting the misrecognition in the text segment, inserting the text segment and the corrected text to the history list.

11. The method of claim 1, wherein when it is determined not to automatically correct the text segment based on the correction information obtained from the history list, alerting the first user to possible misrecognition.

12. The method of claim 11, wherein alerting the first user to possible misrecognition comprises highlighting the text segment to the first user.

13. At least one non-transitory computer readable medium storing instructions that, when executed on at least one processor, perform a method of correcting speech recognition errors, the method comprising:

converting, using at least one automatic speech recognition engine, speech data input by a first user to generate a text segment;

determining whether at least a portion of the text segment appears as a misrecognition in a history list, the history list comprising;

a plurality of misrecognitions that resulted from errors in automated speech recognition previously performed on speech input received from the first user; and

correction information associated with each of the plurality of misrecognitions indicating how the first user corrected the associated misrecognition;

when the at least a portion of the text segment appears in the history list as one of the plurality of misrecognitions,

obtaining, from the history list, the correction information associated with the one misrecognition;

determining whether to automatically correct the text segment based on the correction information obtained from the history list;

when it is determined not to automatically correct the text segment based on the correction information obtained from the history list, outputting the text segment without correction for presentation to the first user; and

when it is determined to automatically correct the text segment based on the correction information obtained from the history list, automatically correcting the at least a portion of the text segment using the correction information obtained from the history list;

when it is determined that the first user edited the text segment to produce corrected text,

determining whether to insert the text segment and the corrected text into the history list at least in part by determining whether the first user is replacing the text segment which was correctly recognized as presented; and

inserting the text segment and the corrected text to the history list when it is determined that the text segment and the corrected text is to be inserted into the history list.

14. The at least one non-transitory computer readable medium of claim 13, wherein when it is determined to automatically correct the text segment based on the correction information obtained from the history list, presenting the text segment with the corrected at least a portion of the text segment to the first user.

15. The at least one non-transitory computer readable medium of claim 13, wherein outputting the text segment without correction for presentation to the first user comprises outputting the text segment to the first user along with the correction information as a suggested correction for the at least a portion of the text segment.

16. The at least one non-transitory computer readable medium of claim 13, wherein determining whether to insert the text segment and the corrected text into the history list is performed using a confidence value generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.

17. The at least one non-transitory computer readable medium of claim 13, wherein determining whether to insert the text segment and the corrected text into the history list comprises determining whether the corrected text is in an N-best list generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.

18. A system for correcting speech recognition errors, the system comprising:

at least one input interface for receiving speech data input by a first user;

at least one automatic speech recognition engine configured to convert the speech data input by the first user into a textual representation including at least one text segment; and

at least one processor configured to;

determine whether at least a portion of the text segment appears as a misrecognition in a history list, the history list comprising;

a plurality of misrecognitions that resulted from errors in automated speech recognition previously performed on speech input received from the first user; and

correction information associated with each of the plurality of misrecognitions indicating how the first user corrected the associated misrecognition,

when the at least a portion of the text segment appears in the history list as one of the plurality of misrecognitions,

obtain, from the history list, the correction information associated with the one misrecognition;

determine whether to automatically correct the text segment based on the correction information obtained from the history list;

when it is determined not to automatically correct the text segment based on the correction information obtained from the history list, output the text segment without correction for presentation to the first user; and

when it is determined to automatically correct the text segment based on the correction information obtained from the history list, automatically correct the at least a portion of the text segment using the correction information obtained from the history list;

when it is determined that the first user edited the text segment to produce corrected text,

determine whether to insert the text segment and the corrected text into the history list at least in part by determining whether the first user is replacing the text segment which was correctly recognized as presented; and

insert the text segment and the corrected text to the history list when it is determined that the text segment and the corrected text is to be inserted into the history list.

19. The system of claim 18, wherein when it is determined to automatically correct the text segment based on the correction information obtained from the history list, presenting the text segment with the corrected at least a portion of the text segment to the first user.

20. The system of claim 18, wherein a correction component operates with and/or is part of a presentation component configured to provide a presentation of the textual representation.

21. The system of claim 18, wherein outputting the text segment without correction for presentation to the first user comprises outputting the text segment along with the correction information as a suggested correction for the at least a portion of the text segment.

22. The system of claim 18, wherein determining whether to insert the text segment and the corrected text into the history list is performed using a confidence value generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.

23. The system of claim 18, wherein determining whether to insert the text segment and the corrected text into the history list comprises determining whether the corrected text is in an N-best list generated by the at least one automatic speech recognition engine when converting the speech data input by the first user to generate the text segment.