Detecting potential significant errors in speech recognition results

• US 9,818,398 B2
• Filed: 05/15/2015
• Issued: 11/14/2017
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ganong, III, William F., Vemula, Raghu, Fleming, Robert
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/713,372
• Publication Number:
  US 20150248882A1
• Time in Patent Office: 914 Days
• Field of Search: 704239, 704275
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2015/085 Methods for reducing search complexity, pruning

Abstract

In some embodiments, recognition results produced by a speech processing system (which may include two or more recognition results, including a top recognition result and one or more alternative recognition results) based on an analysis of a speech input, are evaluated for indications of potential errors. In some embodiments, the indications of potential errors may include discrepancies between recognition results that are meaningful for a domain, such as medically-meaningful discrepancies. The evaluation of the recognition results may be carried out using any suitable criteria, including one or more criteria that differ from criteria used by an ASR system in determining the top recognition result and the alternative recognition results from the speech input. In some embodiments, a recognition result may additionally or alternatively be processed to determine whether the recognition result includes a word or phrase that is unlikely to appear in a domain to which speech input relates.

Claims

1. A method comprising:

evaluating two or more results of a recognition, by an automatic speech recognition (ASR) system on a speech input, using at least one criterion that differs from criteria used by the ASR system in determining the two or more results, wherein the two or more results were identified by the ASR system as likely to be accurate recognition results for the speech input and comprise a first recognition result identified by the ASR system as most likely to be a correct recognition result for the speech input and at least one alternative recognition result identified by the ASR system as a potential recognition result for the speech input; and

in response to determining that the at least one criterion is met by the two or more results, triggering presentation, via a user interface, of an alert concerning one of the two or more results.

2. The method of claim 1, wherein the evaluating the two or more results using the at least one criterion comprises evaluating the two or more results for medically-meaningful discrepancies between the two or more results.

3. The method of claim 1, wherein the evaluating and triggering are performed by an entity other than the ASR system.

4. The method of claim 1, wherein:

the evaluating the two or more results using the at least one criterion comprises determining whether the two or more results comprise an indication of a potential error in the first recognition result that may cause a meaning of the first recognition result to differ from a meaning of the speech input.

5. The method of claim 4, wherein:

the at least one alternative recognition result comprises a second recognition result;

the method further comprises semantically interpreting each of the first recognition result and the second recognition result to determine at least one first fact expressed in the first recognition result and at least one second fact expressed in the second recognition result; and

the determining whether the two or more results comprise an indication of a potential error that may cause the meaning of the first recognition result to differ from the meaning of the speech input comprises determining whether there is a difference between the at least one first fact and the at least one second fact.

6. The method of claim 4, wherein determining whether the two or more results comprise an indication of a potential error that may cause the meaning of the first recognition result to differ from a meaning of the speech input comprises determining whether the first recognition result includes a first member of a set of words or phrases, each member of the set comprising a word or phrase and being associated with at least one other member of the set, and determining whether the second recognition result includes at least one other member associated with the first member of the set.

7. The method of claim 6, wherein:

the first member of the set of words or phrases is associated with a second member of the set with which the first member is acoustically confusable and that, when substituted for the first member in a recognition result, changes a medical meaning of the recognition result; and

the determining whether the second recognition result includes the at least one other member associated with the first member of the set comprises determining whether the second recognition result includes the second member of the set.

8. The method of claim 1, wherein:

the method further comprises selecting, from the plurality of results, the two or more results to be evaluated using the at least one criterion, the two or more results being fewer than all of the plurality of results.

9. The method of claim 1, wherein the triggering an alert concerning one of the two or more results comprises presenting a visual and/or audible message.

10. The method of claim 1, wherein the evaluating the two or more results identified by the ASR system using the at least one criterion comprises evaluating an N best list of recognition results identified by the ASR system without identifying a new order of recognition results in the N best list.

11. The method of claim 1, further comprising:

evaluating prosody information and/or information indicating the presence of one or more hesitation vocalizations, produced by the ASR system based on the speech input, to determine whether a speaker exhibited one or more signs of uncertainty when providing the speech input; and

in response to determining that the speaker exhibited one or more signs of uncertainty, triggering an alert concerning the two or more results.

12. At least one non-transitory computer-readable storage medium having encoded thereon computer-executable instructions that, when executed by at least one computer, cause the at least one computer to carry out a method comprising:

evaluating two or more results of a recognition, by an automatic speech recognition (ASR) system on a speech input, using at least one criterion that differs from criteria used by the ASR system in determining the two or more results, wherein the two or more results were identified by the ASR system as likely to be accurate recognition results for the speech input and comprise a first recognition result identified by the ASR system as most likely to be a correct recognition result for the speech input and at least one alternative recognition result identified by the ASR system as a potential recognition result for the speech input; and

in response to determining that the at least one criterion is met by the two or more results, triggering presentation, via a user interface, of an alert concerning one of the two or more results.

13. The at least one computer-readable storage medium of claim 12, wherein:

the evaluating the two or more results using the at least one criterion comprises determining whether the two or more results comprise an indication of a potential error in the first recognition result that may cause a meaning of the first recognition result to differ from a meaning of the speech input.

14. The at least one computer-readable storage medium of claim 13, wherein:

the at least one alternative recognition result comprises a second recognition result;

the method further comprises semantically interpreting each of the first recognition result and the second recognition result to determine at least one first fact expressed in the first recognition result and at least one second fact expressed in the second recognition result; and

the determining whether the two or more results comprise an indication of a potential error that may cause the meaning of the first recognition result to differ from the meaning of the speech input comprises determining whether there is a difference between the at least one first fact and the at least one second fact.

15. An apparatus comprising:

at least one processor; and

at least one storage medium having encoded thereon executable instructions that, when executed by at least one processor, cause the at least one processor to carry out a method, the method comprising;

evaluating two or more results of a recognition, by an automatic speech recognition (ASR) system on a speech input, using at least one criterion that differs from criteria used by the ASR system in determining the two or more results, wherein the two or more results were identified by the ASR system as likely to be accurate recognition results for the speech input and comprise a first recognition result identified by the ASR system as most likely to be a correct recognition result for the speech input and at least one alternative recognition result identified by the ASR system as a potential recognition result for the speech input; and

in response to determining that the at least one criterion is met by the two or more results, triggering presentation, via a user interface, of an alert concerning one of the two or more results.

16. The apparatus of claim 15, wherein the evaluating the two or more results using the at least one criterion comprises evaluating the two or more results for medically-meaningful discrepancies between the two or more results.

17. The apparatus of claim 15, wherein:

the evaluating the two or more results using the at least one criterion comprises determining whether the two or more results comprise an indication of a potential error in the first recognition result that may cause a meaning of the first recognition result to differ from a meaning of the speech input.

18. The apparatus of claim 17, wherein:

the at least one alternative recognition result comprises a second recognition result;

the method further comprises semantically interpreting each of the first recognition result and the second recognition result to determine at least one first fact expressed in the first recognition result and at least one second fact expressed in the second recognition result; and

the determining whether the two or more results comprise an indication of a potential error that may cause the meaning of the first recognition result to differ from the meaning of the speech input comprises determining whether there is a difference between the at least one first fact and the at least one second fact.

19. The apparatus of claim 17, wherein:

determining whether the two or more results comprise an indication of a potential error that may cause the meaning of the first recognition result to differ from a meaning of the speech input comprises determining whether the first recognition result includes a first member of a set of words or phrases, each member of the set comprising a word or phrase and being associated with at least one other member of the set, and determining whether the second recognition result includes at least one other member associated with the first member of the set.

20. The apparatus of claim 19, wherein:

the first member of the set of words or phrases is associated with a second member of the set with which the first member is acoustically confusable and that, when substituted for the first member in a recognition result, changes a medical meaning of the recognition result; and

the determining whether the second recognition result includes the at least one other member associated with the first member of the set comprises determining whether the second recognition result includes the second member of the set.