Natural error handling in speech recognition

• US 20040024601A1
• Filed: 07/31/2002
• Published: 02/05/2004
• Est. Priority Date: 07/31/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Gopinath, Ramesh A., Maison, Benoit, Wu, Brian C.
• Granted Patent: US 7,386,454 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

A user interface, and associated techniques, that permit a fast and efficient way of correcting speech recognition errors, or of diminishing their impact. The user may correct mistakes in a natural way, essentially by repeating the information that was incorrectly recognized previously. Such a mechanism closely approximates what human-to-human dialogue would be in similar circumstances. Such a system fully takes advantage of all the information provided by the user, and on its own estimates the quality of the recognition in order to determine the correct sequence of words in the fewest number of steps.

Claims

1. A method of handling errors in speech recognition, said method comprising the steps of:

accepting speech input from a user and recognizing the speech input;

providing feedback of the recognized speech input to the user; and

accepting corrective information from the user relating to the occurrence of one or more errors in the recognized speech input.

2. The method according to claim 1, wherein said step of accepting corrective information comprises permitting the user to signal the presence of one or more errors and to perform at least one of:

indicating at least one location of the one or more errors, and providing at least one correction to the one or more errors.

3. The method according to claim 2, wherein said step of accepting corrective information comprises permitting the user to signal the presence of one or more errors and to perform both of:

indicating at least one location of the one or more errors, and providing at least one correction to the one or more errors.

4. The method according to claim 2, wherein said step of accepting corrective information comprises permitting the use of context-sensitive natural correction commands.

5. The method according to claim 2, wherein said step of accepting corrective information comprises providing context-sensitive clarification dialogs.

6. The method according to claim 2, wherein said step of accepting corrective information comprises:

accepting from the user a plurality of repetitions of corrective information; and

using at least a portion of the repetitions of corrective information to arrive at a best hypothesis of corrective information.

7. The method according to claim 1, wherein said recognizing step comprises:

producing a list of hypotheses related to the speech input, each hypothesis comprising a sequence of at least one word; and

scoring each word, with higher scores being associated with words that have been more likely to have been correctly recognized.

8. The method according to claim 7, further comprising the step of transforming the score of each word to form a new list of hypotheses, whereby the transformed score of represents the estimated probability that the corresponding word is correct.

9. The method according to claim 8, further comprising the step of comparing words from the new list of hypotheses with a current best hypothesis and determining a new best hypothesis for each word.

10. The method according to claim 9, wherein said comparing step comprises comparing like words when possible.

11. The method according to claim 10, further comprising the steps of:

providing additional feedback of recognized speech input subsequent to said step of accepting corrective information and associating at least one confidence score with such speech input;

said step of providing additional feedback comprising the representation of recognized speech input having lower confidence scores in a first manner and the representation of recognized speech input having higher confidence scores in a second manner, the first manner involving a lower level of representation than the second manner.

12. An apparatus for handling errors in speech recognition, said apparatus comprising:

an input arrangement which accepts speech input from a user;

said input arrangement comprising a recognizer which recognizes the speech input;

a feedback arrangement which provides feedback of the recognized speech input to the user; and

a correction arrangement which accepts corrective information from the user relating to the occurrence of one or more errors in the recognized speech input.

13. The apparatus according to claim 12, wherein said correction arrangement is adapted to permit the user to signal the presence of one or more errors and to perform at least one of:

indicating at least one location of the one or more errors, and providing at least one correction to the one or more errors.

14. The apparatus according to claim 13, wherein said correction arrangement is adapted to permit the user to signal the presence of one or more errors and to perform both of:

indicating at least one location of the one or more errors, and providing at least one correction to the one or more errors.

15. The apparatus according to claim 13, wherein said correction arrangement is adapted to permit the use of context-sensitive natural correction commands.

16. The apparatus according to claim 13, wherein said correction arrangement is adapted to provide context-sensitive clarification dialogs.

17. The apparatus according to claim 13, wherein said correction arrangement is adapted to:

accept from the user a plurality of repetitions of corrective information; and

use at least a portion of the repetitions of corrective information to arrive at a best hypothesis of corrective information.

18. The apparatus according to claim 12, wherein said recognizer is adapted to:

produce a list of hypotheses related to the speech input, each hypothesis comprising a sequence of at least one word; and

score each word, with higher scores being associated with words that have been more likely to have been correctly recognized.

19. The apparatus according to claim 18, further comprising a transforming arrangement which transforms the score of each word to form a new list of hypotheses, whereby the transformed score of represents the estimated probability that the corresponding word is correct.

20. The apparatus according to claim 19, further comprising a comparing arrangement which compares words from the new list of hypotheses with a current best hypothesis and determines a new best hypothesis for each word.

21. The apparatus according to claim 20, wherein said comparing arrangement is adapted to compare like words when possible.

22. The apparatus according to claim 21, wherein said feedback arrangement is further adapted to:

provide additional feedback of recognized speech input subsequent to said step of accepting corrective information and associating at least one confidence score with such speech input;

wherein, in providing additional feedback, said feedback arrangement represents recognized speech input having lower confidence scores in a first manner and represents recognized speech input having higher confidence scores in a second manner, the first manner involving a lower level of representation than the second manner.

23. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for handling errors in speech recognition, said method comprising the steps of:

accepting speech input from a user and recognizing the speech input;

providing feedback of the recognized speech input to the user; and

accepting corrective information from the user relating to the occurrence of one or more errors in the recognized speech input.