Method and system for preventing error amplification in natural language dialogues

• US 20040098253A1
• Filed: 10/14/2003
• Published: 05/20/2004
• Est. Priority Date: 11/30/2000
• Status: Active Grant

• Current Assignee: Shadow Prompt Technology AG
• Original Assignee: Enterprise Integration Group, Inc. (Perspecta, Inc.)
• Inventors: Stringham, Rex, Balentine, Bruce, Munroe, Justin, Melaragno, Ralph
• Granted Patent: US 7,194,409 B2
• Time in Patent Office: Days
• US Class Current: 704/215
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00

Abstract

A method and system for allowing a user to interface to an interactive voice response system via natural language commands. The method and apparatus transitions users to a known state upon poor recognition results, transitions users to a lateral state upon entry of a command corresponding to a lateral state, and provides intermediate barge-in states to account for user speech patterns

Claims

1. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

detecting an initial user speech from the user;

determining whether the user speech restarts within a predetermined amount of time;

upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

upon a determination that the user speech does not restart, using the initial user speech.

2. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

determining whether the user has begun speaking, resulting in an initial user speech;

upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

upon a determination that the user speech restarts, performing the substeps of;

generating a second recognition result based upon the second user speech; and

reporting the second recognition result; and

upon a determination that the user speech does not restart, reporting the first recognition result.

3. The method of claim 2, wherein the predetermined amount of time is between one and four seconds.

4. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

determining whether the user has begun speaking, resulting in an initial user speech;

upon a determination that the user has begun speaking, performing the substeps of;

halting the playing of the prompt; and

generating a first recognition result based upon the initial user speech;

determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

upon a determination that the user speech restarts, performing the substeps of;

generating a second recognition result based upon the second user speech;

determining whether the first recognition result or the second recognition result is to be used;

upon a determination that the first recognition result is to be used, reporting the first recognition result; and

upon a determination that the second recognition result is to be used, reporting the second recognition result; and

upon a determination that the user speech does not restart, using the first recognition result.

5. The method of claim 4, wherein the predetermined amount of time is between one and four seconds.

6. The method of claim 4, wherein each of the first recognition result and the second recognition result comprises a confidence value, and wherein the step of determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

7. The method of claim 4, wherein the step of determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result comprising a multi-token result.

8. A method of providing a voice interface between a machine and a user, the method comprising the steps performed by the machine of:

playing a prompt;

receiving a voice command from the user indicating a preferred option;

performing a voice recognition algorithm on the voice command, the voice recognition algorithm returning a recognized command and a confidence value;

determining whether the recognized command corresponds to a valid function and whether the confidence value is acceptable;

upon a determination that the recognized command corresponds to a valid function and that the confidence value is acceptable, performing the actions corresponding to the recognized command;

upon a determination that the recognized command does not correspond to a valid function or that the confidence value is unacceptable, playing an alternative prompt for providing an alternative method for the user to enter a valid command;

determining whether the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts; and

upon a determination that the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts, transitioning into a known state.

9. A method of providing a voice interface between a machine and a user, the method comprising the steps performed by the machine of:

playing a prompt;

receiving a voice command from the user indicating a preferred option;

determining if the voice command is essentially a single-token command;

upon a determination that the voice command is essentially a single-token command, playing a multi-token prompt having the single-token command; and

upon a determination that the voice command is not essentially a single-token command, performing the voice command.

10. The method of claim 9, further comprising the step of, upon a determination that the voice command is not essentially a single-token command, playing a success prompt.

11. A method of determining a grammar of an interaction state of an interactive voice response system, the method comprising the steps of:

initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

adding to the grammar a second set of models corresponding to valid actions of a lateral state; and

adding to the grammar a third set of models corresponding to valid actions of an entry state.

12. The method of claim 11, wherein the method further comprises the step of adding to the grammar a fourth set of models corresponding to valid global commands.

13. The method of claim 11, wherein the second set of models comprise a subset of valid actions of the lateral state, the subset of valid actions being resistant to out-of-grammar errors.

14. The method of claim 11, wherein the third set of models comprise a subset of valid actions of the entry state, the subset of valid actions being resistant to out-of-grammar errors.

15. A method of determining a grammar of an interaction state of an interactive voice response system, the method comprising the steps of:

initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

adding to the grammar a second set of models comprising a subset of valid actions of a lateral state, the subset of valid actions being resistant to out-of-grammar errors; and

adding to the grammar a third set of models comprising a subset of valid actions of an entry state, the subset of valid actions of the entry state being resistant to out-of-grammar errors corresponding to valid actions of the entry state.

16. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for detecting an initial user speech from the user;

means for determining whether the user speech restarts within a predetermined amount of time;

means for, upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

means for, upon a determination that the user speech does not restart, using the initial user speech.

17. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for determining whether the user has begun speaking, resulting in an initial user speech;

means for, upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

means for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

means for, upon a determination that the user speech restarts, generating a second recognition result

based upon the second user speech, and reporting the second recognition result; and

means for, upon a determination that the user speech does not restart, reporting the first recognition result.

18. The apparatus of claim 17, wherein the predetermined amount of time is between one and four seconds.

19. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for determining whether the user has begun speaking, resulting in an initial user speech;

means for, upon a determination that the user has begun speaking, halting the playing of the prompt, and generating a first recognition result based upon the initial user speech;

means for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

means for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, determining whether the first recognition result or the second recognition result is to be used, reporting the first recognition result upon a determination that the first recognition result is to be used, and reporting the second recognition result upon a determination that the second recognition result is to be used; and

means for, upon a determination that the user speech does not restart, using the first recognition result.

20. The apparatus of claim 19, wherein the predetermined amount of time is between one and four seconds.

21. The apparatus of claim 19, wherein each of the first recognition result and the second recognition result comprises a confidence value, and wherein the means for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

22. The apparatus of claim 19, wherein the means for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result comprising a multi-token result.

23. An apparatus for providing a voice interface between a machine and a user, the apparatus comprising:

means for playing a prompt;

means for receiving a voice command from the user indicating a preferred option;

means for performing a voice recognition algorithm on the voice command, the voice recognition algorithm returning a recognized command and a confidence value;

means for determining whether the recognized command corresponds to a valid function and whether the confidence value is acceptable;

means for, upon a determination that the recognized command corresponds to a valid function and that the confidence value is acceptable, performing the actions corresponding to the recognized command;

means for, upon a determination that the recognized command does not correspond to a valid function or that the confidence value is unacceptable, playing an alternative prompt for providing an alternative method for the user to enter a valid command;

means for determining whether the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts; and

means for, upon a determination that the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts, transitioning into a known state.

24. An apparatus for providing a voice interface between a machine and a user, the apparatus comprising:

means for playing a prompt;

means for receiving a voice command from the user indicating a preferred option;

means for determining if the voice command is essentially a single-token command;

means for, upon a determination that the voice command is essentially a single-token command, playing a multi-token prompt having the single-token command; and

means for, upon a determination that the voice command is not essentially a single-token command, performing the voice command.

25. The apparatus of claim 24, further comprising means for, upon a determination that the voice command is not essentially a single-token command, playing a success prompt.

26. An apparatus for determining a grammar of an interaction state of an interactive voice response system, the apparatus comprising:

means for initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

means for adding to the grammar a second set of models corresponding to valid actions of a lateral state; and

means for adding to the grammar a third set of models corresponding to valid actions of an entry state.

27. The apparatus of claim 26, wherein the apparatus further comprises means for adding to the grammar a fourth set of models corresponding to valid global commands.

28. The apparatus of claim 26, wherein the second set of models comprise a subset of valid actions of the lateral state, the subset of valid actions being resistant to out-of-grammar errors.

29. The apparatus of claim 26, wherein the third set of models comprise a subset of valid actions of the entry state, the subset of valid actions being resistant to out-of-grammar errors.

30. An apparatus for determining a grammar of an interaction state of an interactive voice response system, the apparatus comprising:

means for initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

means for adding to the grammar a second set of models comprising a subset of valid actions of a lateral state, the subset of valid actions being resistant to out-of-grammar errors; and

means for adding to the grammar a third set of models comprising a subset of valid actions of an entry state, the subset of valid actions of the entry state being resistant to out-of-grammar errors corresponding to valid actions of the entry state.

31. A computer program product for allowing a user to barge-in over prompts, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for playing a prompt;

computer program code for detecting an initial user speech from the user;

computer program code for determining whether the user speech restarts within a predetermined amount of time;

computer program code for, upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

computer program code for, upon a determination that the user speech does not restart, using the initial user speech.

32. A computer program product for allowing a user to barge-in over prompts, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for playing a prompt;

computer program code for determining whether the user has begun speaking, resulting in an initial user speech;

computer program code for, upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

computer program code for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

computer program code for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, and reporting the second recognition result; and

computer program code for, upon a determination that the user speech does not restart, reporting the first recognition result.

33. The computer program product of claim 32, wherein the predetermined amount of time is between one and four seconds.

34. A computer program product for allowing a user to barge-in over prompts, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for playing a prompt;

computer program code for determining whether the user has begun speaking, resulting in an initial user speech;

computer program code for, upon a determination that the user has begun speaking, halting the playing

of the prompt, and generating a first recognition result based upon the initial user speech;

computer program code for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech;

computer program code for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, determining whether the first recognition result or the second recognition result is to be used, reporting the first recognition result upon a determination that the first recognition result is to be used, and reporting the second recognition result upon a determination that the second recognition result is to be used; and

computer program code for, upon a determination that the user speech does not restart, using the first recognition result.

35. The computer program product of claim 34, wherein the predetermined amount of time is between one and four seconds.

36. The computer program product of claim 34, wherein each of the first recognition result and the second recognition result comprises a confidence value, and wherein the computer program code for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

37. The computer program product of claim 34, wherein the computer program code for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result comprising a multi-token result.

38. A computer program product for providing a voice interface between a machine and a user, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for playing a prompt;

computer program code for receiving a voice command from the user indicating a preferred option;

computer program code for performing a voice recognition algorithm on the voice command, the voice recognition algorithm returning a recognized command and a confidence value;

computer program code for determining whether the recognized command corresponds to a valid function and whether the confidence value is acceptable;

computer program code for, upon a determination that the recognized command corresponds to a valid function and that the confidence value is acceptable, performing the actions corresponding to the recognized command;

computer program code for, upon a determination that the recognized command does not correspond to a valid function or that the confidence value is unacceptable, playing an alternative prompt for providing an alternative method for the user to enter a valid command;

computer program code for determining whether the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts; and

computer program code for, upon a determination that the voice recognition algorithm failed to return a valid recognized command and an acceptable confidence value after a plurality of consecutive attempts, transitioning into a known state.

39. A computer program product for providing a voice interface between a machine and a user, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for playing a prompt;

computer program code for receiving a voice command from the user indicating a preferred option;

computer program code for determining if the voice command is essentially a single-token command;

computer program code for, upon a determination that the voice command is essentially a single-token command, playing a multi-token prompt having the single-token command; and

computer program code for, upon a determination that the voice command is not essentially a single-token command, performing the voice command.

40. The computer program product of claim 40, further comprising computer program code for, upon a determination that the voice command is not essentially a single-token command, playing a success prompt.

41. A computer program product for determining a grammar of an interaction state of an interactive voice response system, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

computer program code for adding to the grammar a second set of models corresponding to valid actions of a lateral state; and

computer program code for adding to the grammar a third set of models corresponding to valid actions of an entry state.

42. The computer program product of claim 41, wherein the apparatus further comprises computer program code for adding to the grammar a fourth set of models corresponding to valid global commands.

43. The computer program product of claim 41, wherein the second set of models comprise a subset of valid actions of the lateral state, the subset of valid actions being resistant to out-of-grammar errors.

44. The computer program product of claim 41, wherein the third set of models comprise a subset of valid actions of the entry state, the subset of valid actions being resistant to out-of-grammar errors.

45. A computer program product for determining a grammar of an interaction state of an interactive voice response system, the computer program product having a medium with a computer program embodied thereon, the computer program comprising:

computer program code for initializing the grammar with a first set of models corresponding to valid actions of the interaction state;

computer program code for adding to the grammar a second set of models comprising a subset of valid actions of a lateral state, the subset of valid actions being resistant to out-of-grammar errors; and

computer program code for adding to the grammar a third set of models comprising a subset of valid actions of an entry state, the subset of valid actions of the entry state being resistant to out-of-grammar errors.