Method and system for preventing error amplification in natural language dialogues

• US 7,194,409 B2
• Filed: 11/30/2001
• Issued: 03/20/2007
• Est. Priority Date: 11/30/2000
• Status: Expired due to Term

• Current Assignee: Shadow Prompt Technology AG
• Original Assignee: Enterprise Integration Group, Inc. (Perspecta, Inc.)
• Inventors: Stringham, Rex, Balentine, Bruce, Munroe, Justin, Melaragno, Ralph
• Primary Examiner(s): Storm; Donald L.
• Application Number:
  US10/433,021
• Publication Number:
  US 20040098253A1
• Time in Patent Office: 1,936 Days
• Field of Search: 704/253, 704/233, 704/275, 379/88.01, 379/88.02, 379/88.03, 379/88.04
• US Class Current: 704/253
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00

Abstract

A method and system for allowing a user to interface to an interactive voice response system via natural language commands. The system plays a prompt that initiates user interaction. In certain embodiments, the system detects initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt. Then, the system determines whether the user speech restarts (second user speech) within a predetermined time period, wherein the predetermined time period is dependent upon whether the initial user speech began during the prompt or during the silence. If the user speech does restart, then the system uses the second user speech for recognition purposes. If the user speech does not restart, then the system uses the initial user speech for recognition purposes.

Claims

1. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

determining whether a second user speech starts within a predetermined amount of time, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

upon a determination that the second user speech starts, using the second user speech for recognition purposes; and

upon a determination that the second user speech does not start, using the initial user speech for recognition purposes.

2. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

upon a determination that the user speech restarts, performing the substeps of;

generating a second recognition result based upon the second user speech; and

reporting the second recognition result; and

upon a determination that the user speech does not restart, reporting the first recognition result.

3. The method of claim 2, wherein the predetermined amount of time is between one and four seconds.

4. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

upon a determination that the user has begun speaking, performing the substeps of;

halting the playing of the prompt; and

generating a first recognition result based upon the initial user speech;

determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

upon a determination that the user speech restarts, performing the substeps of;

generating a second recognition result based upon the second user speech;

determining whether the first recognition result or the second recognition result is to be used;

upon a determination that the first recognition result is to be used, reporting the first recognition result; and

upon a determination that the second recognition result is to be used, reporting the second recognition result; and

upon a determination that the user speech does not restart, using the first recognition result.

5. The method of claim 4, wherein the predetermined amount of time is between one and four seconds.

6. The method of claim 4, wherein each of the first recognition result based at least partly upon the initial user speech and the second recognition result based at least partly upon the second user speech comprises a confidence value, and wherein the step of determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

7. The method of claim 4, wherein the step of determining whether the first recognition result based at least partly upon the initial user speech or the second recognition result based at least partly upon the second user speech is to be used is performed by selecting the result comprising a multi-token result.

8. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

means for determining whether a second user speech starts within a predetermined amount of time, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

means for, upon a determination that the second user speech starts, using the second user speech for recognition purposes; and

means for, upon a determination that the second user speech does not start, using the initial user speech for recognition purposes.

9. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

means for, upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

means for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

means for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, and reporting the second recognition result; and

means for, upon a determination that the user speech does not restart, reporting the first recognition result.

10. The apparatus of claim 9, wherein the predetermined amount of time is between one and four seconds.

11. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

means for, upon a determination that the user has begun speaking, halting the playing of the prompt, and generating a first recognition result based upon the initial user speech;

means for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

means for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, determining whether the first recognition result or the second recognition result is to be used, reporting the first recognition result upon a determination that the first recognition result is to be used, and reporting the second recognition result upon a determination that the second recognition result is to be used; and

means for, upon a determination that the user speech does not restart, using the first recognition result.

12. The apparatus of claim 11, wherein the predetermined amount of time is between one and four seconds.

13. The apparatus of claim 11, wherein each of the first recognition result based at least partly upon the initial user speech and the second recognition result based at least partly upon the second user speech comprises a confidence value, and wherein the means for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

14. The apparatus of claim 11, wherein the means for determining whether the first recognition result based at least partly upon the initial user speech or the second recognition result based at least partly upon the second user speech is to be used is performed by selecting the result comprising a multi-token result.

15. A computer program product for allowing a user to barge-in over prompts, the computer program product having a computer-readable medium with a computer-executable program embodied thereon, the computer-executable program comprising:

computer program code for playing a prompt;

computer program code for detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

computer program code for determining whether a second user speech starts within a predetermined amount of time, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

computer program code for, upon a determination that the second user speech starts, using the second user speech for recognition purposes; and

computer program code for, upon a determination that the second user speech does not start, using the initial user speech for recognition purposes.

16. A computer program product for allowing a user to barge-in over prompts, the computer program product having a computer-readable medium with a computer-executable program embodied thereon, the computer-executable program comprising:

computer program code for playing a prompt;

computer program code for determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

computer program code for, upon a determination that the user has begun speaking, generating a first recognition result based upon the initial user speech;

computer program code for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

computer program code for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, and reporting the second recognition result; and

computer program code for, upon a determination that the user speech does not restart, reporting the first recognition result.

17. The computer program product of claim 16, wherein the predetermined amount of time is between one and four seconds.

18. A computer program product for allowing a user to barge-in over prompts, the computer program product having a computer-readable medium with a computer-executable program embodied thereon, the computer-executable program comprising:

computer program code for playing a prompt;

computer program code for determining whether the user has begun speaking, resulting in an initial user speech, wherein the initial user speech begins during the prompt or during a silence after the prompt;

computer program code for, upon a determination that the user has begun speaking, halting the playing of the prompt, and generating a first recognition result based upon the initial user speech;

computer program code for determining whether the user speech restarts within a predetermined amount of time, resulting in a second user speech, wherein a first predetermined amount of time corresponds to initial user speech during the prompt and a second predetermined amount of time corresponds to initial user speech during the silence;

computer program code for, upon a determination that the user speech restarts, generating a second recognition result based upon the second user speech, determining whether the first recognition result or the second recognition result is to be used, reporting the first recognition result upon a determination that the first recognition result is to be used, and reporting the second recognition result upon a determination that the second recognition result is to be used; and

computer program code for, upon a determination that the user speech does not restart, using the first recognition result.

19. The computer program product of claim 18, wherein the predetermined amount of time is between one and four seconds.

20. The computer program product of claim 18, wherein each of the first recognition result and the second recognition result comprises a confidence value, and wherein the computer program code for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result with the best confidence value.

21. The computer program product of claim 18, wherein the computer program code for determining whether the first recognition result or the second recognition result is to be used is performed by selecting the result comprising a multi-token result.

22. A method of allowing a user to barge-in over prompts, the method comprising the steps performed by a machine of:

playing a prompt;

detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

determining whether the user speech restarts within a predetermined amount of time, wherein the predetermined amount of time is varied at least partly in dependence upon a confidence level assigned to the initial user speech and wherein the confidence level is dependent upon whether the initial user speech begins during the prompt or during the silence;

upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

upon a determination that the user speech does not restart, using the initial user speech.

23. An apparatus for allowing a user to barge-in over prompts, the apparatus comprising:

means for playing a prompt;

means for detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

means for determining whether the user speech restarts within a predetermined amount of time, wherein the predetermined amount of time is varied at least partly in dependence upon a confidence level assigned to the initial user speech and wherein the confidence level is dependent upon whether the initial user speech begins during the prompt or during the silence;

means for, upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

means for, upon a determination that the user speech does not restart, using the initial user speech.

24. A computer program product for allowing a user to barge-in over prompts, the computer program product having a computer-readable medium with a computer-executable program embodied thereon, the computer-executable program comprising:

computer program code for playing a prompt;

computer program code for detecting an initial user speech from the user, wherein the initial user speech begins during the prompt or during a silence after the prompt;

computer program code for determining whether the user speech restarts within a predetermined amount of time, wherein the predetermined amount of time is varied at least partly in dependence upon a confidence level assigned to the initial user speech and wherein the confidence level is dependent upon whether the initial user speech begins during the prompt or during the silence;

computer program code for, upon a determination that the user speech restarts, using the user speech after the restart for recognition purposes; and

computer program code for, upon a determination that the user speech does not restart, using the initial user speech.