User barge-in enablement in large vocabulary speech recognition systems

• US 6,651,043 B2
• Filed: 04/03/2001
• Issued: 11/18/2003
• Est. Priority Date: 12/31/1998
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Ammicht, Egbert, Gorin, Allen Louis
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): Azad, Abul K.
• Application Number:
  US09/825,020
• Publication Number:
  US 20010011217A1
• Time in Patent Office: 959 Days
• Field of Search: 704/231, 704/233, 704/270, 704/272, 704/275, 704/253, 704/254, 704/251, 704/244, 704/228, 704/214
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

An interactive voice response unit which provides beneficial operation by including means to handle unconstrained input such as natural speech and to allow barge-in includes a prompter, a recognizer of speech signals, a meaningful phrase detector and classifier, and a turn-taking module, all under control of a dialog manager. In the course of listening to user input while outputting a voiced message, the voice response unit processes the received signal and ascertains whether it is receiving an utterance that is intended to interrupt the prompt, or merely noise or an utterance that is not meant to be used by the arrangement. The unit is sensitive to the speed and context of the speech provided by the user and is thus able to distinguish between a situation where a speaker is merely pausing and a situation where a speaker is done speaking.

Claims

1. A method for communicating with a customer who communicates into a mouthpiece of telephone comprising the steps of:

applying a signal received by said mouthpiece to a preprocessing module that discards from said signal those components of said signal that fail to meet preselected usefulness criteria threshold test determined based on task-related speech of the speaker, resulting in an intermediate signal;

applying said intermediate signal to a phrase detector, to detect meaningful phrases within said intermediate signal; and

supplying to a controller data that corresponds to detected meaningful phrases.

2. The method of claim 1 where said preprocessing module accepts analog signals.

3. The method of claim 1 where said preprocessing module accepts digital signals that result from an A/D conversion of analog signals.

4. The method of claim 1 where said preprocessing module is adapted to accept digital signals that result from an A/D conversion of analog signals, or analog signals.

5. The method of claim 1 where said preprocessing module is adapted to accept signals resulting from symbols modulating an analog carrier.

6. The method of claim 1 where said preprocessing module performs its usefulness criteria determination function in the presence of a prompt signal being delivered to said customer by performing a processing step that removes influence of said prompt from its usefulness criteria determination.

7. The method of claim 1 where said phrase detector carries out the step of analyzing context of detected phrases and determining from context whether said customer has completed sending information.

8. The method of claim 1 where signal components that are received by said mouthpiece that result from other than sounds made by said customer are said components of said signal that fail to meet preselected usefulness criteria.

9. A method for communicating with a customer who speaks to a mouthpiece of a telephone comprising the steps of:

analyzing a signal received by said mouthpiece to determine whether said signal meets preselected usefulness criteria threshold test determined based on task-related speech of the speaker;

applying to a phrase detector those components of said signal that meet said usefulness criteria, to detect meaningful phrases within said signal; and

supplying to a controller data that corresponds to said meaningful phrases.

10. The method of claim 9 where said usefulness criteria differentiate between noise and non-noise signals.

11. The method of claim 9 where said usefulness criteria differentiate between speech and non-speech signals.

12. The method of claim 9 where said usefulness criteria differentiate between speech that is related to telecommunication functions and speech that is unrelated to telecommunication functions.

13. The method of claim 9 where said usefulness criteria accept DTMF signals as signals that meet said usefulness criteria.

14. The method of claim 9 further comprising the steps of:

said controller determining that an action is to be taken, even before a silence period is detected in said incoming signal, and

said controller taking such action even before said silence period is detected in said incoming signal.

15. The method of claim 14 where said action is a prompt signal delivered to said customer.

16. The method of claim 14 where said action is a cutting off of a prompt signal delivered to said customer.

17. The method of claim 14 where said action is a stopping of a prompt signal delivered to customer.

18. The method of claim 9 where said processing step that removes influence is an echo canceling processing step.

19. The method of claim 9 where said phrase detector analyzes a rate at which words are detected, and lengths of silences between detected words, to control a determination that a response is completed.

20. The method of claim 9 where said phrase detector detects significant speech phrases through word spotting.

21. The method of claim 9 where said phrase detector analyzes inflection of detected words.

22. The method claim 9 where said phrase detector employs detected words and detected silences between words to ascertain likelihood that useful input in said incoming signal has ended.

23. The method claim 9 where said phrase detector carries out the steps of:

translating meaningful phrases found in said output signals of said recognizer into tasks to be provided to said controller, and

ascertaining lengths of silences between significant signal segments detected by said phrase detector to ascertain likelihood that useful input in said incoming signal has ended.

24. The method claim 9 where said phrase detector carries out the step of determining, from grammatical construct detected phrases, whether said customer has completed sending information.

25. The method of claim 9 where said signal received by said mouthpiece comprises one or more components from a set containing elements a, b, c, and d, where

element a is a signal component that results from sounds made by said customer and intended to communicate information to said mouthpiece,

element b is a signal component that results from sounds made by said customer that is not intended to communicate information to said mouthpiece,

element c is a signal component that results from non-speech sounds made by said customer, and

element d is a signal component that results from sounds made by other than said customer,

and where element a is the only element that meets said usefulness criteria.