System and method of providing an automated data-collection in spoken dialog systems

• US 8,185,399 B2
• Filed: 01/05/2005
• Issued: 05/22/2012
• Est. Priority Date: 01/05/2005
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Tur, Gokhan, Di Fabbrizio, Giuseppe, Hakkani-Tur, Dilek Z., Rahim, Mazin G., Renger, Bernard S.
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US11/029,798
• Publication Number:
  US 20060149555A1
• Time in Patent Office: 2,694 Days
• Field of Search: 704231-257, 704270-278
• US Class Current: 704/275
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

The invention relates to a system and method for gathering data for use in a spoken dialog system. An aspect of the invention is generally referred to as an automated hidden human that performs data collection automatically at the beginning of a conversation with a user in a spoken dialog system. The method comprises presenting an initial prompt to a user, recognizing a received user utterance using an automatic speech recognition engine and classifying the recognized user utterance using a spoken language understanding module. If the recognized user utterance is not understood or classifiable to a predetermined acceptance threshold, then the method re-prompts the user. If the recognized user utterance is not classifiable to a predetermined rejection threshold, then the method transfers the user to a human as this may imply a task-specific utterance. The received and classified user utterance is then used for training the spoken dialog system.

Claims

1. A method comprising:

training, via a processor, an automatic speech recognition module, a spoken language understanding module and a dialog management module using task-independent call-types of a previous application;

recognizing a received user utterance in response to a prompt to a user using the automatic speech recognition module to yield a recognized user utterance;

classifying the recognized user utterance using the spoken language understanding module to yield a classification;

if the recognized user utterance is classifiable by the spoken language understanding module with a confidence more than an acceptance threshold, then acting according to a call-type associated with the classification; and

if the recognized user utterance is not classifiable by the spoken language understanding module to a predetermined rejection threshold and if a dialog-wide re-prompt counter is exceeded, then transferring the user to a human, wherein the received user utterance is transcribed, labeled and used for training the spoken dialog system, and

wherein at least one of the acceptance threshold and the predetermined rejection threshold are related to an entity referenced in the recognized user utterance.

2. The method of claim 1, wherein if the recognized user utterance is not classifiable to a predetermined rejection threshold, then the received user utterance contains an utterance with a task-specific call-type.

3. The method of claim 1, wherein the recognized user utterance is classifiable as generic.

4. The method of claim 1, wherein the method is performed by an automated hidden human.

5. The method of claim 1, wherein the received user utterance is used to design the spoken dialog system for identifying call-types.

6. The method of claim 1, wherein the spoken dialog system performs call-type classification.

7. The method of claim 1, further comprising:

if the recognized user utterance is not classifiable to a predetermined threshold, then re-prompting the user, wherein the step of re-prompting the user further comprises re-prompting the user to seek to get the user to identify a purpose of a call.

8. The method of claim 1, wherein the spoken language understanding module is a generic module.

9. The method of claim 1, wherein the method further comprises re-prompting the user if the received user utterance is silence.

10. A system comprising:

a processor;

means for controlling the processor to obtain task-independent training data;

means for controlling the processor to present a prompt to a user;

means for controlling the processor to recognize a received user utterance in response to the prompt using an automatic speech recognition engine to yield a recognized user utterance;

means for controlling the processor to classify the recognized user utterance using a spoken language understanding module;

means for controlling the processor to prompt the user if the recognized user utterance is classifiable by the spoken language understanding module with a confidence more than a predetermined acceptance threshold; and

means for controlling the processor to transfer the user to a human if the recognized user utterance is not classifiable by the spoken language understanding module to a predetermined rejection threshold and if a dialog-wide re-prompt counter is exceeded, wherein the received user utterance is transcribed, labeled and used for training the spoken dialog system, and

wherein at least one of the acceptance threshold and the predetermined rejection threshold are related to an entity referenced in the recognized user utterance.

11. The system of claim 10, wherein if the user utterance is not classifiable to a predetermined rejection threshold, then the utterance contains a task-specific call-type.

12. The system of claim 10, wherein the recognized user utterance is classifiable as generic.

13. The system of claim 10, wherein the system is an automated hidden human.

14. The system of claim 10, wherein the received user utterance is used to design the spoken dialog system for identifying call-types.

15. The system of claim 10, wherein the spoken dialog system performs call-type classification.

16. The system of claim 10, wherein further comprising means, if the recognized user utterance is not classifiable to a pre-determined threshold, for controlling the processor to reprompt the user, wherein the means for re-prompting the user further comprises means for re-prompting the user to seek to get the user to identify a purpose of a call.

17. The system of claim 10, wherein the spoken language understanding module is a generic module.

18. A system for gathering data for use in training a spoken dialog system, the system comprising:

a processor; and

a computer-readable storage medium storing instructions for controlling the processor to perform a method comprising;

determining task-independent training data from a library formed using data from previous applications;

recognizing a received user utterance in response to a prompt to a user using an automatic speech recognition engine to yield a recognized user utterance;

classifying the recognized user utterance using a spoken language understanding module; and

transferring the user to a human if the recognized user utterance is not classifiable by the spoken language understanding module to a predetermined rejection threshold and if a dialog-wide re-prompt counter is exceeded, wherein the received user utterance is transcribed, labeled and used for training the spoken dialog system,

adjusting at least one of the acceptance threshold and the predetermined rejection threshold is based on an entity referenced in the recognized user utterance.

19. A non-transitory computer-readable medium storing instructions for controlling a computing device to gather data for use in training a spoken dialog system, the instructions comprising:

managing a library of training data obtained from previous applications;

recognizing a received user utterance in response to a prompt to a user using an automatic speech recognition engine to yield a recognized user utterance;

classifying the recognized user utterance using a spoken language understanding module; and

if the recognized user utterance is not classifiable by the spoken language understanding module to a predetermined rejection threshold and if a dialog-wide re-prompt counter is exceeded, then transferring the user to a human, wherein the received user utterance is transcribed, labeled and used for training the spoken dialog system,

wherein the predetermined rejection threshold is related to an entity referenced in the recognized user utterance.

20. The non-transitory computer-readable medium of claim 19, wherein the recognized user utterance is classifiable as generic.

21. The non-transitory computer-readable medium of claim 19, wherein the received and classified user utterance is used to design the spoken dialog system for identifying call-types.

22. The non-transitory computer-readable medium of claim 19, wherein the spoken dialog system performs call-type classification.

23. The non-transitory computer-readable medium of claim 19, further comprising:

if the recognized user utterance is not understood or classifiable to a predetermined acceptance threshold, then re-prompting the user, wherein the step of re-prompting the user further comprises re-prompting the user to seek to get the user to identify a purpose of a call.

24. The non-transitory computer-readable medium of claim 19, wherein the spoken language understanding module is a generic module.

25. A method comprising:

presenting a prompt to a user in a call;

recognizing via a processor an utterance received in response to the prompt using an automatic speech recognition module;

classifying the utterance using a generic spoken language understanding module;

determining whether the recognition and classification meet a threshold;

if at least one of the recognition by the automatic speech recognition module and classification by the spoken language understanding module does not meet the threshold and if a dialog-wide re-prompt counter is exceeded, then transferring the user to a human agent wherein the utterance is transcribed, labeled and used for training the spoken dialog system; and

if at least one of the recognition and classification does meet the threshold, then routing the call according to a call-type associated with the classification,

wherein the threshold is related to an entity referenced in the utterance.

26. The method of claim 25, wherein the threshold is one of an acceptance threshold and a rejection threshold.

27. The method of claim 26, wherein if the recognition or the classification does not meet the threshold, the method further comprises determining whether a re-prompt counter is exceeded before re-prompting the user.

28. The method of claim 25, wherein recognizing the utterance relates to identifying the entity in the utterance.

29. The method of claim 28, wherein the entity is a named entity.