System and method of providing an automated data-collection in spoken dialog systems

• US 8,694,324 B2
• Filed: 05/21/2012
• Issued: 04/08/2014
• Est. Priority Date: 01/05/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Di Fabbrizio, Giuseppe, Hakkani-Tur, Dilek Z., Rahim, Mazin G., Renger, Bernard S., Tur, Gokhan
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US13/476,150
• Publication Number:
  US 20120232898A1
• Time in Patent Office: 687 Days
• Field of Search: 704231-257, 704270-278
• US Class Current: 704/275
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

The invention relates to a system and method for gathering data for use in a spoken dialog system. An aspect of the invention is generally referred to as an automated hidden human that performs data collection automatically at the beginning of a conversation with a user in a spoken dialog system. The method comprises presenting an initial prompt to a user, recognizing a received user utterance using an automatic speech recognition engine and classifying the recognized user utterance using a spoken language understanding module. If the recognized user utterance is not understood or classifiable to a predetermined acceptance threshold, then the method re-prompts the user. If the recognized user utterance is not classifiable to a predetermined rejection threshold, then the method transfers the user to a human as this may imply a task-specific utterance. The received and classified user utterance is then used for training the spoken dialog system.

Claims

1. A method comprising:

training a spoken dialog system using task-independent call-types of a previous application;

recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;

determining an acceptance threshold and a rejection threshold based on an entity referenced in the recognized user utterance;

classifying the recognized user utterance, to yield a classification, where the classification meets one of;

the rejection threshold, the acceptance threshold, and both the rejection threshold and the acceptance threshold;

when the classification meets the acceptance threshold acting according to a call-type associated with the classification; and

when the classification meets the rejection threshold, and does not meet the acceptance threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.

2. The method of claim 1, wherein when the classification does not meet the rejection threshold, the user utterance comprises a task-specific call-type.

3. The method of claim 1, wherein the spoken dialog system performs call-type classification.

4. The method of claim 1, further comprising:

when the classification meets a re-prompt threshold, issuing a prompt to the user to attempt receiving the user utterance a second time.

5. The method of claim 4, wherein when the received user utterance is received the second time and is silence, the classification meets the re-prompt threshold.

6. The method of claim 5, wherein upon a re-prompt counter meeting a re-prompt number, setting the classification to meet the rejection threshold.

7. The method of claim 1, wherein the method is performed by an automated hidden human.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed on the processor, perform operations comprising;

training a spoken dialog system using task-independent call-types of a previous application;

recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;

determining an acceptance threshold and a rejection threshold based on an entity referenced in the recognized user utterance;

classifying the recognized user utterance, to yield a classification, where the classification meets one of;

the rejection threshold, the acceptance threshold, and both the rejection threshold and the acceptance threshold;

when the classification meets the acceptance threshold acting according to a call-type associated with the classification; and

when the classification meets the rejection threshold, and does not meet the acceptance threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.

9. The system of claim 8, wherein when the classification does not meet the rejection threshold, the user utterance comprises a task-specific call-type.

10. The system of claim 8, wherein the spoken dialog system performs call-type classification.

11. The system of claim 8, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

when the classification meets a re-prompt threshold, issuing a prompt to the user to attempt receiving the user utterance a second time.

12. The system of claim 11, wherein when the received user utterance is received the second time and is silence, the classification meets the re-prompt threshold.

13. The system of claim 12, wherein upon a re-prompt counter meeting a re-prompt number, setting the classification to meet the rejection threshold.

14. The system of claim 8, wherein the operations are performed by an automated hidden human.

15. A computer-readable storage device having instructions stored which, when executed by a computing device, perform operations comprising:

training a spoken dialog system using task-independent call-types of a previous application;

recognizing a user utterance using the spoken dialog system, to yield a recognized user utterance;

determining an acceptance threshold and a rejection threshold based on an entity referenced in the recognized user utterance;

classifying the recognized user utterance, to yield a classification, where the classification meets one of;

the rejection threshold, the acceptance threshold, and both the rejection threshold and the acceptance threshold;

when the classification meets the acceptance threshold acting according to a call-type associated with the classification; and

when the classification meets the rejection threshold, transcribing the recognized user utterance and using the recognized user utterance for further training of the spoken dialog system.

16. The computer-readable storage device of claim 15, wherein when the classification does not meet the rejection threshold, the user utterance comprises a task-specific call-type.

17. The computer-readable storage device of claim 15, wherein the spoken dialog system performs call-type classification.

18. The computer-readable storage device of claim 15, the computer-readable storage device having additional instructions which result in the operations further comprising:

when the classification meets a re-prompt threshold, issuing a prompt to the user to attempt receiving the user utterance a second time.

19. The computer-readable storage device of claim 18, wherein when the received user utterance is received the second time and is silence, the classification meets the re-prompt threshold.

20. The computer-readable storage device of claim 19, wherein upon a re-prompt counter meeting a re-prompt number, setting the classification to meet the rejection threshold.