Method and system for predicting understanding errors in a task classification system
First Claim
1. A method of monitoring natural language understanding of a user'"'"'s input communication in a task classification system that operates on a task objective of a user, the method comprising:
- determining whether the user'"'"'s input communication is free of natural language understanding (NLU) errors and can be understood in order to make a task classification decision, wherein if the user'"'"'s input communication contains NLU errors and cannot be understood and a task classification decision cannot be made, the method further comprises the steps of;
determining whether a probability of understanding the user'"'"'s input communication including the NLU errors exceeds a first threshold, wherein if the first threshold is exceeded, further dialog is conducted with the user, otherwise,routing the user to a human for further assistance.
7 Assignments
0 Petitions
Accused Products
Abstract
This invention concerns a method and system for monitoring an automated dialog system for the automatic recognition of language understanding errors based on a user'"'"'s input communications in a task classification system. The method may include determining whether the user'"'"'s input communication can be understood in order to make a task classification decision. If the user'"'"'s input communication cannot be understood and a task classification decision cannot be made, a probability of understanding the user'"'"'s input communication may be determined. If the probability exceeds a first threshold, further dialog may be conducted with the user. Otherwise, the user may be directed to a human for assistance. In another possible embodiment, the method operates as above except that if the probability exceeds a second threshold, the second threshold being higher than the first, then further dialog may be conducted with the user using the current dialog strategy. However, if the probability falls between a first threshold and a second threshold, the dialog strategy may be adapted in order to improve the chances of conducting a successful dialog with the user. This process may be cumulative. In particular, the first dialog exchange may be stored in a database. Then, a second dialog exchange is conducted with the user. As a result, a second determination is made as to whether the user'"'"'s input communication can be understood can be conducted based on the stored first exchange and the current second exchanges. This cumulative process may continue using a third and fourth exchange, if necessary.
-
Citations
29 Claims
-
1. A method of monitoring natural language understanding of a user'"'"'s input communication in a task classification system that operates on a task objective of a user, the method comprising:
-
determining whether the user'"'"'s input communication is free of natural language understanding (NLU) errors and can be understood in order to make a task classification decision, wherein if the user'"'"'s input communication contains NLU errors and cannot be understood and a task classification decision cannot be made, the method further comprises the steps of; determining whether a probability of understanding the user'"'"'s input communication including the NLU errors exceeds a first threshold, wherein if the first threshold is exceeded, further dialog is conducted with the user, otherwise, routing the user to a human for further assistance. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A language understanding monitoring system that operates in a task classification system, comprising:
-
a task classification processor that determines whether a task classification decision can be made based on an understanding of a user'"'"'s input communication; a dialog manager that outputs dialog to the user; a language understanding monitor that determines whether a probability of understanding the user'"'"'s input communication exceeds a first threshold in the cases where a task classification decision cannot be made, wherein if the first threshold is exceeded, the language understanding monitor prompts the dialog manager to conduct further dialog with the user. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
-
Specification