Voice response system

• US 8,145,494 B2
• Filed: 10/17/2008
• Issued: 03/27/2012
• Est. Priority Date: 02/20/2006
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Horioka, Masaru, Atake, Yoshinori, Tahara, Yoshinori
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Borsetti, Greg
• Application Number:
  US12/253,849
• Publication Number:
  US 20090141871A1
• Time in Patent Office: 1,257 Days
• Field of Search: 704/275, 704/270, 704/E15.04, 379/265.11
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Abstract

A voice response system attempts to respond to spoken user input and to provide computer-generated responses. If the system decides it cannot provide valid responses, the current state of user session is determined and forwarded to a human operator for further action. The system maintains a recorded history of the session in the form of a dialog history log. The dialog history and information as to the reliability of past speech recognition efforts is employed in making the current state determination. The system includes formatting rules for controlling the display of information presented to the human operator.

Claims

1. A voice interactive system for responding to a voice inquiry from a user during a session between the user and the system, comprising:

a combination of hardware and software configured to embody;

a voice interactive unit for recognizing speech in the voice inquiry from the user and providing responses to the inquiry;

a dialog history log for storing a record of a dialog between the voice interactive unit and the user;

a dialog state diagram definition file for defining each expected dialog state for the session and information regarding at least display of a dialog state diagram;

a dialog state determination model comprising information for estimating a current dialog state according to the dialog history log from a service start time to a current time;

a dialog information analyzing unit, responsive to a determination by the voice interactive unit that it cannot provide a valid response to the voice inquiry from the user, for estimating the current dialog state between the user and the voice interactive unit based on the dialog history log and the dialog state determination model; and

a dialog state display unit remote from the user for displaying the dialog state diagram, including a representation of each expected dialog state for the session, and further including a representation of at least one dialog state, other than a transition between dialog states, not yet entered during the session, according to the dialog state diagram definition file, and for visually presenting the current dialog state estimated by the dialog information analyzing unit on said dialog state diagram to an operator other than the user.

2. The voice interactive system according to claim 1, wherein the dialog history log includes at least one of:

speech recognition results from the start of the service to occurrence of operator transfer;

a reliability indicator for a speech recognition result;

one or more input items of a speech recognition result;

an answer sentence or ID thereof;

a start of service; and

an end of the service.

3. The voice interactive system according to claim 1, wherein the dialog state diagram definition file defines:

each dialog state in the diagram state diagram and arrangement information for display of the dialog state diagram; and

one or more input items to be displayed in each dialog state.

4. The voice interactive system according to claim 1, wherein the dialog state determination model includes an empirical data assembly or an algorithm assembly for estimating the current dialog state from the dialog history log.

5. The voice interactive system according to claim 1, wherein the dialog information analyzing unit calculates reliability of an item entered by the user and estimates an input history.

6. The voice interactive system according to claim 1, wherein the dialog state display unit visually highlights the current dialog state estimated by the dialog information analyzing unit on the dialog state diagram.

7. The voice interactive system according to claim 1, wherein the dialog state display unit uses the dialog history log to determine, with respect to an item entered by the user, reliability of a result of speech recognition performed by the voice interactive unit, and displays the item in a particular color depending on the determined reliability.

8. The voice interactive system according to claim 1, wherein the dialog state display unit uses the dialog history log to obtain, with respect to an item entered by the user, an input history of the user, and displays the item in a different character size according to its order in the input history.

9. A system for responding to a voice inquiry from a user during a session between the user and the system, the system comprising at least one processor configured to:

recognize speech in the voice inquiry from the user, and respond to the inquiry;

store a dialog history with the user in a dialog history log;

responsive to a determination that the system cannot provide a valid system response to spoken input items, estimate a current dialog state of the user, based on the dialog history log and a dialog state determination model comprising information for estimating the current dialog state according to the dialog history log from the service start time to the current time; and

display remotely from the user a dialog state diagram, including a representation of each expected dialog state for the session, and further including a representation of at least one dialog state, other than a transition between dialog states, not yet entered during the session, according to a dialog state diagram definition file defining each expected dialog state for the session and at least information regarding display of the dialog state diagram, and visually present the estimated current dialog state on the dialog state diagram to an operator other than the user.

10. The system according to claim 9, wherein the dialog history log includes at least one of:

speech recognition results from a start of service to an occurrence of forwarding to an operator,

a reliability for a speech recognition result,

an already entered item in a speech recognition result,

an answer sentence or ID thereof,

a start of the service, and

an end of the service.

11. The system according to claim 9, wherein the dialog state diagram definition file defines:

each dialog state in the dialog state diagram and arrangement information for display of the dialog state diagram, and

one or more input items to be displayed in each dialog state.

12. The system according to claim 9, wherein the dialog state determination model includes an assembly of empirical data or an assembly of algorithms for estimating the current dialog state from the dialog history log.

13. The system according to claim 9, wherein the at least one processor is further configured to calculate the reliability of an item entered by the user and take into account an input history.

14. The system according to claim 9, wherein the at least one processor is further configured to highlight the estimated current dialog state on the dialog state diagram.

15. A non-transitory computer readable medium having instructions encoded thereon that, when executed by at least one processor included in a voice interactive system, cause the at least one processor to execute a method for responding to inquiries by voice from a user during a session between the user and the system, the method comprising acts of:

recognizing speech in the voice from the user and responding to the inquiries;

storing a dialog history with the user in a dialog history log;

responsive to a determination that a valid response to spoken input items cannot be provided, estimating a current dialog state between the user and a voice interactive unit based on the dialog history log, and a dialog state determination model comprising information for estimating the current dialog state according to the dialog history log from the service start time to the current time; and

displaying a dialog state diagram, including a representation of each expected dialog state for the session, and further including a representation of at least one dialog state, other than a transition between dialog states, not yet entered during the session, according to a dialog state diagram definition file for defining each expected dialog state for the session and information regarding at least display of the dialog state diagram, and visually presenting the estimated current dialog state in the dialog state diagram to an operator.