System and method of dialog trajectory analysis
First Claim
1. A method of analyzing dialogs, the method comprising:
- receiving, via a processor, call-logs associated with a plurality of dialogs between a dialog system and users and external information about at least one user;
extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising at least a time stamp associated with a turn in the plurality of dialogs;
inferring a second portion of the turn-by-turn details unavailable in the call-logs based on the first portion of the turn-by-turn details using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and
generating, from the first portion of the turn-by-turn details, the external information about the user, and the second portion of the turn-by-turn details, an empirical call-flow representation of the dialog.
4 Assignments
0 Petitions
Accused Products
Abstract
The invention comprises computer-readable media, methods and systems for performing a dialog analysis and using that analysis to release an updated spoken dialog system. The method embodiment of the invention comprises receiving data associated with dialogs, extracting turn by turn details of the dialog and generating from the extracted details an empirical call-flow representation of the dialog. The call data may be call-logs and user audio. The empirical call-flow representation may be a finite-state machine with nodes that represent call-states and the arcs between nodes represent user responses. Nodes may also represent sub-dialogs. The call-flow representation is presented graphically to a user for easier analysis and understanding. Significant changes in the dialog can be identified as hot-spots for improvement in the next release of the spoken dialog system.
41 Citations
30 Claims
-
1. A method of analyzing dialogs, the method comprising:
-
receiving, via a processor, call-logs associated with a plurality of dialogs between a dialog system and users and external information about at least one user; extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising at least a time stamp associated with a turn in the plurality of dialogs; inferring a second portion of the turn-by-turn details unavailable in the call-logs based on the first portion of the turn-by-turn details using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and generating, from the first portion of the turn-by-turn details, the external information about the user, and the second portion of the turn-by-turn details, an empirical call-flow representation of the dialog. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A method of generating a visual representation of a plurality of dialogs, the method comprising:
-
identifying, via a processor, data associated with the plurality of dialogs between a dialog system and users and external information at least one user retrieved from a database, the data comprising call-logs for the plurality of dialogs identifying a first portion of turn-by-turn details of the plurality of dialogs; appending a dialog outcome to the data, the dialog outcome received in a data feed of the call logs; inferring a second portion of the turn-by-turn details of the plurality of dialogs unavailable from the call-logs based on the plurality of dialogs outcome using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and generating from the appended dialog outcome, identified data, the second portion of the turn-by-turn details, and the external information about the user, a dialog representation comprising nodes related to a dialog state and arcs related to user responses. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24)
-
-
25. A system for analyzing dialogs, the system comprising:
-
a processor; means for controlling the processor to receive data associated with a plurality of dialogs between a dialog system and users and external information about at least one user retrieved from a database, the data comprising call-logs for the plurality of dialogs; means for controlling the processor to extract a first portion of turn-by-turn details of the plurality of dialogs from a data feed of the call-logs comprising at least a time stamp associated with a turn in the plurality of dialogs; means for controlling the processor to infer a second portion of the turn-by-turn details unavailable in the call-logs based on the first portion of the turn-by-turn details using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response, the call-flow specification describing actions that should be performed based on user input; and means for controlling the processor to generate from the first portion of the turn-by-turn details, the external information about the user, and the second portion of the turn-by-turn details an empirical call-flow representation of the plurality of dialogs. - View Dependent Claims (26)
-
-
27. A system for analyzing dialogs, the system comprising:
-
a processor; a first module configured to control the processor to receive call-logs associated with a plurality of dialogs between a dialog system and users and external information about at least one user; a second module configured to control the processor to extract a first portion of turn-by-turn details of dialogs from of the call-logs comprising at least a time stamp associated with a in the plurality of dialogs; a third module configured to control the processing to infer a second portion of the turn-by-turn details unavailable in the call-logs based on the first portion of the turn-by-turn details using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and a fourth module configured to control the processor to generate, from the first portion of the turn-by-turn details, the external information about at least one user, and the second portion of the turn-by-turn details an empirical call-flow representation of the plurality of dialogs. - View Dependent Claims (28)
-
-
29. A non-transitory computer-readable medium storing instructions for controlling a computing device to analyze dialogs, the instructions comprising:
-
receiving call-logs associated with a plurality of dialogs between a dialog system and users and external information about at least one user; extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising at least a time stamp associated with a turn in the plurality of dialogs; inferring a second portion of the turn-by-turn details unavailable in the call-logs based on the first portion of the turn-by-turn details using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and generating, from the first portion of the turn-by-turn details, the external information about the at least one user, and the second portion of the turn-by-turn details an empirical call-flow representation of the plurality of dialogs.
-
-
30. A non-transitory computer-readable medium storing instructions for controlling a computing device to analyze dialogs, the instructions comprising:
-
identifying data associated with a plurality of dialogs between a dialog system and users and external information about at least one user retrieved from a database; appending a dialog outcome to the data, the dialog outcome received in a data feed of call logs; inferring a second portion of the turn-by-turn details of the dialog unavailable from the call-logs based on the dialog outcome using a call-flow specification as a guide, the second portion of the turn-by-turn details comprising an interleaved sequence of at least two attributes that characterize a system state and a user response; and generating from the dialog outcome, identified data, the second portion of the turn-by-turn details, and the external information about the user, a dialog representation comprising nodes related to a dialog state and arcs related to user responses.
-
Specification