System and method of dialog trajectory analysis

• US 8,949,131 B2
• Filed: 11/26/2012
• Issued: 02/03/2015
• Est. Priority Date: 01/05/2005
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Abella, Alicia, Wright, Jeremy Huntley
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/685,067
• Publication Number:
  US 20130077771A1
• Time in Patent Office: 799 Days
• Field of Search: 704257-258, 704270-275
• US Class Current: 704/270.1
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals

Abstract

The invention comprises computer-readable media, methods and systems for performing a dialog analysis and using that analysis to release an updated spoken dialog system. The method embodiment of the invention comprises receiving data associated with dialogs, extracting turn by turn details of the dialog and generating from the extracted details an empirical call-flow representation of the dialog. The call data may be call-logs and user audio. The empirical call-flow representation may be a finite-state machine with nodes that represent call-states and the arcs between nodes represent user responses. Nodes may also represent sub-dialogs. The call-flow representation is presented graphically to a user for easier analysis and understanding. Significant changes in the dialog can be identified as hot-spots for improvement in the next release of the spoken dialog system.

Claims

1. A method comprising:

receiving call-logs associated with a plurality of dialogs between a dialog system and a user;

receiving external information about the user;

extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising a time stamp associated with a turn in the plurality of dialogs;

identifying, via a processor, a second portion of the turn-by-turn details based on the first portion of the turn-by-turn details using a call-flow specification, wherein the second portion is unavailable in the call-logs and wherein the second portion comprises an interleaved sequence of two attributes which characterize a system state and a user response; and

generating an empirical call-flow representation of one of the dialogs between the dialog system and the user based on the first portion, the second portion, and the external information, wherein the empirical call-flow representation places emphasis on a selection of the one dialog, wherein the selection is modified when there is an option for an expanded view and when a change has occurred from a reference period associated with the call-logs and a current period.

2. The method of claim 1, wherein extracting the first portion of turn-by-turn details of dialogs further comprises identifying details of interactions between the dialog system and the user.

3. The method of claim 1, further comprising visually presenting the empirical call-flow.

4. The method of claim 3, wherein the empirical call-flow is presented on a web-page.

5. The method of claim 1, wherein extracting the first portion of the turn-by-turn details of dialogs further comprises partitioning the call-logs into two sub-dialogs.

6. The method of claim 5, wherein one of the two sub-dialogs ends in one of:

proceed to next dialog, transfer to agent, end call by the dialog system, and hang-up by caller.

7. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving call-logs associated with a plurality of dialogs between a dialog system and a user;

receiving external information about the user;

extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising a time stamp associated with a turn in the plurality of dialogs;

identifying, via a processor, a second portion of the turn-by-turn details based on the first portion of the turn-by-turn details using a call-flow specification, wherein the second portion is unavailable in the call-logs and wherein the second portion comprises an interleaved sequence of two attributes which characterize a system state and a user response; and

generating an empirical call-flow representation of one of the dialogs between the dialog system and the user based on the first portion, the second portion, and the external information, wherein the empirical call-flow representation places emphasis on a selection of the one dialog, wherein the selection is modified when there is an option for an expanded view and when a change has occurred from a reference period associated with the call-logs and a current period.

8. The system of claim 7, wherein extracting the first portion of turn-by-turn details of dialogs further comprises identifying details of interactions between the dialog system and the user.

9. The system of claim 7, the computer-readable storage medium having additional instruction stored which result in operations comprising visually presenting the empirical call-flow.

10. The system of claim 9, wherein the empirical call-flow is presented on a web-page.

11. The system of claim 7, wherein extracting the first portion of the turn-by-turn details of dialogs further comprises partitioning the call-logs into two sub-dialogs.

12. The system of claim 11, wherein one of the two sub-dialogs ends in one of:

proceed to next dialog, transfer to agent, end call by the dialog system, and hang-up by caller.

13. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving call-logs associated with a plurality of dialogs between a dialog system and a user;

receiving external information about the user;

extracting a first portion of turn-by-turn details of dialogs from the call-logs comprising a time stamp associated with a turn in the plurality of dialogs;

identifying, via a processor, a second portion of the turn-by-turn details based on the first portion of the turn-by-turn details using a call-flow specification, wherein the second portion is unavailable in the call-logs and wherein the second portion comprises an interleaved sequence of two attributes which characterize a system state and a user response; and

generating an empirical call-flow representation of one of the dialogs between the dialog system and the user based on the first portion, the second portion, and the external information, wherein the empirical call-flow representation places emphasis on a selection of the one dialog, wherein the selection is modified when there is an option for an expanded view and when a change has occurred from a reference period associated with the call-logs and a current period.

14. The computer-readable storage device of claim 13, wherein extracting the first portion of turn-by-turn details of dialogs further comprises identifying details of interactions between the dialog system and the user.

15. The computer-readable storage device of claim 13, the computer-readable storage device having additional instruction stored which result in operations comprising visually presenting the empirical call-flow.

16. The computer-readable storage device of claim 15, wherein the empirical call-flow is presented on a web-page.

17. The computer-readable storage device of claim 13, wherein extracting the first portion of the turn-by-turn details of dialogs further comprises partitioning the call-logs into two sub-dialogs.

18. The computer-readable storage device of claim 17, wherein one of the two sub-dialogs ends in one of:

proceed to next dialog, transfer to agent, end call by the dialog system, and hang-up by caller.