Assessing the structural quality of conversations

• US 10,311,895 B2
• Filed: 06/05/2018
• Issued: 06/04/2019
• Est. Priority Date: 09/26/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Akkiraju, Rama Kalyani T., Mahmud, Jalal, Sinha, Vibha S., Xu, Anbang, Gundecha, Pritam S., Bhuiyan, MD Mansurul A., Oraby, Shereen M.
• Primary Examiner(s): McFadden, Susan I
• Application Number:
  US15/997,967
• Publication Number:
  US 20190096426A1
• Time in Patent Office: 364 Days
• Field of Search: 704221
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06Q 10/06316 Sequencing of tasks or work
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04M 2203/2038 Call context notifications
• CPC Class Codes: H04M 2203/552 Call annotations
• CPC Class Codes: H04M 2203/558 Databases
• CPC Class Codes: H04M 3/2236 Quality of speech transmission monitoring
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

Input of a conversation is received. The conversation includes at least a first user. An utterance of the conversation is analyzed to identify a dialog act attribute, an emotion attribute, and a tone attribute. The dialog act attribute, emotion attribute, and tone attribute are annotated to the utterance of the conversation. The conversation is validated based on the annotated attributes compared with a threshold. The annotated conversation and the validation of the conversation are stored.

Claims

1. A computer program product comprising:

one or more non-transitory computer readable storage media and program instructions stored on the one or more non-transitory computer readable storage media, the program instructions comprising;

program instructions to receive an input of a conversation, in real time, between at least a first user and an automated response system, wherein the automated response system comprises an application that performs an automated task;

program instructions to analyze an utterance of the conversation to identify a dialog act attribute, an emotion attribute, and a tone attribute;

program instructions to annotate the utterance of the conversation with the dialog act attribute, the emotion attribute, and the tone attribute;

program instructions to validate the conversation based on the annotated attributes in comparison with a threshold, wherein program instructions to validate the conversation comprise;

program instructions to compare each annotated utterance in the conversation to a known database of best practice annotated conversations;

program instructions to determine a number of violations and a type of each violation using an algorithm selected from the group consisting of;

a sequence alignment algorithm, an information retrieval method, a sequence-to-one model, and a weighted edit distance;

program instructions to score each annotated utterance based on the number of violations, based on the comparison of each annotated utterance to the known database of best practice annotated conversations;

program instructions to rank each utterance based on the scoring; and

program instructions to suggest a response for a second user to make to the first user based on each ranked utterance;

program instructions to store the annotated conversation and the validation of the conversation;

program instructions to determine that the validation of the conversation indicates that the conversation has not met the threshold;

program instructions to transfer the first user to a second user;

program instructions to retrieve a plurality of stored annotated conversations;

program instructions to analyze the plurality of stored annotated conversations, wherein the analysis determines a number of violations found in each annotated conversation;

program instructions to rank each annotated conversation in the plurality of stored annotated conversations based on the analysis; and

program instructions to send at least one recommendation to a third user for improving at least one annotated conversation in the plurality of stored annotated conversations, wherein the at least one recommendation includes a change to a pre-defined dialog.