Adaptive learning of actionable statements in natural language conversation

• US 9,904,669 B2
• Filed: 01/13/2016
• Issued: 02/27/2018
• Est. Priority Date: 01/13/2016
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gunaratna, Dalkandura Arachchige Kalpa Shashika Silva, Motahari Nezhad, Hamid Reza
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US14/995,116
• Publication Number:
  US 20170199866A1
• Time in Patent Office: 776 Days
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Identifying actionable statements in communications may include: extracting features from at least one training statement; training a pattern recognition module to identify one or more types of patterns in actionable statements based at least in part on the features; and generating an actionable statement identification model using the trained action verb module and the trained pattern recognition module. Identifying actionable statements in communications is preferably adaptive in a continuous manner (e.g. based on user feedback), and may also include: determining whether a statement includes an actionable statement; predicting an actionable statement class of the actionable statement based on a pattern represented in the statement; and outputting the predicted actionable statement class to a user. Corresponding systems and computer program products are also disclosed.

Claims

1. A computer-implemented method for automatically identifying actionable statements in electronic communications, the method comprising:

obtaining a first set of rules that define actionable statements as a function of tags, tokens, and contextual elements associated with target verbs;

extracting features from at least one training statement based on the first set of rules, wherein the features include one or more of tags and language token types for words of the statement;

training a pattern recognition module to identify one or more types of patterns in actionable statements based at least in part on the features; and

training an action verb module to identify dependency of the actionable statements based on at least some of the features;

generating an actionable statement identification model using the trained action verb module and the trained pattern recognition module, the actionable statement identification model including a plurality of target nodes, each node having associated therewith one or more active features and one or more corresponding weights of the one or more active features;

adaptively training the actionable statement model by at least one of promoting and demoting the weights corresponding to the active features associated with some or all of the plurality of target nodes,

wherein weights corresponding to the active features are promoted in response to determining a true positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond; and

wherein weights corresponding to the active features are demoted in response to determining a false positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond;

receiving an incoming electronic communication;

extracting the features of the words and phrases in statements of the incoming electronic communication;

filtering the statements to identify the statements that include one or more action verbs as actionable statements and the statements that do not include any action verbs as filtered out statements;

outputting the statements that were removed by the filtering to a user for user feedback;

analyzing statement patterns of the actionable statements to predict a type of each of the actionable statements, by applying the actionable statement identification model to the actionable statements;

outputting a predicted actionable statement type to the user for user feedback; and

performing continual training of the actionable statement identification model by providing the user feedback identifying statements that haven not been seen before or the user feedback indicating a different type for a statement than the predicted type to the actionable statement identification model.

2. The method as recited in claim 1, comprising:

building an actionable verb dictionary using the trained action verb module; and

wherein the adaptive training is performed continuously as a prediction module predicts types of actionable statements present in one or more sample statements.

3. The method as recited in claim 1, wherein the action verb module is trained to identify dependency of actionable verb statements based on determining whether an action verb is enclosed by another verb.

4. The computer-implemented method as recited in claim 1, wherein weights corresponding to the active features are promoted in response to determining a true positive mismatch between active features of a particular statement within an electronic communication and the active features to which the weights correspond.

5. The computer-implemented method as recited in claim 1, wherein promoting the weights corresponding to the active features associated with some or all of the plurality of target nodes comprises, for all active features A_i and corresponding weights w_t,i of a target node t among the plurality of target nodes, multiplying the corresponding weights w_t,i by a promotion constant α

_t in response to determining that a sum of the weights w_t,i for the node t and set of active features A_t is less than a node threshold Q_t.

6. The computer-implemented method as recited in claim 1, wherein demoting the weights corresponding to the active features associated with some or all of the plurality of target nodes comprises, for all active features A_i and corresponding weights w_t,i of a target node t among the plurality of target nodes, multiplying the corresponding weights w_t,i by a demotion constant β

_t in response to determining that a sum of the weights w_t,i for the node t and set of active features A_t is greater than a node threshold Q_t.

7. The computer-implemented method as recited in claim 1, wherein the plurality of target nodes comprise a “

promise”

node, a “

request”

node, and an “

other”

node.

8. The computer-implemented method as recited in claim 1,

wherein the tags and token types correspond to one or more of:

a pronoun (PRON) token;

an auxiliary (AUX) token;

one or more personal named entities (NER) tags relating to a focus verb present in the training statement;

a mood of one or more verbs present in the training statement;

whether one or more verbs present in the training statement are action verbs;

whether one or more verbs present in the training statement enclose another verb or verbs present in the training statement;

whether a number of enclosed action verbs present in the training statement is greater than zero; and

a tense of at least one verb present in the training statement.

9. The computer-implemented method as recited in claim 8, wherein the mood of the one or more verbs present in the training statement are selected from:

normal, imperative, and infinitive.

10. The computer-implemented method as recited in claim 8, wherein past-tense verbs present in the training statement are disqualified from being considered action verbs.

11. The computer-implemented method as recited in claim 1, wherein the features are extracted using an Annotation Query Language (AQL).

12. The computer-implemented method as recited in claim 1,

wherein the action verb module is trained to identify dependency of actionable verb statements based on determining whether an action verb is enclosed by another verb;

wherein weights corresponding to the active features are promoted in response to determining a true positive mismatch between active features of a particular statement within an electronic communication and the active features to which the weights correspond;

wherein promoting the weights corresponding to the active features associated with some or all of the plurality of target nodes comprises, for all active features A_i and corresponding weights w_{t, i} of a target node t among the plurality of target nodes, multiplying the corresponding weights w_t,i by a promotion constant α

_t in response to determining that a sum of the weights w_t,i for the node t and set of active features A_t is less than a node threshold Q_t; and

wherein the plurality of target nodes comprise a “

promise”

node, a “

request”

node, and an “

other”

node.

13. A computer program product for identifying actionable statements in electronic communications, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

obtain a first set of rules that define actionable statements as a function of tags, tokens, and contextual elements associated with target verbs;

extract features from at least one training statement based on the first set of rules, wherein the features include one or more of tags and language token types for words of the statement;

train a pattern recognition module to identify one or more types of patterns in actionable statements based at least in part on the features; and

train an action verb module to identify dependency of the actionable statements based on at least some of the features;

generate an actionable statement identification model using the trained action verb module and the trained pattern recognition module, the actionable statement identification model including a plurality of target nodes, each node having associated therewith one or more active features and one or more corresponding weights of the one or more active features;

adaptively train the actionable statement model by at least one of promoting and demoting the weights corresponding to the active features associated with some or all of the plurality of target nodes,

wherein weights corresponding to the active features are promoted in response to determining a true positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond; and

wherein weights corresponding to the active features are demoted in response to determining a false positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond;

receive an incoming electronic communication;

extract the features of the words and phrases in statements of the incoming electronic communication;

filter the statements to identify the statements that include one or more action verbs as actionable statements and the statements that do not include any action verbs as filtered out statements;

output the statements that were removed by the filtering to a user for user feedback;

analyze statement patterns of the actionable statements to predict a type of each of the actionable statements, by applying the actionable statement identification model to the actionable statements;

output a predicted actionable statement type to the user for user feedback; and

perform continual training of the actionable statement identification model by providing the user feedback identifying statements that haven not been seen before or the user feedback indicating a different type for a statement than the predicted type to the actionable statement identification model.

14. A system, comprising:

a processor and logic integrated with and/or executable by the processor, the logic being configured to cause the processor to;

determine whether a statement, within an electronic communication, includes an actionable statement,

wherein determining whether a statement includes an actionable statement comprises;

obtaining a first set of rules that define actionable statements as a function of tags, tokens, and contextual elements associated with target verbs;

extracting features from at least one training statement based on the first set of rules;

training a pattern recognition module to identify one or more types of patterns in actionable statements based at least in part on the features; and

training an action verb module to identify dependency of the actionable statements based on at least some of the features;

generating an actionable statement identification model using the trained action verb module and the trained pattern recognition module, the actionable statement identification model including a plurality of target nodes, each node having associated therewith one or more active features and one or more corresponding weights of the one or more active features;

adaptively training the actionable statement model by at least one of promoting and demoting the weights corresponding to the active features associated with some or all of the plurality of target nodes,

wherein weights corresponding to the active features are promoted in response to determining a true positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond; and

wherein weights corresponding to the active features are demoted in response to determining a false positive match between active features of a particular statement within an electronic communication and the active features to which the weights correspond;

in response to determining the statement includes an actionable statement, predicting, from among a plurality of potential actionable statement classes, an actionable statement class of the actionable statement, wherein the prediction is based on a pattern represented in the statement and comprises;

computing a weight of each of the plurality of potential actionable statement classes based on the pattern represented in the statement; and

determining from among the plurality of potential actionable statement classes, an actionable statement class having an associated weight with a value higher than weights associated with other of the plurality of potential actionable statement classes is the predicted actionable statement class of the actionable statement; and

outputting the predicted actionable statement class of the actionable statement to a user;

receiving feedback from the user in response to outputting the predicted actionable statement class;

providing the user feedback to the actionable statement model to facilitate adaptively training the actionable statement model; and

wherein the predicted actionable statement class is selected from among a plurality of actionable statement classes comprising promises.

15. The system as recited in claim 14, comprising,

in response to determining the statement does not include an actionable verb, outputting an indication that the statement is not an actionable statement without performing the predicting;

in response to determining no single actionable statement class has an associated weight with a value higher than one or more weights associated with other of the actionable statement classes, evaluating a support score of one or more of the actionable statement classes; and

wherein the predicting comprises determining an actionable statement class having an associated weight with a value higher than one or more weights associated with other of the actionable statement classes;

wherein the predicting is based on at least a personalized actionable statement model and a shared actionable statement model; and

wherein the personalized actionable statement model is given priority over the shared actionable statement model;

wherein the action verb module is trained to identify dependency of actionable verb

statements based on determining whether an action verb is enclosed by another verb;

wherein weights corresponding to the active features are promoted in response to determining a true positive mismatch between active features of a particular statement within an electronic communication and the active features to which the weights correspond;

wherein promoting the weights corresponding to the active features associated with some or all of the plurality of target nodes comprises, for all active features At and corresponding weights w_t,i of a target node t among the plurality of target nodes, multiplying the corresponding weights w_t,i by a promotion constant at in response to determining that a sum of the weights w_t,i for the node t and set of active features A_t is less than a node threshold Q_t; and

wherein the plurality of target nodes comprise a “

promise”

node, a “

request”

node, and an “

other”

node.