Unsupervised template extraction

• US 10,558,760 B2
• Filed: 10/23/2017
• Issued: 02/11/2020
• Est. Priority Date: 07/28/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hudson, Eddy, Kaufmann, Joseph M., Parameswaran, Niyati
• Primary Examiner(s): Adesanya, Olujimi A
• Application Number:
  US15/791,009
• Publication Number:
  US 20190034410A1
• Time in Patent Office: 841 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06V 30/1985 Syntactic analysis, e.g. using a grammatical approach syntactic image representation G06V30/18171
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Abstract

An approach is provided that improves a question answering (QA) computer system by automatically generating relationship templates. Event patterns are extracted from data in a corpus utilized by the QA computer system. The extracted event patterns are analyzed with the analysis resulting in a number of clusters of related event patterns. Relationship templates are then created from the plurality of clusters of related event patterns and these relationship templates are then utilized to visually interact with the corpus.

Claims

1. A method implemented by an information handling system that includes a memory and a processor, that improves a question answering (QA) computer system by automatically generating relationship templates, the method comprising:

extracting a plurality of event patterns corresponding to a plurality of events from data in a corpus utilized by the QA computer system;

analyzing the extracted event patterns resulting in a plurality of clusters of related event patterns;

creating one or more relationship templates from the plurality of clusters of related event patterns, wherein a first one of the or more relationship templates comprises a first set of the plurality of events included in a first one of the plurality of clusters;

displaying the first relationship template as a first graphical representation on a display, wherein the first graphical representation displays the first set of events; and

in response to receiving an input selection that selects the first graphical representation, displaying a set of second graphical representations on the display that represent a set of roles within the first relationship template.

2. The method of claim 1 further comprising:

expanding the corpus utilized by the QA system, wherein the expanding further comprises;

receiving a plurality of text data outside the corpus;

retrieving a plurality of sub-clusters, wherein each of the sub-clusters is based on the created one or more relationship templates; and

expanding a plurality of input arguments included in the event patterns by using a set of related event patterns found in the plurality of text data outside the corpus, wherein the method further comprises;

matching portions of the corpus with the cluster that includes the related event pattern corresponding to the plurality of input arguments.

3. The method of claim 1 wherein the extracting further comprises:

clustering the event patterns using hierarchical agglomerative clustering techniques so that the event patterns that are closer together are clustered in the same event pattern, wherein each cluster of event patterns forms the basis of one of the one or more relationship templates.

4. The method of claim 1 wherein the analyzing further comprises:

converting an argument in each of the extracted event patterns into vectors by using distributional semantics, wherein the converting results in a plurality of word values each corresponding to one of a plurality of words in the extracted event pattern;

calculating a similarity score between the plurality of words based on the word values pertaining to the respective words; and

identifying a plurality of sets of similar words based on a comparison of the calculated similarities.

5. The method of claim 4 further comprising:

selecting each of the arguments and a successive argument to the selected argument; and

calculating the similarity score between the selected argument and the successive argument.

6. The method of claim 5 further comprising:

performing the selecting and calculating on each pair of arguments and storing the similarity scores of all of the pairs;

comparing the similarity scores to a threshold;

in response to the threshold revealing a high similarity score between the arguments in one or more of the pairs of arguments;

clustering the event pattern corresponding with the selected arguments with the event pattern corresponding with the selected arguments'"'"' respective successive arguments.

7. The method of claim 5 wherein the similarity is calculated using a cosine similarity algorithm.