Generating answer variants based on tables of a corpus

• US 10,331,684 B2
• Filed: 06/03/2016
• Issued: 06/25/2019
• Est. Priority Date: 06/03/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chaubal, Amrish V., Eyigoz, Kadriye E., Santos Moraes, Priscilla, Sinha, Ravi S.
• Primary Examiner(s): Morrison, Jay A
• Assistant Examiner(s): Caiado, Antonio J
• Application Number:
  US15/172,216
• Publication Number:
  US 20170351677A1
• Time in Patent Office: 1,117 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/345 Summarisation for human users

Abstract

Mechanisms are provided for implementing a candidate answer variant engine. The mechanisms receive an indication of a structured portion of content in a corpus, generate a plurality of groupings of elements of the structured portion of content, and generate, for each grouping of elements in the plurality of groupings of elements, and for each element in the grouping of elements, a corresponding grouping vector representation, corresponding to the element. The mechanisms, for each grouping vector representation of each grouping of elements in the plurality of groupings of elements perform a similarity measure calculation between the grouping vector representation and a vector representation of an input question, and select an element corresponding to the grouping vector representation for inclusion as a candidate answer variant based on results of the similarity measure calculation. The mechanisms perform question answering operations based on an analysis of one or more candidate answer variants.

Claims

1. A method, in a data processing system comprising a processor and a memory, the method comprising:

configuring the processor, based on instructions loaded into the memory and executed by the processor, to implement a candidate answer variant engine, that operates in conjunction with a Question Answering (QA) cognitive computing system to generate an answer output in response to the QA cognitive computing system receiving an input question from a client computing device, wherein configuring the processor further comprises configuring the processor to implement an element grouping engine and an element grouping similarity analysis engine of the candidate answer variant engine;

receiving, by the candidate answer variant engine, from a QA processing pipeline of the QA cognitive computing system based on computerized natural language processing of the input question, an indication of a structured portion of content in a corpus of electronic content processed by the QA cognitive computing system to answer input questions;

generating, by the element grouping engine of the candidate answer variant engine, a plurality of groupings of elements of the structured portion of content;

generating, by the element grouping engine of the candidate answer variant engine, for each grouping of elements in the plurality of groupings of elements, and for each element in the grouping of elements, a corresponding grouping vector representation, corresponding to the element;

for each grouping vector representation of each grouping of elements in the plurality of groupings of elements;

performing, by the element grouping similarity analysis engine of the candidate answer variant engine, a similarity measure calculation between the grouping vector representation and a vector representation of the input question; and

selecting, by the element grouping similarity analysis engine of the candidate answer variant engine, an element corresponding to the grouping vector representation for inclusion as a candidate answer variant in a candidate answer variant data structure based on results of the similarity measure calculation;

performing, by the QA cognitive computing system, question answering operations for answering the input question based on a cognitive analysis, by the QA processing pipeline of the QA cognitive computing system, of one or more candidate answer variants in the candidate answer variant data structure; and

outputting, by the QA cognitive computing system, a final answer to the client computing device based on results of performing the question answering operations.

2. The method of claim 1, wherein the structured portion of content comprises a table data structure present in a portion of content of the corpus, and wherein each grouping of elements in the plurality of grouping of elements comprises a triad of at least one row header, at least one column header, and a cell content.

3. The method of claim 1, wherein the structured portion of content is a structured portion of content corresponding to a candidate answer found in the corpus by a primary search operation of the QA cognitive computing system.

4. The method of claim 1, wherein performing the similarity measure calculation comprises calculating a cosine similarity between the grouping vector representation and the vector representation of the input question.

5. The method of claim 4, wherein the similarity measure calculation comprises calculating the cosine similarity between other elements of the grouping vector representation than an element of the grouping vector representation being considered for selection as a candidate answer variant from the grouping of elements.

6. The method of claim 5, wherein selecting the element corresponding to the grouping vector representation for inclusion as a candidate answer variant in the candidate answer variant data structure based on results of the similarity measure calculation comprises:

comparing a cosine similarity value calculated as the cosine similarity between the other elements of the grouping vector representation and the vector representation of the input question, with a threshold value; and

selecting the element being considered for selection as a candidate answer variant to be a candidate answer variant in response to the cosine similarity value having a predefined relationship with the threshold value.

7. The method of claim 1, wherein generating a plurality of groupings of elements of the structured portion of content comprises:

analyzing at least one of metadata or computer code associated with the structured portion of content to identify a structure of the structured portion of content;

identifying elements of the structure based on the analysis; and

generating groupings of elements based on the identified elements and the identified structured of the structured portion of content.

8. The method of claim 1, wherein the vector representation of the input question comprises a vector representation of one or more features of the input question identified by the natural language processing of the input question.

9. The method of claim 1, wherein the similarity measure calculation comprises at least one of a textual syntactic similarity measure calculation or textual semantic similarity measure calculation.

10. The method of claim 1, wherein performing question answering operations for answering the input question based on an analysis of one or more candidate answer variants in the candidate answer variant data structure comprises:

generating, for each candidate answer variant in the candidate answer variant data structure, a confidence score value indicating a confidence that a corresponding candidate answer variant is a correct answer for the input question;

ranking the candidate answer variants relative to one another, and to other candidate answers found during a primary search operation of a question answering (QA) system, based on the generated confidence scores to generate a ranked listing data structure; and

selecting the final answer from the ranked listing data structure.

11. A computer program product comprising a non-transitory computer readable medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

configure the computing device, based on instructions loaded into a memory and executed by a processor of the computing device, to implement a candidate answer variant engine, that operates in conjunction with a Question Answering (QA) cognitive computing system to generate an answer output in response to the QA cognitive computing system receiving an input question from a client computing device, wherein configuring the computing device further comprises configuring the computing device to implement an element growing engine and an element grouping similarity analysis engine of the candidate answer variant engine;

receive, by the candidate answer variant engine, from a QA processing pipeline of the QA cognitive computing system based on computerized natural language processing of the input question, an indication of a structured portion of content in a corpus of electronic content processed by the QA cognitive computing system to answer input questions an indication of a structured portion of content in a corpus;

generate, by the element grouping engine of the candidate answer variant engine, a plurality of groupings of elements of the structured portion of content;

generate, by the element grouping engine of the candidate answer variant engine, for each grouping of elements in the plurality of groupings of elements, and for each element in the grouping of elements, a corresponding grouping vector representation, corresponding to the element, of each of the other elements in the grouping of elements;

for each grouping vector representation of each grouping of elements in the plurality of groupings of elements;

perform, by the element grouping similarity analysis engine of the candidate answer variant engine, a similarity measure calculation between the grouping vector representation and a vector representation of the input question; and

select, by the element grouping similarity analysis engine of the candidate answer variant engine, an element corresponding to the grouping vector representation for inclusion as a candidate answer variant in a candidate answer variant data structure based on results of the similarity measure calculation;

perform, by the QA cognitive computing system, question answering operations for answering the input question based on a cognitive analysis, by the QA processing pipeline of the QA cognitive computing system, of one or more candidate answer variants in the candidate answer variant data structure; and

output, by the QA cognitive computing system a final answer to the client computing device based on results of performing the question answering operations.

12. The computer program product of claim 11, wherein the structured portion of content comprises a table data structure present in a portion of content of the corpus, and wherein each grouping of elements in the plurality of grouping of elements comprises a triad of at least one row header, at least one column header, and a cell content.

13. The computer program product of claim 11, wherein the structured portion of content is a structured portion of content corresponding to a candidate answer found in the corpus by a primary search operation.

14. The computer program product of claim 11, wherein performing the similarity measure calculation comprises calculating a cosine similarity between the grouping vector representation and the vector representation of the input question.

15. The computer program product of claim 14, wherein the similarity measure calculation comprises calculating the cosine similarity between other elements of the grouping vector representation than an element of the grouping vector representation being considered for selection as a candidate answer variant from the grouping of elements.

16. The computer program product of claim 15, wherein selecting the element corresponding to the grouping vector representation for inclusion as a candidate answer variant in the candidate answer variant data structure based on results of the similarity measure calculation comprises:

comparing a cosine similarity value calculated as the cosine similarity between the other elements of the grouping vector representation and the vector representation of the input question, with a threshold value; and

selecting the element being considered for selection as a candidate answer variant to be a candidate answer variant in response to the cosine similarity value having a predefined relationship with the threshold value.

17. The computer program product of claim 11, wherein generating a plurality of groupings of elements of the structured portion of content comprises:

analyzing at least one of metadata or computer code associated with the structured portion of content to identify a structure of the structured portion of content;

identifying elements of the structure based on the analysis; and

generating groupings of elements based on the identified elements and the identified structured of the structured portion of content.

18. The computer program product of claim 11, wherein the vector representation of the input question comprises a vector representation of one or more features of the input question identified by the natural language processing of the input question.

19. The computer program product of claim 11, wherein the similarity measure calculation comprises at least one of a textual syntactic similarity measure calculation or textual semantic similarity measure calculation.

20. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

configure the processor to implement a candidate answer variant engine, that operates in conjunction with a Question Answering (QA) cognitive computing system to generate an answer output in response to the QA cognitive computing system receiving an input question from a client computing device, wherein configuring the processor further comprises configuring the processor to implement an element grouping engine and an element grouping similarity analysis engine of the candidate answer variant engine;

receive, by the candidate answer variant engine, from a QA processing pipeline of the QA cognitive computing system based on computerized natural language processing of the input question, an indication of a structured portion of content in a corpus of electronic content processed by the QA cognitive computing system to answer input questions an indication of a structured portion of content in a corpus;

generate, by the element grouping engine of the candidate answer variant engine, a plurality of groupings of elements of the structured portion of content;

generate, by the element grouping engine of the candidate answer variant engine, for each grouping of elements in the plurality of groupings of elements, and for each element in the grouping of elements, a corresponding grouping vector representation, corresponding to the element, of each of the other elements in the grouping of elements;

for each grouping vector representation of each grouping of elements in the plurality of groupings of elements;

perform, by the element grouping similarity analysis engine of the candidate answer variant engine, a similarity measure calculation between the grouping vector representation and a vector representation of the input question; and

select, by the element grouping similarity analysis engine of the candidate answer variant engine, an element corresponding to the grouping vector representation for inclusion as a candidate answer variant in a candidate answer variant data structure based on results of the similarity measure calculation;

perform, by the QA cognitive computing system, question answering operations for answering the input question based on a cognitive analysis, by the QA processing pipeline of the QA cognitive computing system, of one or more candidate answer variants in the candidate answer variant data structure; and

output, by the QA cognitive computing system, a final answer to the client computing device based on results of performing the question answering operations.