Scoring candidates using structural information in semi-structured documents for question answering systems

• US 10,223,441 B2
• Filed: 09/24/2011
• Issued: 03/05/2019
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fan, James J., Ferrucci, David A.
• Primary Examiner(s): Nguyen, Thu N
• Application Number:
  US13/244,351
• Publication Number:
  US 20120089622A1
• Time in Patent Office: 2,719 Days
• Field of Search: None
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence

Abstract

A system, program product, and methodology automatically scores candidate answers to questions in a question and answer system. In the candidate answer scoring method, a processor device performs one or more of receiving one or more candidate answers associated with a query string, the candidates obtained from a data source having semi-structured content; identifying one or more documents with semi-structured content from the data source having a candidate answer; and for each identified document: extracting one or more entity structures embedded in the identified document; determining a number of the entity structures in the identified document that appear in the received input query; and, computing a score for a candidate answer in the document as a function of the number Overall system efficiency is improved by giving the correct candidate answers higher scores through leveraging context-dependent structural information such as links to other documents and embedded tags.

Claims

1. A computer-implemented method for automatically scoring candidate answers to questions in a question and answer system comprising the steps of:

receiving an input query string;

performing a query analysis upon said input query string to obtain query terms;

obtaining a candidate answer from at least one document in a data corpus using said query terms;

identifying one or more entity structures embedded in said at least one document;

extracting said one or more entity structures embedded in said at least one document, said embedded entity structures comprising user embedded tags or links to other documents;

determining a number of said entity structures having terms in said embedded tags or links that match query terms in the received input text query; and

,

computing a confidence score for said candidate answer as a function of said number of entity structures having terms in said embedded tags or links that match query terms in the query string,

wherein at least one of the steps of the method is performed by a processor device.

2. The computer-implemented method as in claim 1, further comprising:

accessing a table having entries, each entry including an identified document'"'"'s corresponding identifier information, and a corresponding value of entity identifiers from the one or more entity structures embedded in that identified document, wherein said determining a number of said entity structures comprises traversing said table to identify said number of associated entity structures associated with a document.

3. The computer-implemented method as in claim 2, wherein said table includes a hash look-up table.

4. The computer-implemented method as in claim 1, wherein said determining a number of said entity structures having terms in said embedded tags or links that match query terms comprises:

determining a similarity of the term in an embedded tag or link and said query terms; and

estimating a relevance of the candidate answer to the question based on the determined similarity.

5. The computer-implemented method as in claim 4, wherein said determining a similarity comprises:

applying a comparison function to generate a resulting comparison score, said candidate score comprising said comparison score.

6. The computer-implemented method as in claim 4, wherein said terms includes one of:

a text, a title of a document having further content, a Uniform resource locator (URL), or a Uniform Resource Indicator.

7. The computer-implemented method as in claim 6, further comprising:

accessing a further document by following an embedded link; and

for said further document;

extracting one or more embedded structured entities;

determining a number of said structured entities in said further document that match query terms in the received input text query; and

,

computing a further score for a candidate answer in said further document as a function of said number of said structured entities in said further document that match query terms in the received input text query.

8. A computer-implemented method for automatically scoring candidate answers to questions in a question and answer system comprising the steps of:

receiving one or more candidate answers associated with a query string, said candidates obtained from a data source having semi-structured content;

identifying one or more documents with one or more entity structures embedded therein from said data source having a candidate answer; and

for each identified document;

extracting one or more entity structures embedded in said identified document, said embedded entity structures comprising user embedded tags or links to other documents;

determining a number of said entity structures having terms in said embedded tags or links that match query terms in the received input query; and

,

computing a score for a candidate answer in said document as a function of said number of entity structures having terms in said embedded tags or links that match query terms in the query string,

wherein at least one of the steps of the method is performed by a processor device.

9. The computer-implemented method as in claim 8, further comprising:

accessing a table having entries, each entry including an identified document'"'"'s corresponding identifier information, and a corresponding value of entity identifiers from the one or more entity structures embedded in that identified document, wherein said determining a number of said entity structures comprises;

traversing said table to identify said number of associated entity structures associated with a document.

10. The computer-implemented method as in claim 8, wherein said determining a number of said entity structures having terms in said embedded tag or link that match query terms comprises:

determining a similarity of the term in an embedded tag or link and said query terms; and

estimating a relevance of the candidate answer to the question based on the determined similarity.

11. The computer-implemented method as in claim 10 wherein said determining a similarity comprises:

applying a comparison function to generate a resulting comparison score, said candidate score comprising said comparison score.