Lexical answer type confidence estimation and application

• US 8,600,986 B2
• Filed: 08/29/2012
• Issued: 12/03/2013
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: International Business Machines Corporation
• Inventors: Fan, James J., Ferrucci, David A., Gondek, David C., Kalyanpur, Aditya A., Lally, Adam P., Murdock, James W., Zadrozny, Wlodek W.
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/597,795
• Publication Number:
  US 20120323906A1
• Time in Patent Office: 461 Days
• Field of Search: None
• US Class Current: 707/728
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

A system, method and computer program product for automatically estimating the confidence of a detected LAT to provide a more accurate overall score for an obtained candidate answer. A confidence “score” or value of each detected LAT is obtained, and the system and method performs combining the confidence score with a degree of match between a LAT and an AnswerType of the candidate answer to provide improved overall score for the candidate answer.

Claims

1. A system for automatically generating answers to questions comprising:

a memory storage device;

one or more processor devices, each in communication to said memory device and configured for performing a method comprising;

determining a lexical answer type (LAT) associated with an input query;

computing a first score for said query LAT, wherein said first score is indicative of a quality of said query LAT;

obtaining a candidate answer to said input query from an information source;

determining an answer type (AT) for said candidate answer;

comparing said query LAT with said candidate answer AT;

computing a second score representing a degree of match between said compared query LAT with said candidate answer AT; and

combining said first score and said second score to provide a total score indicative of a quality of said candidate answer.

2. The system of claim 1, wherein said one or more processor devices for computing a first score for said query LAT is configure to carry out said computing step by:

extracting a set of features describing said determined query LAT; and

,

applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said confidence value.

3. The system of claim 2, wherein said one or more processor devices are configured to apply one or more of:

a logistic regression model or a decision tree model constructed manually or automatically from prior sets of training instances.

4. The system of claim 2, wherein a feature of said extracted feature set comprises a pattern rule, said one or more processor devices for extracting a set of features further performs:

identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs.

5. The system of claim 2, wherein a feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said one or more processor devices for extracting a set of features further performs:

checking prior instances of a detected LAT word; and

computing a LAT word frequency based on said prior instances.

6. The system of claim 2, wherein a feature of said extracted feature set comprises a part of speech of the candidate LAT, said one or more processor devices for extracting a set of features further performs:

obtaining a parse tree data structure associated with said query; and

traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT.

7. The system of claim 2, wherein a feature of said extracted feature set comprises a co-reference information, said one or more processor devices for extracting a set of features further performs:

determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.

8. The system of claim 2, wherein said one or more processor devices for computing a second score is configure to carry out said computing step by:

matching said candidate answer against instances in a corpus or a knowledge base;

retrieving a type associated with an instance in the corpus or the knowledge base; and

,

matching said query LAT with said received type to produce a type coercion (TyCor) value, said second score comprising said TyCor value.

9. The system of claim 8, wherein a query has n candidate LATs, said one or more processor devices for combining to provide said total score performs a summation according to:

(confidence score_LAT1)(TyCor_LAT1)+(confidence score_LAT2)(TyCor_LAT2)+ . . . +(confidence score_LATn)(TyCor_LATn)

wherein confidence score_LAT1, confidence score_LAT2, and confidence score_LATn are produced confidence values representing estimated accuracies of each determined query LAT, and TyCor_LAT1, TyCor_LAT2, and TyCor_LATn are the produced type coercion (TyCor) values of a candidate answer with each LAT.

10. A computer program product for automatically generating answers to questions, the computer program device comprising a storage medium readable by a processing circuit and storing instructions run by the processing circuit for performing a method, the method comprising:

determining a lexical answer type (LAT) associated with an input query;

computing a first score for said query LAT, wherein said first score is indicative of a quality of said query LAT;

obtaining a candidate answer to said input query from an information source;

determining an answer type (AT) for said candidate answer;

comparing said query LAT with said candidate answer AT;

computing a second score representing a degree of match between the compared query LAT with said candidate answer AT; and

combining said first score and said second score to provide a total score indicative of a quality of said candidate answer.

11. The computer program product for automatically generating answers to questions of claim 10, wherein said computing a first score for each LAT associated with said input query comprises:

extracting a set of features describing said computed query LAT; and

,

applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said LAT confidence value.

12. The computer program product for automatically generating answers to questions of claim 11, wherein:

a first feature of said extracted feature set comprises a pattern rule, said extracting a set of features comprising identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs; and

wherein,

a second feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said extracting a set of features further comprising checking prior instances of a detected LAT word and computing a LAT word frequency based on said prior instances; and

wherein,

a third feature of said extracted feature set comprises a part of speech of the candidate LAT, said extracting a set of features further comprising obtaining a parse tree data structure associated with said query; and

traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT; and

, wherein,

a fourth feature of said extracted feature set comprises a co-reference information, said extracting a set of features further comprising determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.