Lexical answer type confidence estimation and application

• US 8,510,296 B2
• Filed: 09/23/2011
• Issued: 08/13/2013
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: International Business Machines Corporation
• Inventors: Fan, James J., Ferrucci, David A., Gondek, David C., Kalyanpur, Aditya A., Lally, Adam P., Murdock, James W., Zadrozny, Wlodek W.
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/242,290
• Publication Number:
  US 20120078890A1
• Time in Patent Office: 690 Days
• Field of Search: None
• US Class Current: 707/728
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F

Abstract

A system, method and computer program product for automatically estimating the confidence of a detected LAT to provide a more accurate overall score for an obtained candidate answer. A confidence “score” or value of each detected LAT is obtained, and the system and method performs combining the confidence score with a degree of match between a LAT and an AnswerType of the candidate answer to provide improved overall score for the candidate answer.

Claims

1. A computer-implemented method for automatically generating answers to questions comprising the steps of:

determining a lexical answer type (LAT) associated with an input query;

computing a first score for said input query LAT, wherein said first score is indicative of a quality of said query LAT;

obtaining a candidate answer to said input query from an information source;

determining an answer type (AT) for said candidate answer;

comparing said query LAT with said candidate answer AT;

computing a second score representing a degree of match between said compared query LAT with said candidate answer AT; and

combining said first score and said second score to provide a total score indicative of a quality of said candidate answer,

wherein a processing device automatically performs one or more of said determining a query LAT, computing said first score and second score, obtaining candidate answers, determining a AT, comparing and combining.

2. The computer-implemented method of claim 1, wherein said computing a first score for said query LAT associated with said input query comprises:

extracting a set of features describing said determined query LAT; and

,

applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said confidence value.

3. The computer-implemented method of claim 2, wherein said applied model is one or more of:

a logistic regression model or a decision tree model constructed manually or automatically from prior sets of training instances.

4. The computer-implemented method of claim 2, wherein a feature of said extracted feature set comprises a pattern rule, said extracting a set of features comprising:

identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs.

5. The computer-implemented method of claim 2, wherein a feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said extracting a set of features further comprising:

checking prior instances of a detected LAT word; and

computing a LAT word frequency based on said prior instances.

6. The computer-implemented method of claim 2, wherein a feature of said extracted feature set comprises a part of speech of the candidate LAT, said extracting a set of features further comprising:

obtaining a parse tree data structure associated with said query; and

traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT.

7. The computer-implemented method of claim 2, wherein a feature of said extracted feature set comprises a co-reference information, said extracting a set of features further comprising:

determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.

8. The computer-implemented method of claim 2, wherein one or more features of said extracted feature set are extracted in parallel.

9. The computer-implemented method of claim 2, wherein said computing a second score representing a degree of match between said compared query LAT with said candidate answer AT includes:

matching said candidate answer against instances in a corpus or a knowledge base;

retrieving a type associated with an instance in the corpus or the knowledge base; and

,

matching said query LAT with said retrieved type to produce a type coercion (TyCor) value, said second score comprising said TyCor value.

10. The computer-implemented method of claim 9, wherein a query has up to n candidate LATs, said combining to provide said total score is a summation according to:

(confidence score_LAT1)(TyCor_{LAT 1})+(confidence score_LAT2)(TyCor_{LAT 2})+ . . . +(confidence score_LATn)(TyCor_{LAT n})

wherein confidence score_LAT1, confidence score_LAT2, and confidence score_LATn are produced confidence values representing estimated accuracies of each determined LAT of said query, and TyCor_LAT1, TyCor_LAT2, and TyCor_LATn are the produced type coercion (TyCor) values of a candidate answer with each LAT.