UTILIZING FAILURES IN QUESTION AND ANSWER SYSTEM RESPONSES TO ENHANCE THE ACCURACY OF QUESTION AND ANSWER SYSTEMS
First Claim
1. A computerized device for enhancing the accuracy of a question-answer system, comprising:
- a question-answer system comprising software for performing a plurality of question answering processes;
a receiver receiving a question into said question-answer system;
a processor connected to said question-answer system, said processor;
generating a plurality of candidate answers to said question, determining a confidence score for each of said plurality of candidate answers, evaluating sources of evidence used to generate said plurality of candidate answers,identifying missing information from a corpus of data, said missing information comprising any information that improves a confidence score for a candidate answer,generating at least one follow-on inquiry based on said missing information; and
a network interface outputting said at least one follow-on inquiry to external sources separate from said question-answer system to obtain responses to said at least one follow-on inquiry and receiving from said external sources at least one response to said at least one follow-on inquiry, andsaid processor inputting said at least one response into said corpus of data.
1 Assignment
0 Petitions
Accused Products
Abstract
A computerized device for enhancing the accuracy of a question-answer system. The computerized device comprises a question-answer system comprising software for performing a plurality of question answering processes. A receiver receives a question into the question-answer system. A processor that generates a plurality of candidate answers to the question is connected to the question-answer system. The processor determines a confidence score for each of the plurality of candidate answers. The processor evaluates sources of evidence used to generate the plurality of candidate answers. The processor identifies missing information from a corpus of data. The missing information comprises any information that improves a confidence score for a candidate answer. The processor generates at least one follow-on inquiry based on the missing information. A network interface outputs the at least one follow-on inquiry to external sources separate from the question-answer system.
75 Citations
25 Claims
-
1. A computerized device for enhancing the accuracy of a question-answer system, comprising:
-
a question-answer system comprising software for performing a plurality of question answering processes; a receiver receiving a question into said question-answer system; a processor connected to said question-answer system, said processor; generating a plurality of candidate answers to said question, determining a confidence score for each of said plurality of candidate answers, evaluating sources of evidence used to generate said plurality of candidate answers, identifying missing information from a corpus of data, said missing information comprising any information that improves a confidence score for a candidate answer, generating at least one follow-on inquiry based on said missing information; and a network interface outputting said at least one follow-on inquiry to external sources separate from said question-answer system to obtain responses to said at least one follow-on inquiry and receiving from said external sources at least one response to said at least one follow-on inquiry, and said processor inputting said at least one response into said corpus of data. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A computer system for enhancing the accuracy of a question answering system, comprising:
-
an automated question answering (QA) system comprising; a corpus of data, a processor, said processor having software for performing a plurality of question answering processes; a receiver; and a network interface, said receiver receiving a question into said automated QA system, said processor generating a plurality of candidate answers to said question, said processor evaluating sources of evidence used to generate said plurality of candidate answers to identify marginal evidence, said marginal evidence contributing only partially to a candidate answer, said processor determining a confidence score for each of said plurality of candidate answers, said processor identifying information not provided by said marginal evidence that would enable said QA system to further develop said confidence score, said processor automatically generating at least one follow-on inquiry based on said information, said network interface outputting said at least one follow-on inquiry to external sources separate from said QA system and receiving from said external sources at least one response to said at least one follow-on inquiry, said processor inputting at least one response into said corpus of data, and said processor automatically developing additional logical rules and additional evidence for said automated QA based on said at least one response to said at least one follow-on inquiry. - View Dependent Claims (9, 10, 11, 12, 13)
-
-
14. A question answering (QA) system comprising:
-
a processor; an evidence analysis module, said evidence analysis module being connected to said processor; a first interface connected to said processor; a second interface connected to said processor; and a corpus of data connected to said evidence analysis module, said first interface receiving a first question to be answered by said QA system, said processor creating a collection of candidate answers to said first question from said corpus of data, each said candidate answer having supporting evidence and a confidence score generated by said processor, said evidence analysis module producing a second question based on said supporting evidence, said processor presenting said second question through said second interface to one or more external sources separate from said QA system to obtain responses to said second question, said processor receiving at least one response or knowledge item from said one or more external sources through said second interface, said processor inputting said at least one response or knowledge item into said corpus of data, and said processor automatically developing additional logical rules and additional evidence for said QA based on said at least one response or knowledge item. - View Dependent Claims (15, 16, 17, 18)
-
-
19. A non-transitory computer readable storage medium readable by a computerized device, said computerized device comprising a question-answer system comprising question answering processes, said non-transitory computer readable storage medium storing instructions executable by said computerized device to perform a method comprising:
-
identifying missing information from a corpus of data, said missing information comprising any information that improves a confidence for a candidate answer to a question; generating a follow-on inquiry, said follow-on inquiry prompting for said missing information to be provided; outputting said follow-on inquiry to an external source; receiving a response to said follow-on inquiry from said external source; and adding said response to said corpus of data. - View Dependent Claims (20, 21, 22, 23, 24, 25)
-
Specification