EVIDENCE EVALUATION SYSTEM AND METHOD BASED ON QUESTION ANSWERING

• US 20110066587A1
• Filed: 09/17/2009
• Published: 03/17/2011
• Est. Priority Date: 09/17/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ferrucci, David A., Zadrozny, Wlodek W.
• Granted Patent: US 8,280,838 B2
• Time in Patent Office: Days
• US Class Current: 706/54
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

An evidence evaluation method and system based on question answering converts a report of analyzed information and/or a model of information into a collection of questions, determines answers for the collection of questions. A fact in the report is marked as being supported if one or more of the answers for the collection of questions support the fact. A fact in the report of analyzed facts is marked as being refuted if one or more of the answers for the collection of questions refute the fact. The method and system also may collect the answers as evidence and add the evidence to the model of information to create an updated model of information. The steps may be repeated using the updated report and updated model.

Claims

1. An evidence evaluation method based on question answering, comprising:

converting information into a collection of questions by a processor ;

determining answers to the collection of questions;

marking a fact in the information as being supported if one or more of the answers to the collection of questions support the fact;

marking a fact in the information as being refuted if one or more of the answers to the collection of questions refute the fact;

collecting evidence that support the answers; and

adding the evidence to a model of information to create an updated model of information.

2. The method of claim 1, wherein the information includes a report prepared from analyzing data.

3. The method of claim 1, wherein the information includes the model of information.

4. The method of claim 1, wherein the model of information includes structured data.

5. The method of claim 2, wherein the report includes natural language text.

6. The method of claim 1, wherein the step of marking a fact in the information as being supported further includes annotating the fact with metadata describing supporting answer for the fact.

7. The method claim 1, wherein the evidence include lack of support if one or more of the answers to the collection of questions refute the fact.

8. The method of claim 1, wherein an updated report is created by analyzing the updated model.

9. The method of claim 8, wherein the steps of converting, determining, marking, collecting and adding are repeated with the updated report.

10. The method of claim 1, wherein the steps of converting, determining, marking, collecting and adding are repeated with the updated model.

11. The method of claim 1, wherein the steps are performed as a background process as analysis is performed to create the information.

20. The program storage device claim 1, wherein the evidence includes lack of support if one or more of the answers to the collection of questions refute the fact.

12. A system for an evidence evaluation method based on question answering, comprising:

an editor operable to allow entering of information;

an editor plug-in operable to automatically convert the information into a collection of questions;

question answering processing module operable to determine answers for the collection of questions;

said editor plug-in further operable to mark a fact in the information as being supported if one or more of the answers for the collection of questions support the fact and marking a fact in the information as being refuted if one or more of the answers for the collection of questions refute the fact, said editor plug-in further operable to collect the answers as evidence and add the evidence to the model of information to create an updated model of information.

13. The system of claim 12, wherein the information includes a report.

14. The system of claim 12, wherein the information includes a model of information, the model including structured data.

15. The system of claim 12, further including:

reasoning and representation processing module operable to convert external data into structured data, wherein the question answering processing module uses the converted external data to determine answers to the collection of questions.

16. The system of claim 15, wherein the external data includes data from one or more of an information database, or Internet, or combinations thereof.

17. The system of claim 12, wherein the updated model of information is used to further as the information for forming a collection of questions.

18. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform an evidence evaluation method based on question answering, comprising:

converting information into a collection of questions by a processor;

determining answers to the collection of questions;

marking a fact in the information as being supported if one or more of the answers to the collection of questions support the fact;

marking a fact in the information as being refuted if one or more of the answers to the collection of questions refute the fact;

collecting evidence that support the answers; and

adding the evidence to a model of information to create an updated model of information.

19. The program storage device of claim 18, wherein the step of marking a fact in the information as being supported further includes annotating the fact with metadata describing supporting answer for the fact.