FACT CHECKING USING AND AIDING PROBABILISTIC QUESTION ANSWERING

• US 20120078826A1
• Filed: 09/28/2011
• Published: 03/29/2012
• Est. Priority Date: 09/29/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ferrucci, David A., Kalyanpur, Aditya A., Lally, Adam P., Gondek, David C., Patwardham, Siddharth
• Granted Patent: US 8,972,321 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system, a method and a computer program product for verifying a statement are provided. The system is configured to receive a statement. The system is configured to decompose the received statement into one or more sets of question and answer pairs. The system is configured to determine a confidence value of each answer in the one or more question and answer pair sets. The system is configured to combine the determined confidence values. The combined confidence values represent a probability that the received statement is evaluated as true.

Claims

1. A method for verifying a statement, the method comprising:

receiving, at a first computing system including at least one memory device and at least one processor connected to the memory device, a statement;

decomposing the received statement, by the first computing system, into one or more sets of question and answer pairs;

determining, by the first computing system, a confidence value of each answer in the one or more question and answer pair sets; and

combining, by the first computing system, the determined confidence values, the combined confidence values representing a probability that the received statement is evaluated as true.

2. The method according to claim 1, further comprising:

identifying words and phrases in the received statement;

replacing the identified words and phrases in the received statement with one or more words that can be used to create questions in the one or more question and answer pair sets; and

indicating the identified words and phrases in the received statement as answers in the one or more question and answer pair sets.

3. The method according to claim 1, wherein the decomposing further comprises:

identifying one or more of;

at least one clause in the received statement, at least one syntactic label in the received statement, at least one conjunction in the received statement, configuration information in the received statement, at least one pre-modifier or post-modifier in the received statement, and at least one relative qualifier in the received statement;

separating the received statement according to one or more of;

the identified at least one clause, the identified at least one syntactic label, the identified at least one conjunction, the identified configuration information, the identified at least one pre-modifier or post-modifier, and the identified at least one relative qualifier.

4. The method according to claim 2, wherein the one or more words that can be used to create the questions include one or more of:

who, why, when, what, where, and how.

5. The method according to claim 1, wherein the step of determining includes:

running a search engine or querying a database with one or more questions of the one or more question and answer pair sets;

obtaining one or more candidate responses from the search engine or the database; and

comparing the obtained candidate responses to one or more answers of the one or more question and answer pair sets.

6. The method according to claim 1, wherein the step of determining includes:

running a second computing system with one or more questions of the one or more question and answer pair sets;

obtaining one or more candidate responses from the second computing system; and

comparing the obtained candidate responses to one or more answers of the one or more question and answer pair sets.

7. The method according to claim 5, wherein the second computing system is an artificial intelligence computer.

8. The method according to claim 5, wherein if an obtained candidate response includes a compared answer in the one or more question and answer pair sets or if the compared answer includes the obtained candidate response, the compared answer receives a confidence value higher than a pre-determined threshold value.

9. The method according to claim 5, wherein if an obtained candidate response does not include a compared answer in the one or more question and answer pairs and if the compared answer does not include the obtained candidate response, the compared answer receives a confidence value lower than a pre-determined threshold value.

10. The method according to claim 1, wherein the step of combining includes:

multiplying each said confidence value.

11. The method according to claim 1, wherein the step of combining includes:

fitting the determined confidence values to training data in order to obtain weights associated with the determined confidence values;

obtaining at least one mathematical model including the obtained weights; and

using the at least one obtained mathematical model in order to calculate the probability that the received statement is evaluated as true.

12. The method according to claim 11, wherein the machine learning technique includes one or more of:

a Bayesian network, supervised learning, unsupervised learning, and decision tree.

13. The method according to claim 11, wherein the regression technique includes one or more of:

a linear regression, a logistic regression, a non-linear regression.

14. A system for verifying a statement, the system comprising:

a first computing system including at least one memory device and at least one processor connected to the memory device,

wherein the first computing system is configured to;

receive a statement;

decompose the received statement into one or more sets of question and answer pairs;

determine a confidence value of each answer in the one or more question and answer pair sets; and

combine the determined confidence values, the combined confidence values representing a probability that the received statement is evaluated as true.

15. The system according to claim 14, further comprising:

identify words and phrases in the received statement;

replace the identified words and phrases in the received statement with one or more words that can be used to create questions in the one or more question and answer pair sets; and

indicate the identified words and phrases in the received statement as answers in the one or more question and answer pair sets.

16. The system according to claim 14, wherein to decompose the received statement, the first computing system is configured to identify one or more of:

at least one clause in the received statement, at least one syntactic label in the received statement, at least one conjunction in the received statement, configuration information in the received statement, at least one pre-modifier or post-modifier in the received statement, and at least one relative qualifier in the received statement, and

the first computing system is further configured to separate the received statement according to one or more of;

the identified at least one clause, the identified at least one syntactic label, the identified at least one conjunction, the identified configuration information, the identified at least one pre-modifier or post-modifier, and the identified at least one relative qualifier.

17. The system according to claim 15, where in the one or more words that can be used to create the questions include one or more of:

who, why, when, what, where, and how.

18. The system according to claim 14, wherein to determine the confidence value, the first computing system is configured to:

run a search engine or querying a database with one or more questions of the one or more question and answer pair sets;

obtain one or more candidate responses from the search engine or the database; and

compare the obtained candidate responses to one or more answers of the one or more question and answer pair sets.

19. The system according to claim 14, wherein to determine the confidence value, the first computing system is configured to:

run a second computing system with one or more questions of the one or more question and answer pair sets;

obtain one or more candidate responses from the second computing system; and

compare the obtained candidate responses to one or more answers of the one or more question and answer pair sets.

20. The system according to claim 14, wherein to combine the determined confidence values, the first computing system is configured to:

multiply each said confidence value.

21. The system according to claim 14, wherein to combine the determined confidence values, the first computing system is configured to:

fit the determined confidence values to training data in order to obtain weights associated with the determined confidence values;

obtain at least one mathematical model including the obtained weights; and

use the at least one obtained mathematical model in order to calculate the probability that the received statement is evaluated as true.

22. A computer program product for verifying a statement, the computer program product comprising a non-transitory storage medium readable by a processing circuit and storing instructions run by the processing circuit for performing a method, the method comprising:

receiving, at a first computing system including at least one memory device and at least one processor connected to the memory device, a statement;

decomposing the received statement, by the first computing system, into one or more sets of question and answer pairs;

determining, by the first computing system, a confidence value of each answer in the one or more question and answer pair sets; and

combining, by the first computing system, the determined confidence values, the combined confidence values representing a probability that the received statement is evaluated as true.

23. The computer program product according to claim 22, wherein the step of determining includes:

running a second computing system with one or more questions of the one or more question and answer pair sets;

obtaining one or more candidate responses from the second computing system; and

comparing the obtained candidate responses to one or more answers of the one or more question and answer pair sets.

24. A method for determining a confidence value of an answer in a question and answer pair set, the method comprising:

performing one or more of;

running a search engine with a question corresponding to the answer in the question and answer pair set, querying a database with the corresponding question, and running a computing system with the corresponding question;

obtaining a candidate response from one or more of;

the search engine, database and the computing system; and

comparing the obtained candidate response against the answer in the question and answer pair set.

25. The method according to claim 24, wherein the step of the comparing includes:

evaluating whether if the obtained candidate response includes the answer or whether the answer includes the obtained candidate response.