Corroborating facts extracted from multiple sources

• US 8,682,913 B1
• Filed: 03/31/2005
• Issued: 03/25/2014
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Betz, Jonathan T.
• Primary Examiner(s): LIN, SHEW FEN
• Application Number:
  US11/097,688
• Time in Patent Office: 3,281 Days
• Field of Search: 707/755, 707/738, 707/701, 707/804
• US Class Current: 707/755
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method and system for corroborating facts extracted as attribute-value pairs from multiple sources. Facts associated with a common subject are identified and extracted from multiple sources. From the extracted facts, attributes satisfying a first corroboration requirement are identified. Attribute-value pairs satisfying a second corroboration requirement and having any of the identified attributes that satisfy the first corroboration requirement are identified. Predefined actions may be performed with respect to the identified attribute-value pairs and/or with respect to their status in a fact database.

Claims

1. A computer-implemented method for corroborating facts, comprising:

at a computer system including one or more processors and memory storing one or more programs, the one or more processors executing the one or more programs to perform the operations of;

identifying a plurality of source documents including facts associated with a common subject;

identifying, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborating a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, by determining that the respective attribute-value pair meets one or more predefined corroboration requirements; and

in response to corroboration of the fact, updating a status of the respective attribute value pair in a fact repository;

wherein corroborating the fact includes;

determining that a respective attribute of the respective attribute-value pair meets a first corroboration requirement; and

determining that the respective attribute-value pair meets a second corroboration requirement.

2. The method of claim 1, wherein the first corroboration requirement with respect to the respective attribute is based on a number of source documents having at least one attribute-value pair associated with the common subject, wherein the attribute of the attribute-value pair comprises the respective attribute.

3. The method of claim 1, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the respective attribute-value pair associated with the common subject.

4. The method of claim 1, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the particular attribute-value pair associated with the common subject and respective importance metrics of the source documents.

5. The method of claim 1, further comprising reducing the first corroboration requirement with respect to an attribute of an attribute-value pair in a source document, when the source document includes at least a first predefined number of attribute-value pairs meeting the second corroboration requirement.

6. The method of claim 1, further comprising reducing the second corroboration requirement with respect to a first attribute-value pair in a source document, when the source document includes at least a second predefined number of other attribute-value pairs meeting the second corroboration requirement.

7. A computer-implemented method for corroborating facts, comprising:

at a computer system including one or more processors and memory, the memory of the computer system storing one or more programs to be executed by the one or more processors of the computer system,

identifying a plurality of source documents including facts associated with a common subject;

identifying, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborating a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, by;

identifying a candidate attribute from the identified attribute-value pairs;

determining a first score based on the candidate attribute;

determining that the first score exceeds a first predefined threshold;

identifying a candidate value associated with the candidate attribute;

determining a second score based on the candidate attribute and the candidate value;

determining that the second score exceeds second scores of other candidate values by at least a predefined margin; and

in response to corroboration of the fact, updating a status of the respective attribute-value pair in a fact repository.

8. A system for corroborating facts, comprising:

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions to;

identifying a plurality of source documents including facts associated with a common subject;

identify, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborate a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, by determining that the respective attribute-value pair meets one or more predefined corroboration requirements; and

in response to corroboration of the fact, updating a status of the respective attribute value pair in a fact repository;

wherein the instructions to corroborate a fact includes instructions to;

determine that a respective attribute of the respective attribute-value pair meets a first corroboration requirement; and

determining that the respective attribute-value pair meets a second corroboration requirement.

9. The system of claim 8, wherein the first corroboration requirement with respect to the respective attribute is based on a number of source documents having at least one attribute-value pair associated with the common subject, wherein the attribute of the attribute-value pair comprises the respective attribute.

10. The system of claim 8, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the respective attribute-value pair associated with the common subject.

11. The system of claim 8, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the particular attribute-value pair associated with the common subject and respective importance metrics of the source documents.

12. The system of claim 8, wherein the instructions to corroborate a fact further includes instructions to reduce the first corroboration requirement with respect to an attribute of an attribute-value pair in a source document, when the source document includes at least a first predefined number of attribute-value pairs meeting the second corroboration requirement.

13. The system of claim 8, wherein instructions to corroborate a fact further includes instructions to reduce the second corroboration requirement with respect to a first attribute-value pair in a source document, when the source document includes at least a second predefined number of other attribute-value pairs meeting the second corroboration requirement.

14. A system for corroborating facts, comprising:

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions to;

identify a plurality of source documents including facts associated with a common subject;

identify, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborate a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, wherein the instructions to corroborate the fact include instructions to;

identify a candidate attribute from the identified attribute-value pairs;

determine a first score based on the candidate attribute;

determining that the first score exceeds a first predefined threshold;

identify a candidate value associated with the candidate attribute;

determine a second score based on the candidate attribute and the candidate value;

determine that the second score exceeds second scores of other candidate values by at least a predefined margin; and

in response to corroboration of the fact, update a status of the respective attribute-value pair in a fact repository.

15. A computer program product for use in conjunction with a computer system, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:

identifying a plurality of source documents including facts associated with a common subject;

identifying, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborating a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, by determining that the respective attribute-value pair meets one or more predefined corroboration requirements; and

in response to corroboration of the fact, updating a status of the respective attribute value pair in a fact repository;

wherein corroborating the fact includes;

determining that a respective attribute of the respective attribute-value pair meets a first corroboration requirement; and

determining that the respective attribute-value pair meets a second corroboration requirement.

16. The computer program product of claim 15, wherein the first corroboration requirement with respect to the respective attribute is based on a number of source documents having at least one attribute-value pair associated with the common subject, wherein the attribute of the attribute-value pair comprises the respective attribute.

17. The computer program product of claim 15, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the respective attribute-value pair associated with the common subject.

18. The computer program product of claim 15, wherein the second corroboration requirement with respect to the respective attribute-value pair is based on a number of source documents having the respective attribute-value pair associated with the common subject and respective importance metrics of the source documents.

19. The computer program product of claim 15, further comprising reducing the first corroboration requirement with respect to an attribute of an attribute-value pair in a source document, when the source document includes at least a first predefined number of attribute-value pairs meeting the second corroboration requirement.

20. The computer program product of claim 15, further comprising reducing the second corroboration requirement with respect to a first attribute-value pair in a source document, when the source document includes at least a second predefined number of other attribute-value pairs meeting the second corroboration requirement.

21. A computer program product for use in conjunction with a computer system, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:

identifying a plurality of source documents including facts associated with a common subject;

identifying, from the plurality of source documents, one or more attribute-value pairs associated with the common subject;

corroborating a fact, comprising a respective attribute-value pair of the one or more attribute-value pairs associated with the common subject, by;

identifying a candidate attribute from the identified attribute-value pairs;

determining a first score based on the candidate attribute;

determining that the first score exceeds a first predefined threshold;

identifying a candidate value associated with the candidate attribute;

determining a second score based on the candidate attribute and the candidate value;

determining that the second score exceeds second scores of other candidate values by at least a predefined margin; and

in response to corroboration of the fact, updating a status of the respective attribute-value pair in a fact repository.