Anchor text summarization for corroboration

• US 9,208,229 B2
• Filed: 03/31/2006
• Issued: 12/08/2015
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Betz, Jonathan T., Zhao, Shubin
• Primary Examiner(s): KINSAUL, DANIEL W
• Application Number:
  US11/394,552
• Publication Number:
  US 20070143282A1
• Time in Patent Office: 3,539 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A system and method for corroborating a set of facts. If the anchor text of the references to a document matches the name of a set of facts, the referenced document is used to corroborate the set of facts. By analyzing the anchor text of the references to the document, the system is capable of determining if a document is relevant to the set of facts. These documents can then be used to corroborate or refute the facts, thereby improving their overall quality.

Claims

1. A computer-implemented method for corroborating a set of facts included in a fact repository, the method comprising:

at a back end computer system including one or more processors and memory storing one or more programs, the one or more processors executing the one or more programs to perform the operations of;

identifying a set of facts associated with an object, the set of facts having been previously extracted from multiple documents, each fact comprising an attribute-value pair, including a fact attribute type and a fact value, and wherein the object is associated with an entity having a name fact attribute type;

receiving a first document not included in the multiple documents and a reference to the first document, said reference comprising user-viewable anchor text extracted from a second document;

determining that the user-viewable anchor text matches the name of the entity associated with the object;

determining that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document; and

responsive to determining that the user-viewable anchor text matches the name of the entity associated with the object and that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document, corroborating the set of facts using the first document;

the corroborating comprising;

identifying one or more facts in the first document, each identified fact having an attribute-value pair; and

comparing a respective attribute-value pair of the set of facts to an identified attribute-value pair in the first document; and

updating the set of facts in accordance with the corroborating, wherein the updating includes one or both of storing an attribute-value pair from the first document in the set of facts in association with the object or adjusting a status of an attribute-value pair of the set of facts.

2. The method of claim 1, wherein receiving the reference to the document comprises receiving a set of candidate anchor texts, and wherein determining if the anchor text matches the name of the entity associated with the object comprises:

selecting an anchor text from the set of candidate anchor texts; and

determining if the selected anchor text matches the name of the entity associated with the object.

3. The method of claim 2, wherein selecting the anchor text from the set of candidate anchor texts comprises aggregating the set of candidate anchor texts.

4. The method of claim 2, wherein selecting the anchor text from the set of candidate anchor texts comprises performing n-gram clustering on at least one member of the set of candidate anchor texts.

5. The method of claim 1, further comprising:

analyzing the first document to determine if the first document contains data likely to be relevant to the object.

6. The method of claim 5, wherein the data likely to be relevant to the object comprises the name of the entity associated with the object.

7. The method of claim 6, wherein the first document comprises an HTML document comprising a header, and wherein analyzing the first document includes searching the header of the HTML document for the name of the entity associated with the object.

8. The method of claim 5, wherein the data likely to be relevant to the object comprises anchor text.

9. The method of claim 8, wherein the first document comprises an HTML document comprising a header, and wherein analyzing the first document includes searching the header of the HTML document for the anchor text.

10. The method of claim 1, further comprising:

analyzing the anchor text to determine if the first document contains data likely to be relevant to the object.

11. The method of claim 10, wherein analyzing the anchor text to determine if the first document contains data likely to be relevant to the object comprises comparing the anchor text to a list of known noise text.

12. The method of claim 1, wherein the name of the entity associated with the object comprises an object name.

13. The method of claim 1, wherein the set of facts comprises an object.

14. The method of claim 1, wherein adjusting the status includes marking the attribute-value pair as proper for retention in the set of facts.

15. The method of claim 1, wherein adjusting the status includes marking the attribute-value pair for deletion from the set of facts.

16. The method of claim 1, wherein adjusting the status includes marking the attribute-value pair as proper for access from the set of facts.

17. The method of claim 1, wherein the name of the entity is selected from the group consisting of:

a person'"'"'s name, a place name, a company name, and an organization name.

18. A back end computer system for corroborating a set of facts included in a fact repository, the system comprising:

one or more processors;

memory; and

one or more programs stored in the memory, the one or more programs comprising instructions to;

identify a set of facts associated with an object, the set of facts having been previously extracted from multiple documents of a collection of documents, each fact comprising an attribute-value pair, including a fact attribute type and a fact value, and wherein the object is associated with an entity having a name fact attribute type;

receive a first document not included in the multiple documents from the collection of documents and a reference to the first document, said reference comprising user-viewable anchor text extracted from a second document from the collection of documents;

determine that the user-viewable anchor text matches the name of the entity associated with the object;

determine that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document; and

responsive to determining that the user-viewable anchor text matches the name of the entity associated with the object and that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document, corroborate the set of facts using the first document;

the corroborating comprising;

identifying one or more facts in the first document, each identified fact having an attribute-value pair; and

comparing a respective attribute-value pair of the set of facts to an identified attribute-value pair in the first document; and

update the set of facts in accordance with the corroborating, wherein the updating includes one or both of storing an attribute-value pair from the first document in the set of facts in association with the object or adjusting a status of an attribute-value pair of the set of facts.

19. The system of claim 18, wherein said set of facts comprises an object, and wherein said name comprises an object name.

20. A non-transitory computer readable storage medium storing one or more programs for execution by one or more processors of a back end computer system, the one or more programs and comprising instructions for:

identifying a set of facts associated with an object, the set of facts having been previously extracted from multiple documents of a collection of documents, each fact comprising an attribute-value pair, including a fact attribute type and a fact value, and wherein the object is associated with an entity having a name fact attribute type;

receiving a first document not included in the multiple documents from the collection of documents and a reference to the first document, said reference comprising user-viewable anchor text extracted from a second document from the collection of documents;

determining that the user-viewable anchor text matches the name of the entity associated with the object;

determining that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document; and

responsive to determining that the user-viewable anchor text matches the name of the entity associated with the object and that one or both of the name of the entity associated with the object or the user-viewable anchor text appears in the first document, corroborating the set of facts using the first document;

the corroborating comprising;

identifying one or more facts in the first document, each identified fact having an attribute-value pair; and

comparing a respective attribute-value pair of the set of facts to an identified attribute-value pair in the first document; and

updating the set of facts in accordance with the corroborating, wherein the updating includes one or both of storing an attribute-value pair from the first document in the set of facts in association with the object or adjusting a status of an attribute-value pair of the set of facts.

21. The non-transitory computer readable storage medium of claim 20, said computer-readable storage medium further comprising:

program code for analyzing the anchor text to determine if the first document contains valid data.

22. The non-transitory computer readable storage medium of claim 21, wherein said program code for analyzing the anchor text to determine if the first document contains valid data comprises program code for comparing the anchor text to a list of known noise text.

23. The non-transitory computer readable storage medium of claim 20, wherein the name associated with the object comprises an object name.

24. The non-transitory computer readable storage medium of claim 20, wherein the set of facts comprises an object.

25. The non-transitory computer readable storage medium of claim 20, wherein the name of the entity is selected from the group consisting of:

a person'"'"'s name, a place name, a company name, and an organization name.