Identifying the unifying subject of a set of facts

• US 8,719,260 B2
• Filed: 11/22/2011
• Issued: 05/06/2014
• Est. Priority Date: 05/31/2005
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Betz, Jonathan T.
• Primary Examiner(s): Kerzhner, Aleksandr
• Application Number:
  US13/302,755
• Publication Number:
  US 20120072433A1
• Time in Patent Office: 896 Days
• Field of Search: 707/711, 707/723, 707/999.2
• US Class Current: 707/723
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: Y10S 707/96 Object-relational

Abstract

A method and system for identifying a subject of a document and facts included within are described. A source document that includes facts and linking documents that include hyperlinks to the source document are identified. The anchor texts of the hyperlinks are identified and candidate labels are generated based on the anchor texts. One of the candidate labels is selected as the unifying subject of the facts. An information set, distinct from the source document, is stored in a facts database, wherein the stored information set includes the unifying subject, and source document information associating the source document with the information set.

Claims

1. A method of processing a set of documents for generating a facts database, comprising:

at a system having one or more processors and memory storing one or more modules to be executed by the one or more processors;

accessing a source document from a document host;

extracting one or more facts from the source document;

identifying a set of linking documents that have one or more links to the source document, wherein a respective link contains anchor text;

generating a set of candidate labels from the anchor text of the linking documents, a respective candidate label of the set of candidate labels comprising text extracted from the anchor text in the one or more links to the source document in the set of linking documents;

selecting a respective candidate label from the set of candidate labels as a unifying subject of the one or more facts extracted from the source document; and

storing in the facts database an information set distinct from the source document, wherein the information set includes the unifying subject, one or more entries corresponding to the one or more facts extracted from the source document, and source document information for the one or more facts corresponding to the one or more entries.

2. The method of claim 1, further comprising:

selecting one or more second labels of the candidate labels according to predefined criteria; and

associating the selected second labels with the source document and the one or more facts extracted from the source document.

3. The method of claim 1, wherein selecting the candidate label comprises:

for each of the set of candidate labels;

determining a set of frequencies of one or more substrings of the candidate label; and

generating a frequency vector associated with the candidate label based on the set of frequencies; and

determining a centroid vector based on the frequency vectors of the candidate labels, wherein the selected candidate label is associated with a respective frequency vector having a shortest distance to the centroid vector.

4. A server system for processing a set of documents for generating a facts database, comprising:

one or more processors;

memory storing one or more programs to be executed by the one or more processors, the one or more programs including instructions to;

access a source document from a document host;

extract one or more facts from the source document;

identify a set of linking documents that have one or more links to the source document, wherein a respective link contains anchor text; and

generate a set of candidate labels from the anchor text of the linking documents, a respective candidate label of the set of candidate labels comprising text extracted from the anchor text in the one or more links to the source document in the set of linking documents;

select a respective candidate label from the set of candidate labels as a unifying subject of the one or more facts extracted from the source document; and

store in the facts database an information set distinct from the source document, wherein the information set includes the unifying subject, one or more entries corresponding to the one or more facts extracted from the source document, and source document information for the one or more facts corresponding to the one or more entries.

5. The system of claim 4, wherein the one or more programs further include instructions to:

select one or more second labels of the candidate labels according to predefined criteria; and

associate the selected second labels with the source document and the one or more facts extracted from the source document.

6. The system of claim 4, wherein the instructions to select a respective candidate label as a unifying subject include instructions to:

for each of the set of candidate labels;

determine a set of frequencies of one or more substrings of the candidate label; and

generate a frequency vector associated with the candidate label based on the set of frequencies; and

determine a centroid vector based on the frequency vectors of the candidate labels, wherein the selected candidate label is associated with a respective frequency vector having a shortest distance to the centroid vector.

7. A non-transitory computer readable storage medium storing one or more computer programs executed by a computerized server system, the one or more computer programs comprising instructions to generate a facts database, the instructions including:

instructions to access a source document from a document host;

instructions to extract one or more facts from the source document;

instructions to identify a set of linking documents that have one or more links to the source document, wherein a respective link contains anchor text;

instructions to generate a set of candidate labels from the anchor text of the linking documents, a respective candidate label of the set of candidate labels comprising text extracted from the anchor text in the one or more links to the source document in the set of linking documents;

instructions to select a respective candidate label from the set of candidate labels as a unifying subject of the one or more facts extracted from the source document; and

instructions to store in the facts database an information set distinct from the source document, wherein the information set includes the unifying subject, one or more entries corresponding to the one or more facts extracted from the source document, and source document information for the one or more facts corresponding to the one or more entries.

8. The computer readable storage medium of claim 7, further comprising instructions to:

select one or more labels of the candidate labels according to predefined criteria; and

associate the selected second labels with the source document and the one or more facts extracted from the source document.

9. The computer readable storage medium of claim 7, wherein the instructions to select a respective candidate label as a unifying subject comprise instructions to:

for each of the set of candidate labels;

determine a set of frequencies of one or more substrings of the candidate label; and

generate a frequency vector associated with the candidate label based on the set of frequencies; and

determine a centroid vector based on the frequency vectors of the candidate labels, wherein the selected label is associated with a respective frequency vector having a shortest distance to the centroid vector.