Phrase-based indexing in an information retrieval system

• US 20060020607A1
• Filed: 07/26/2004
• Published: 01/26/2006
• Est. Priority Date: 07/26/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Patterson, Anna Lynn
• Granted Patent: US 7,536,408 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

An information retrieval system uses phrases to index, retrieve, organize and describe documents. Phrases are identified that predict the presence of other phrases in documents. Documents are the indexed according to their included phrases. Related phrases and phrase extensions are also identified. Phrases in a query are identified and used to retrieve and rank documents. Phrases are also used to cluster documents in the search results, create document descriptions, and eliminate duplicate documents from the search results, and from the index.

Claims

1. A method of indexing documents in a document collection, the method comprising:

providing a list of phrases;

identifying for a given document, each phrase in the document;

for each phrase in the document, identifying a related phrase also present in the document; and

for each phrase in the document, storing in a posting list of the phrase an identifier of the document, and an indication of each related phrase also present in the document.

2. The method of claim 1, wherein for each phrase g_j, g_k is a related phrase of phrase g_j where the information gain of g_k with respect to g_j exceeds a predetermined threshold.

3. A method of indexing documents in a document collection, the method comprising:

providing a list of valid phrases, wherein each phrase on the list appears a minimum number of times in the document collection, and predicts at least one other phrase;

accessing a plurality of documents in the document collection;

for each accessed document, identifying each phrase in the document, from the list of valid phrases; and

for each identified phrase in the document, storing in a posting list of the phrase an identifier of the document.