Realtime indexing and search in large, rapidly changing document collections

• US 20060294086A1
• Filed: 08/02/2006
• Published: 12/28/2006
• Est. Priority Date: 06/28/2005
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Rose, Daniel E., Walters, Chad, Mao, Jianchang
• Granted Patent: US 7,634,466 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

The present invention is directed to systems and methods for searching content items indexed in real-time. The method according to one embodiment comprises generating an index of word location pairs that identifies the location of one or more words in one or more content items available on a network. One or more additional content items are received over the network. The received content items are stored in a stream search queue, the stream search queue operative to allow for a stream search of the one or more additional content items.

Claims

1. A method for indexing content items, the method comprising:

generating an index of word location pairs that identifies the location of one or more words in one or more content items available on a network;

receiving one or more additional content items over the network; and

storing the received content items in a stream search queue, the stream search queue operative to allow for a stream search of the one or more additional content items.

2. The method of claim 1 wherein generating the index of word location pairs comprises indexing the location of one or more words that appear in one or more web pages.

3. The method of claim 1 comprising:

receiving a query from a user, the query comprising one or more query terms;

executing a stream search of the stream search queue to identify a given one of the query terms and to generate a stream search result set;

executing an index search of the index of word location pairs to identify a given one of the query terms and generate an index result set;

generating a merge result set on the basis of the stream search result set and the index result set; and

transmitting the merge result set to the user for display on a client device.

4. The method of claim 1 comprising:

determining if a threshold has expired; and

if the threshold has expired, indexing the one or more content items from the stream search queue into the word-location index.

5. The method of claim 4 wherein if the threshold has not expired, adding the content item in the stream search queue.

6. The method of claim 1 comprising indexing the one or more content items from the stream search index into the index of word location pairs upon receipt of a subsequent one or more content items.

7. The method of claim 1 wherein generating the index comprises indexing one or more items of personalized information.

8. The method of claim 7 comprising retrieving the personalized information from one or more user profiles.

9. The method of claim 7 comprising:

receiving personalized information from one or more users;

determining if a threshold has expired; and

if the threshold has expired, indexing the one or more items of personalized information into the index of word location pairs.

10. The method of claim 9 wherein if the threshold has not expired, adding the personalized information in the stream search queue.

11. The method of claim 1 comprising

receiving personalized information;

and

indexing the personalized information from the stream search queue into the index of word location pairs upon receipt of a subsequent personalized information.

12. The method of claim 1 wherein executing the stream search and executing the index search comprises applying a scoring technique and wherein generating the merge result set comprises ordering according to the scoring technique.

13. The method for generating a result set, the method comprising:

means for receiving a query from a user, the query comprising one or more query words;

means for executing a stream search of a stream search index to generate a stream search result set, the stream search index operative to provide direct access to one or more content items;

means for executing an index search of a word-location index to generate an index result set, the index operative to provide the location of one or more words in one or more corresponding content items;

means for generating a merge result set on the basis of the stream search result set and the index result set; and

means for transmitting the merge result set to the user for display on a client device.