Using network traffic logs for search enhancement

• US 7,996,397 B2
• Filed: 07/07/2008
• Issued: 08/09/2011
• Est. Priority Date: 04/16/2001
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Zha, Hongyuan, Langlois, Jean-Marc, Borkovsky, Arkady, Cook, Douglas M., Poutanen, Tomi
• Primary Examiner(s): Robinson; Greta L
• Assistant Examiner(s): Chang; Jeffrey
• Application Number:
  US12/168,813
• Publication Number:
  US 20080270404A1
• Time in Patent Office: 1,128 Days
• Field of Search: 707/727
• US Class Current: 707/727
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A method and apparatus for using network traffic logs for search enhancement is disclosed. According to one embodiment, network usage is tracked by generating log files. These log files among other things indicate the frequency web pages are referenced and modified. These log files or information from these log files can then be used to improve document ranking, improve web crawling, determine tiers in a multi-tiered index, determine where to insert a document in a multi-tiered index, determine link weights, and update a search engine index.

Claims

1. A method of placing a document in a multi-tiered index, comprising the steps of:

monitoring, over a period of time, requests for documents made by users;

wherein said requests are intercepted by a device on a network that is remote relatively to a location of said multi-tiered index;

determining which tier of said multi-tiered index to place said document based at least in part on a frequency that said users have requested said document;

placing said document in said tier;

executing a search query using only said tier of said multi-tiered index; and

after search results from said tier of said index have been exhausted, executing said search query using other lower tiers of said multi-tiered index.

2. The method of claim 1, wherein the step of monitoring further includes generating usage logs that identify said frequency and the step of determining further includes determining which tier of said multi-tiered index to place said document based at least in part on said usage logs.

3. The method of claim 1, wherein the step of placing said document in said tier includes removing said document from a relatively lower tier and placing said document in a relatively higher tier in response to said document being requested with a relatively higher frequency than other documents in said lower tier.

4. A non-transitory computer-readable storage medium storing one or more sequences of instructions for placing a document in a multi-tiered index, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

monitoring, over a period of time, requests for documents made by users;

wherein said requests are intercepted by a device on a network that is remote relatively to a location of said multi-tiered index;

determining which tier of said multi-tiered index to place said document based at least in part on a frequency that said users have requested said document;

placing said document in said tier;

executing a search query using only said tier of said multi-tiered index; and

after search results from said tier of said index have been exhausted, executing said search query using other lower tiers of said multi-tiered index.

5. The non-transitory computer-readable storage medium of claim 4, wherein the step of monitoring further includes

generating usage logs that identify said frequency and the step of determining further includes determining which tier of said multi-tiered index to place said document based at least in part on said usage logs.

6. The non-transitory computer-readable storage medium of claim 4, wherein the step of placing said document in said tier includes removing said document from a relatively lower tier and placing said document in a relatively higher tier in response to said document being requested with a relatively higher frequency than other documents in said lower tier.