Using Network Traffic Logs for Search Enhancement

• US 20080270484A1
• Filed: 07/07/2008
• Published: 10/30/2008
• Est. Priority Date: 04/16/2001
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Arkady Borkovsky, Douglas M. Cook, Hongyuan Zha, Jean-Marc Langlois, Tomi Poutanen
• Inventors: Zha, Hongyuan, Langlois, Jean-Marc, Borkovsky, Arkady, Cook, Douglas M., Poutanen, Tomi
• Granted Patent: US 8,203,952 B2
• Time in Patent Office: Days
• US Class Current: 707/200
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A method and apparatus for using network traffic logs for search enhancement is disclosed. According to one embodiment, network usage is tracked by generating log files. These log files among other things indicate the frequency web pages are referenced and modified. These log files or information from these log files can then be used to improve document ranking, improve web crawling, determine tiers in a multi-tiered index, determine where to insert a document in a multi-tiered index, determine link weights, and update a search engine index.

Claims

1. A method for determining when to inspect a document comprising the steps of:

monitoring a statistic associated with said document;

wherein said document is requested by users during a particular period of time; and

using said statistic to determine when to inspect said document.

2. The method of claim 1 further comprises the step of after inspecting said document, updating an index used by a search engine based on new content of said document.

3. The method of claim 1, wherein said particular period of time is a first period of time and further includes the steps of:

monitoring said statistic during a second period of time; and

modifying a first frequency for inspecting said document that was determined during said first period of time to a second frequency that was determined during said second period of time.

4. The method of claim 1, wherein said statistic includes a frequency that said document is modified and the step of monitoring said statistic includes monitoring said frequency that said document is modified.

5. The method of claim 4, wherein the step of monitoring said frequency further includes the steps of:

determining if said document has been modified by passing a document name and a last modified date to an origin server; and

receiving an indication whether said document has been modified from said origin server.

6. The method of claim 4, wherein the step of monitoring said frequency that said document is modified further includes determining if said document is modified at least in part based on a last modified date associated with the HTTP protocol.

7. The method of claim 1, wherein said statistic includes a frequency that said document is requested and the step of monitoring said statistic includes monitoring said frequency that said document is requested.

8. A computer-readable medium carrying one or more sequences of instructions for determining when to inspect a document, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

monitoring a statistic associated with said document;

wherein said document is requested by users during a particular period of time; and

using said statistic to determine when to inspect said document.

9. The computer-readable medium of claim 8 further comprises instructions for performing the step of after inspecting said document, updating an index used by a search engine based on new content of said document.

10. The computer-readable medium of claim 8, wherein said particular period of time is a first period of time and further comprises instructions for performing the steps of:

monitoring said statistic during a second period of time; and

modifying a first frequency for inspecting said document that was determined during said first period of time to a second frequency that was determined during said second period of time.

11. The computer-readable medium of claim 8, wherein said statistic includes a frequency that said document is modified and the step of monitoring said statistic includes

monitoring said frequency that said document is modified.

12. The computer-readable medium of claim 11, wherein the step of monitoring said frequency further includes the steps of:

determining if said document has been modified by passing a document name and a last modified date to an origin server; and

receiving an indication whether said document has been modified from said origin server.

13. The computer-readable medium of claim 11, wherein the step of monitoring said frequency that said document is modified further includes determining if said document is modified at least in part based on a last modified date associated with the HTTP protocol.

14. The computer-readable medium of claim 8, wherein said statistic includes a frequency that said document is requested and the step of monitoring said statistic includes monitoring said frequency that said document is requested.