System and method for efficient and adaptive web accesses filtering

• US 20020178169A1
• Filed: 05/23/2001
• Published: 11/28/2002
• Est. Priority Date: 05/23/2001
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Deeter, Ken T., Chow, Kingsum, Tjaden, Brian C., Nair, Sandeep R.
• Granted Patent: US 6,741,990 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99939 Privileged access

Abstract

An efficient and adaptive system and method for filtering web accesses in an web access log analysis are described. A web log file having a plurality of entries, with each entry containing a field identifying a web access of a visitor, and a list of exclusion web accesses to be filtered from the web log file are provided. An optimum algorithm for searching in the list of exclusion web accesses is selected from a set of algorithms. Based on the selection, a data structure corresponding to the selected algorithm and the exclusion web accesses is built. Comparisons are made between the web access in each entry of the web log file and the data structure to filter or discount entries from the web log file.

Claims

1. A method for filtering web accesses, the method comprising:

providing a web log file having a plurality of entries, each entry containing a field identifying a web access of a visitor;

providing a list of exclusion web accesses to be filtered from the web log file;

selecting an optimum algorithm for searching in the list of exclusion web accesses from a set of algorithms;

building a data structure corresponding to the selected algorithm using the exclusion web accesses in the list of exclusion web accesses; and

filtering the web log file using the data structure corresponding to the selected algorithm, wherein comparisons are made between the web access of each entry of the web log file and the data structure.

2. The method of claim 1, wherein the selection of the optimum algorithm includes exploring the effectiveness of each algorithm in the set of algorithms and data structure with respect to the list of exclusion web accesses and choosing the optimal algorithms from the set of algorithms based on the effectiveness.

3. The method of claim 2, wherein metrics are generated for each algorithm in the set of algorithms to calculate the effectiveness of the algorithm, and performance data of previous analysis runs is used to determine which algorithm is the optimum algorithm.

4. The method of claim 1, wherein the set of algorithms include a binary search algorithm, and a data structure corresponding to a selected binary search algorithm is a sorted list of exclusion web accesses.

5. The method of claim 1, wherein the set of algorithms include a multi-level dynamic table indexing algorithm, and a data structure corresponding to a selected multi-level dynamic table indexing algorithm is a multi-level table.

6. The method of claim 1, wherein the web access contains an IP address, and the list of exclusion web accesses are a list of exclusion IP addresses.

7. The method of claim 1, further comprising:

receiving a filtered web log file; and

analyzing the web log file for obtaining at least one of a traffic report and a visitor behavior report.

8. The method of claim 1, further comprising extracting other information associated with web accesses in entries of the web log file with non-filtered web accesses during the filtering.

9. The method of claim 1, further comprising:

sorting the web log file based on the web access field before filtering; and

using the last used exclusion web accesses first in a comparison between each entry of the web log file and the data structure subsequent to a comparison that results in an entry being filtered from the web log file.

10. A web accesses filtering system, comprising:

a communication medium;

a client server that interacts with a visitor via the communication medium and provides a log file documenting access information pertaining to the visitor, the log file including a plurality of entries, each entry containing a field identifying a web access of the visitor; and

a service server that receives the log file from the client server and filters the log file, wherein the service server is provided with a list of exclusion web accesses to be filtered from the log file, the service server selecting an optimum algorithm for searching in the list of exclusion web accesses from a set of algorithms, building a data structure corresponding to the selected algorithm using the exclusion web accesses in the list of exclusion web accesses, and filtering the log file using the data structure corresponding to the selected algorithm, with comparisons being made between the web access of each entry of the log file and the data structure.

11. The system of claim 10, wherein the selection of the optimum algorithm includes exploring the effectiveness of each algorithm in the set of algorithms and data structure with respect to the list of exclusion web accesses and choosing the optimal algorithms from the set of algorithms based on the effectiveness.

12. The system of claim 11, wherein metrics are generated for each algorithm in the set of algorithms to calculate the effectiveness of the algorithm, and performance data of previous analysis runs is used to determine which algorithm is the optimum algorithm.

13. The system of claim 10, wherein the set of algorithms include a binary search algorithm, and a data structure corresponding to a selected binary search algorithm is a sorted list of exclusion web accesses.

14. The system of claim 10, wherein the set of algorithms include a multi-level dynamic table indexing algorithm, and a data structure corresponding to a selected multi-level dynamic table indexing algorithm is a multi-level table.

16. The computer readable medium of claim 15, wherein the selection of the optimum algorithm includes exploring the effectiveness of each algorithm in the set of algorithms and data structure with respect to the list of exclusion web accesses and choosing the optimal algorithms from the set of algorithms based on the effectiveness.

17. The computer readable medium of claim 16, wherein metrics are generated for each algorithm in the set of algorithms to calculate the effectiveness of the algorithm, and performance data of previous analysis runs is used to determine which algorithm is the optimum algorithm.

18. The computer readable medium of claim 15, wherein the set of algorithms include a binary search algorithm, and a data structure corresponding to a selected binary search algorithm is a sorted list of exclusion web accesses.

19. The computer readable medium of claim 15, wherein the set of algorithms include a multi-level dynamic table indexing algorithm, and a data structure corresponding to a selected multi-level dynamic table indexing algorithm is a multi-level table.

20. The computer readable medium of claim 15, wherein the web access contains an IP address, and the list of exclusion web accesses are a list of exclusion IP addresses.

21. The computer readable medium of claim 15, further comprising computer readable instruction encoded thereon for:

receiving a filtered web log file; and

analyzing the web log file for obtaining at least one of a traffic report and a visitor behavior report.

22. The computer readable medium of claim 15, further comprising computer readable instruction encoded thereon for extracting other information associated with web accesses in entries of the web log file with non-filtered web accesses during the filtering.

23. The computer readable medium of claim 15, further comprising computer readable instruction encoded thereon for:

sorting the web log file based on the web access field before filtering; and

using the last used exclusion web accesses first in a comparison between each entry of the web log file and the data structure subsequent to a comparison that results in an entry being filtered from the web log file.

15. A computer readable medium for use in conjunction with at least one of a web access filtering system and a service server system, the computer readable medium including computer readable instructions encoded thereon for:

providing a web log file having a plurality of entries, each entry containing a field identifying a web access of a visitor;

providing a list of exclusion web accesses to be filtered from the web log file;

selecting an optimum algorithm for searching in the list of exclusion web accesses from a set of algorithms;

building a data structure corresponding to the selected algorithm using the exclusion web accesses in the list of exclusion web accesses; and

filtering the web log file using the data structure corresponding to the selected algorithm, wherein comparisons are made between the web access of each entry of the web log file and the data structure.