METHOD AND SYSTEM FOR DATA LOGGING AND ANALYSIS

• US 20110022678A1
• Filed: 07/27/2009
• Published: 01/27/2011
• Est. Priority Date: 07/27/2009
• Status: Active Grant

• Current Assignee: VeriSign, Inc.
• Original Assignee: VeriSign, Inc.
• Inventors: Kothapalli, Malini, Smith, Christopher A., Bentkofsky, Michael A.
• Granted Patent: US 8,977,705 B2
• Time in Patent Office: Days
• US Class Current: 709/215
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2115 Third party
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

A method of logging internet requests includes defining a plurality of field types of interest and receiving a request from the internet including one or more fields. The method also includes determining that a first field type of a first field of the one or more fields matches one of the plurality of field types of interest and caching at least a portion of the first field in a cache. The method further includes determining that a second field type of a second field of the one or more fields matches one of the plurality of field types of interest caching at least a portion of the second field in the cache. The method includes transmitting the cache to a server, determining, a number of occurrences of a field value of at least one of the first field type or the second field type, and providing a report including the number of occurrences of the field value.

Claims

1. A method of logging internet requests, the method comprising:

defining a plurality of field types of interest;

receiving a request from the internet, wherein the request includes one or more fields;

determining that a first field type of a first field of the one or more fields matches one of the plurality of field types of interest;

caching at least a portion of the first field in a cache;

determining that a second field type of a second field of the one or more fields matches one of the plurality of field types of interest;

caching at least a portion of the second field in the cache;

transmitting the cache to a server;

determining, at the server, a number of occurrences of a field value of at least one of the first field type or the second field type; and

providing a report including the number of occurrences of the field value.

2. The method of claim 1 wherein determining that the first field type matches one of the plurality of field types of interest is performed at a resolution site.

3. The method of claim 1 further comprising not caching a third field of the one or more fields, wherein a third field type of the third field does not match one of the plurality of field types of interest.

4. The method of claim 1 further comprising, receiving, at the server, a cache from each of a plurality of processing engines.

5. The method of claim 1 further comprising transmitting the report to a system operator.

6. The method of claim 1 wherein the portion of the first field is a field type, field length, and field value for the first field and the portion of the second field is a field type, field length, and field value for the second field.

7. The method of claim 1 wherein each of the one or more fields is characterized by at least a field type, a field length, and a field value.

8. The method of claim 1 wherein defining the plurality of field types of interest is performed prior to receiving the request from the internet.

9. A method of reporting on internet requests, the method comprising:

defining a plurality of field types of interest;

receiving a first plurality of requests from the internet at a first processing engine, wherein each of the first plurality of requests includes one or more first fields;

caching a first field of the one or more first fields in a first cache;

caching a second field of the one or more first fields in the first cache;

receiving a second plurality of requests from the internet at a second processing engine, wherein each of the second plurality of requests includes one or more second fields;

caching a third field of the one or more second fields in a second cache;

caching a fourth field of the one or more second fields in the second cache;

transmitting the first cache to a server;

transmitting the second cache to the server;

determining, at the server, a top field value characterized by a higher number of occurrences than other field values; and

providing a report including the top field value and the number of occurrences of the top field value.

10. The method of claim 9 wherein the report comprises a list sorted by the number of occurrences of values from the cached fields.

11. The method of claim 9 further comprising:

determining, at the first processing engine, that a field type of the first field matches one of a plurality of field types of interest prior to caching the first field;

determining, at the first processing engine, that a field type of the second field matches one of the plurality of field types of interest prior to caching the second field;

determining, at the second processing engine, that a field type of the third field matches one of the plurality of field types of interest prior to caching the third field; and

determining, at the second processing engine, that a field type of the fourth field matches one of the plurality of field types of interest prior to caching the fourth field.

12. The method of claim 9 wherein the first plurality of requests and the second plurality of requests comprise at least one of DNS requests, IP requests, OCSP requests, or Whois requests.

13. The method of claim 9 wherein the first plurality of requests and the second plurality of requests comprise at least one of VoIP requests or IM requests.

14. A system comprising:

a plurality of processing engines coupled to the internet and operable to receive requests from the internet, each of the requests including one or more fields;

a server coupled to the plurality of processing engines; and

a reporting system coupled to the server, wherein each of the plurality of processing engines includes a first computer-readable medium storing a first plurality of instructions for controlling a first data processor to form a cache, the first plurality of instructions comprising;

instructions that cause the first data processor to received a plurality of requests over the Internet;

instructions that cause the first data processor to store, in the cache, field values for field types of interest for a subset of the plurality of requests received over the Internet;

instructions that cause the first data processor to increment a counter associated with the a total number of the subset;

instructions that cause the first data processor to determine a time period for forming the cache; and

instructions that cause the first data processor to transmit the cache to the server, wherein the server includes a second computer-readable medium storing a second plurality of instructions for controlling a second data processor to form a report, the second plurality of instructions comprising;

instructions that cause the second data processor to increment counters associated with field values for the plurality of field types of interest;

instructions that cause the second data processor to sort the field values based on the counters; and

instructions that cause the second data processor to transmit the report to the reporting system.

15. The system of claim 14 wherein the requests comprise at least one of DNS requests, IP requests, OCSP requests, or Whois requests.

16. The system of claim 14 wherein the plurality of processing engines are located at a resolution site.

17. The system of claim 16 wherein the resolution site is a component of a DNS resolution system.

18. The system of claim 14 wherein caching the first field comprises storing a field type, a field length, and a field value.

19. The system of claim 14 wherein the cache is transmitted to the server based on passage of a time interval.

20. The system of claim 14 wherein the cache is transmitted to the server based on reaching a cache size limit.

21. The system of claim 14 wherein the server comprises a service thread and a reporter thread.