Method and system for data logging and analysis

• US 8,977,705 B2
• Filed: 07/27/2009
• Issued: 03/10/2015
• Est. Priority Date: 07/27/2009
• Status: Active Grant

• Current Assignee: VeriSign, Inc.
• Original Assignee: VeriSign, Inc.
• Inventors: Smith, Christopher A., Bentkofsky, Michael A., Kothapalli, Malini
• Primary Examiner(s): Pollack, Melvin H
• Application Number:
  US12/510,161
• Publication Number:
  US 20110022678A1
• Time in Patent Office: 2,052 Days
• Field of Search: 709/215
• US Class Current: 709/215
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 21/552 involving long-term monitoring or reporting
• CPC Class Codes: G06F 2221/2101 Auditing as a secondary aspect
• CPC Class Codes: G06F 2221/2115 Third party
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

A method of logging internet requests includes defining a plurality of field types of interest and receiving a request from the internet including one or more fields. The method also includes determining that a first field type of a first field of the one or more fields matches one of the plurality of field types of interest and caching at least a portion of the first field in a cache. The method further includes determining that a second field type of a second field of the one or more fields matches one of the plurality of field types of interest caching at least a portion of the second field in the cache. The method includes transmitting the cache to a server, determining, a number of occurrences of a field value of at least one of the first field type or the second field type, and providing a report including the number of occurrences of the field value.

Claims

1. A method of caching internet requests, the method comprising:

identifying a plurality of user-defined field types of interest;

receiving a request from the internet, wherein the request includes one or more fields;

determining that a first field type of a first field of the one or more fields matches one of the plurality of user-defined field types of interest;

caching at least a portion of the first field in a cache based on the determination that the first field type of the first field of the one or more fields matches one of the plurality of user-defined field types of interest;

determining that a second field type of a second field of the one or more fields matches one of the plurality of user-defined field types of interest;

caching at least a portion of the second field in the cache based on the determination that the second field type of the second field of the one or more fields matches one of the plurality of user-defined field types of interest;

transmitting the cache to a server;

determining, at the server, a number of occurrences of a field value of at least one of the first field type or the second field type in the cache; and

providing a report including the number of occurrences of the field value.

2. The method of claim 1 wherein determining that the first field type matches one of the plurality of user-defined field types is performed at a resolution site.

3. The method of claim 1 further comprising not caching a third field of the one or more fields, wherein a third field type of the third field does not match one of the plurality of user-defined field types.

4. The method of claim 1 further comprising, receiving, at the server, a cache from each of a plurality of processing engines.

5. The method of claim 1 further comprising transmitting the report to a system operator.

6. The method of claim 1 wherein the portion of the first field is a field type, field length, and field value for the first field and the portion of the second field is a field type, field length, and field value for the second field.

7. The method of claim 1 wherein each of the one or more fields is characterized by at least a field type, a field length, and a field value.

8. The method of claim 1 wherein defining the plurality of user-defined field types is performed prior to receiving the request from the internet.

9. A method of reporting on internet requests, the method comprising:

identifying a plurality of user-defined field types of interest;

receiving a first plurality of requests from the internet at a first processing engine, wherein each of the first plurality of requests includes one or more first fields;

determining that a first field type of a first field of the one or more first fields matches one of the plurality of user-defined field types of interest;

caching the first field of the one or more first fields in a first cache based on the determination that the first field type of the first field of one or more first fields matches one of the plurality of user-defined field types of interest;

determining that a second field type of a second field of the one or more first fields matches one of the plurality of user-defined field types of interest;

caching the second field of the one or more first fields in the first cache based on the determination that the second field type of the second field of the one or more first fields matches one of the plurality of user-defined field types of interest;

;

receiving a second plurality of requests from the internet at a second processing engine, wherein each of the second plurality of requests includes one or more second fields;

caching a third field of the one or more second fields in a second cache;

caching a fourth field of the one or more second fields in the second cache;

transmitting the first cache to a server;

transmitting the second cache to the server;

determining, at the server, a top field value characterized by a higher number of occurrences than other field values; and

providing a report including the top field value and the number of occurrences of the top field value.

10. The method of claim 9 wherein the report comprises a list sorted by the number of occurrences of values from the cached fields.

11. The method of claim 9 further comprising:

determining, at the first processing engine, that a field type of the first field matches one of a plurality of user-defined field types prior to caching the first field;

determining, at the first processing engine, that a field type of the second field matches one of the plurality of user-defined field types prior to caching the second field;

determining, at the second processing engine, that a field type of the third field matches one of the plurality of user-defined field types prior to caching the third field; and

determining, at the second processing engine, that a field type of the fourth field matches one of the plurality of user-defined field types prior to caching the fourth field.

12. The method of claim 9 wherein the first plurality of requests and the second plurality of requests comprise at least one of DNS requests, IP requests, OCSP requests, or Whois requests.

13. The method of claim 9 wherein the first plurality of requests and the second plurality of requests comprise at least one of VoIP requests or IM requests.

14. A system comprising:

a plurality of processing engines coupled to the internet and operable to receive requests from the internet, each of the requests including one or more fields;

a server coupled to the plurality of processing engines; and

a reporting system coupled to the server, wherein each of the plurality of processing engines includes a first computer-readable medium storing a first plurality of instructions for controlling a first data processor to form a cache, the first plurality of instructions comprising;

instructions that cause the first data processor to received a plurality of requests over the Internet;

instructions that cause the first data processor to store, in the cache, field values for user-defined field types of interest for one or more fields included in a subset of the plurality of requests received over the Internet, based on the determination that the field types of the one or more fields included in the subset of the plurality of requests match the user-defined field types of interest;

instructions that cause the first data processor to increment a counter associated with the a total number of the subset;

instructions that cause the first data processor to determine a time period for forming the cache; and

instructions that cause the first data processor to transmit the cache to the server, wherein the server includes a second computer-readable medium storing a second plurality of instructions for controlling a second data processor to form a report, the second plurality of instructions comprising;

instructions that cause the second data processor to increment counters associated with field values for the plurality of user-defined field types;

instructions that cause the second data processor to sort the field values based on the counters; and

instructions that cause the second data processor to transmit the report to the reporting system.

15. The system of claim 14 wherein the requests comprise at least one of DNS requests, IP requests, OCSP requests, or Whois requests.

16. The system of claim 14 wherein the plurality of processing engines are located at a resolution site.

17. The system of claim 16 wherein the resolution site is a component of a DNS resolution system.

18. The system of claim 14 wherein caching the first field comprises storing a field type, a field length, and a field value.

19. The system of claim 14 wherein the cache is transmitted to the server based on passage of a time interval.

20. The system of claim 14 wherein the cache is transmitted to the server based on reaching a cache size limit.

21. The system of claim 14 wherein the server comprises a service thread and a reporter thread.