Method and apparatus for customized logging in a network cache

• US 7,613,815 B1
• Filed: 10/16/2001
• Issued: 11/03/2009
• Est. Priority Date: 10/15/2001
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: English, Robert M., Prakash, Ashish
• Primary Examiner(s): Srivastava; Vivek
• Assistant Examiner(s): CHOUDHURY, AZIZUL Q
• Application Number:
  US09/982,243
• Time in Patent Office: 2,940 Days
• Field of Search: 709/203, 709/231
• US Class Current: 709/230
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Custom logging allows users deploying a proxy cache to customize information logged by the proxy cache. Users may choose to log a certain set of fields in any suitable order within log files. Each log file contains headers that define the set of fields logged, thereby making the log file self-contained for analysis tools. Proxy cache customers use log files for several purposes including monitoring and billing. The invention provides a single structure that controls whether a field is selected for logging or not, and its position within the log file. Changes to the format can be made while the proxy cache is running, causing a new header to be written to the log file and enabling administrators to alter the log format on the fly.

Claims

1. A device to log information in a network cache, the device comprising:

a processor and a memory together providing,

an application module to receive requests from a plurality of clients for content maintained by an origin server and to receive responses to the requests from the origin server;

an interface to allow selection of a protocol, including a selection for logging of some or all of a plurality of fields of the protocol that may be present in each of said requests and responses, and a specification of a sequence in which the selected fields are to appear in a log file;

a first data structure to store, for all of the plurality of fields of the protocol, an indication of whether a particular field has been selected for logging, wherein when the particular field has been selected for logging, the indication identifies a position in the specified sequence of the particular selected field;

a second data structure to store information corresponding to each selected field;

a third data structure to store, for each of the selected fields, a reference to the corresponding information stored in the second data structure, wherein each reference is stored in a location of the third data structure that corresponds to the position in the specified sequence of the selected field, and wherein the position is identified by the indication stored in the first data structure that corresponds to the selected field; and

a log module to record information of the received requests and responses into the log file, according to the selected protocol, the selected fields of the protocol, and the specified sequence.

2. The device of claim 1, wherein the interface allows the creation of new fields in addition to the plurality of fields.

3. The device of claim 1, wherein the interface is a graphical user interface.

4. The device of claim 1, wherein the interface is a command line interface.

6. The device of claim 1, wherein the interface further allows changing said selection to modify the log file'"'"'s format while the network cache is running.

7. The method of claim 6, wherein the interface allows creation of new fields in addition to the plurality of fields.

8. The method of claim 6, wherein the information for each field is converted to an ASCII representation and is of variable length.

9. The method of claim 6, wherein each location in the first data structure is pre-initialized to contain a flag before the specified sequence is stored, the flag to be utilized as an indicator that the corresponding field particular was not selected for logging.

10. The method of claim 6, wherein the second data structure and the third data structure are created to respond to logging for the corresponding request or response and destroyed once logging for the corresponding requests or response is completed.

11. The method of claim 6, wherein the first data structure persists through logging for the requests and responses.

12. The method of claim 6, wherein using the third data structure to output the information further comprises sequentially accessing the third data structure to read the position of the information corresponding to each selected field and accessing the second data structure to read information corresponding to each selected field at the position indicated by the reference.

13. The method of claim 6, wherein the interface allows changing the selection and the sequence to modify the log file'"'"'s format while the network cache is running.

5. A method in a computing system of logging information in a network cache, the method comprising:

by a processor of the computing system,

receiving requests from a plurality of clients for content maintained by an origin server;

receiving responses to said requests from the origin server;

receiving inputs via an interface to make a selection for logging of some or all of a plurality of fields of a protocol that may be present in each of said requests and responses and to make a specification of a sequence in which the selected fields are to appear in a log file;

storing in a first data structure, for all of the selected fields, a value indicating whether a particular field has been selected for logging wherein when the particular field has been selected for logging, the value identifies a position in the specified sequence of the particular selected field;

in response to receiving each of the requests and responses,

obtaining information for each selected field associated with the corresponding request or response and storing the information in a second data structure, in a sequence independent of the specified sequence,

storing in a third data structure, for each selected field, a reference to the corresponding information stored in the second data structure, including storing, based on the first data structure, each reference in a location of the third data structure that corresponds to the position in the specified sequence of the selected field; and

using the third data structure to output the information for each selected field in the second data structure to the log file, such that the information for each selected field appears in the log file according to the specified sequence.

14. A device for logging information in a network cache, the network cache serving a plurality of clients on behalf of an origin server, the device comprising:

a processor and a memory together providing,

an interface to allow selection of a protocol, including a selection for logging of some or all of a plurality of fields of a message to be received from anyone of the origin server and the plurality of clients, the fields corresponding to the selected protocol, and a specification of a sequence in which the selected fields are to appear in a log file of the network cache,

wherein the interface further allows changing said selections to modify the log file'"'"'s format while the network cache is running;

a protocol specific application module to obtain information for each selected field associated with the message;

a first data structure to store, for all of the plurality of fields of the selected protocol, a value indicating whether a particular field has been selected, wherein when the particular field has been selected, the value identifies a position in the specified sequence of the particular selected field;

a second data structure to store the obtained information for each selected field;

a third data structure to store a reference to the information stored in the second data structure, including storing the reference in a location of the third data structure that corresponds to the position in the specified sequence of a field corresponding to the reference; and

a protocol independent log module to receive information for each selected field from the protocol specific application module and to store the information for each selected field in the log file in the sequence specified.

15. The device of claim 14, wherein the interface allows creation of new fields in addition to the plurality of fields.

16. The device of claim 15, wherein the interface is a graphical user interface.

17. The device of claim 15, wherein the interface is a command line interface.

18. A method using one or more processors of a proxy device to operate a network cache, the method comprising:

by a processor of the proxy device,

receiving requests from a plurality of clients for contents maintained by a server and responses to the requests from the server, wherein the requests and responses are formatted according to a protocol;

receiving a selection of a first number of fields from a plurality of fields of the protocol that may be present in each of the requests and responses, and receiving a specification of a first sequence in which the first number of fields are to appear in a log file of the network cache;

recording the first number of fields extracted from a first one of the requests and responses into the log file according to the first sequence, wherein the recording includes;

storing in a first data structure, for all of the plurality of fields of the protocol, a value indicating whether a particular field has been selected, wherein when the particular field has been selected, the value identifies a position in the first sequence of the particular selected field;

obtaining information of each of the first number of fields associated with the first one of the requests and responses and storing the information in a second data structure, in a sequence independent of the first sequence;

storing in a third data structure a reference to the information for each of the first number of fields stored in the second data structure, including storing each reference in a location of the third data structure that corresponds to the position in the first sequence of the field corresponding to the reference; and

using the third data structure to output the information for each of the first number of fields in the second data structure to the log file, such that the information for each of the first number of fields appears in the log file according to the first sequence;

while operating the network cache, by a processor of the proxy device,

receiving inputs that changes selection of the plurality of fields from the first number of fields to a second number of fields and specifies a second sequence; and

recording the second number of fields extracted from a second one of the requests and responses into the log file according to the second sequence.

19. The method of claim 18, wherein recording the second number of fields extracted from the second one of the requests and responses into the log file according to the second sequence comprises:

by a processor of the proxy device,

storing in the first data structure, for all of the plurality of fields of the protocol, a value indicating whether a particular field has been selected, wherein when the particular field has been selected, the value identifies the position in the second sequence of the particular selected field;

obtaining information of each of the second number of fields associated with the second one of the requests and responses and storing the information in the second data structure, in a sequence independent of the second sequence;

storing in the third data structure, based on the first data structure, a reference to the information for each of the second number of fields stored in the second data structure, including storing each reference in a location of the third data structure that corresponds to the position in the second sequence of the corresponding field; and

using the third data structure to output the information for each of the second number of fields in the second data structure to the log file, such that the information for each of the second number of fields appears in the log file according to the second sequence.