Methods and apparatus for locating network logs

• US 8,041,683 B1
• Filed: 09/29/2006
• Issued: 10/18/2011
• Est. Priority Date: 09/29/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ninan, Anoop George, Farizon, Boris, Apte, Charuta Vijaykumar, Korolev, Eugenio
• Primary Examiner(s): Kim; Charles
• Assistant Examiner(s): MINA, FATIMA P
• Application Number:
  US11/540,105
• Time in Patent Office: 1,845 Days
• Field of Search: 707/812, 707/672, 709/231, 709/248
• US Class Current: 707/672
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A log collection utility (LCU) operable in a storage area network (SAN) to identify the various types of log files generated in the SAN computes the location of each of the log files, and collects the log files for consolidation and analysis in a common repository. The LCU periodically collects and/or processes the log files by enumerating the location of each of the log files in a log file designator, and generates a script operable for identifying each of the log files sought. A resulting task definition file lists each of the log files identified. The log collection utility therefore finds log files generated by dissimilar applications and nodes in the SAN via a log type definition that enumerates each of the log files in a manner that is agnostic to operating systems, platforms, and directory structures around which the log files are written and stored.

Claims

1. A method for identifying log files at a SAN infrastructure server from hosts in a heterogeneous storage area network (SAN) comprising:

identifying at the SAN infrastructure server a set of hosts in the SAN, each host having at least one log file and operable for log file collection;

each host including an agent configured to manage resources in the SAN;

computing, at the SAN infrastructure server, for each host in the identified, set of hosts, a query type indicative of the manner of locating the log files stored in various locations on the host according to an application executing on the host or agent that generated the log files depending on the manageable entities responsive to the host;

determining at the SAN infrastructure server a path operable to locate the log files on each of the hosts in the set of hosts according to the identification of each of the hosts in the set of hosts;

determining at the SAN infrastructure server an expression indicative of a file specification according to the identification of each of the hosts in the set of hosts, the file specification denoting the log files;

generating at the SAN infrastructure server a script file for discovering the log files on each of the hosts in the set of hosts according to the query type, path and expression, wherein the hosts need not support similar file systems, wherein generating a script file further comprises generating a task definition indicative of operations for gathering the collective set of log files corresponding to the identified set of hosts by;

traversing at least one log file designators;

enumerating, for each log file obtained in the traversal, a task operable to retrieve the corresponding log file; and

storing each of the enumerated tasks in a task definition file;

transferring the script file from the SAN infrastructure server to each host in the identified set of hosts for execution at each respective host in the identified set of hosts; and

receiving at the SAN infrastructure server, in response to execution of the transferred script on each host in the identified set of hosts, identification of each log file type at each respective host in the identified set of hosts.

2. The method of claim 1 wherein generating at the SAN infrastructure server a script file for discovering the log files on each of the hosts in the set of hosts according to the query type, path and expression further comprises:

generating at the SAN infrastructure server commands for performing the identifying, computing, and determining the path and file specification for the log files at each of the hosts in the set of hosts; and

storing at the SAN infrastructure server the generated commands in the script file for transfer from the SAN infrastructure server to each host in the identified set of hosts for execution of the script file to generate a task list of available log files.

3. The method of claim 2 further comprising:

determining, for each host, types of log files installed on that host, the type indicative of a query type for accessing the log files; and

determining a file specification corresponding to each of the determined types of log files.

4. The method of claim 3 wherein the query type is at least one of an environment variable or a registry key, the query type indicative of the storage location of the log files, further comprising selectively retrieving at least one of an environment variable or a registry key.

5. The method of claim 4 wherein the path is indicative of a file system traversal, further comprising:

identifying, from the query type, a storage area for the log files; and

appending, to the identified storage area, the path.

6. The method of claim 5 further comprising determining a category of the log files selected from the group consisting of infrastructure logs, agent logs and third party software logs.

7. The method of claim 1 further comprising storing, for each host, a log file designator indicative of the location of log files on that host, the log file designator independent of the operating system corresponding to each host.

8. The method of claim 7 wherein computing the query type further comprises identifying an operating system for each host.

9. The method of claim 8 further comprising storing each of the log file designators corresponding to each of the hosts in the set of hosts in a designator script file.

10. A SAN infrastructure server for identifying log files in a heterogeneous storage area network (SAN) comprising:

a processor; and

memory encoded with instruction that, When executed on the processor, cause the SAN infrastructure server to;

identify a set of hosts in the SAN, each host having at least one log file and operable for log file collection, each host including an agent configured to manage resources in the SAN;

compute, for each host in the identified set of hosts, a query type indicative of the manner of locating the log files stored in various locations on the host according to an application executing on the host or agent that generated the log files depending on the manageable entities responsive to the host,

determine a path operable to locate the log files on each of the hosts in the set of hosts according to the identification of each hosts in the set of hosts,

determine an expression indicative of a file specification according to the identification of each of the hosts in the set of hosts, the file specification denoting the log files;

generate a script file operable to discover the log files on each of the hosts in the set of hosts according to the query type, path and expression, wherein the hosts need not support similar file systems, the script file generated by generating a task definition indicative of operations for gathering the collective set of log files corresponding to the identified set of hosts by;

traversing at least one log file designators;

enumerating, for each log file obtained in the traversal, a task operable to retrieve the corresponding log file; and

storing each of the enumerated tasks in a task definition file;

transfer the script file to each host in the identified set of hosts for execution at each respective host in the identified set of hosts; and

receive, in response to execution of the transferred script on each host in the identified set of hosts, identification of each log file type at each respective host in the identified set of hosts.

11. The server of claim 10 further comprising:

a designator file indicative of log files on each of the hosts, the log collection utility responsive to the designator file for;

generating commands for performing the identifying, computing, and determining the path and file specification for the log files; and

storing the generated commands in the script file

for transfer from the SAN infrastructure server to each host in the identified set of hosts for execution of the script file to generate a task list of available log files.

12. The server of claim 11 wherein the log collection utility is operable to employ the designator file to:

determine, for each host, types of log files installed on that host, the type indicative of a query type for accessing the log files; and

determine a file specification corresponding to each of the determined types of log files.

13. The server of claim 12 wherein the designator file specifies the query type, the query type being at least one of an environment variable or a registry key, the query type indicative of the storage location of the log files, the executed script file operable to selectively retrieve at least one of an environment variable or a registry key.

14. The server of claim 13 wherein path is indicative of a file system traversal, the executed script file further operable to:

identify, from the query type, a storage area for the log files; and

append, to the identified storage area, the path.

15. The server of claim 10 further comprising, for each host, a log file designator indicative of the location of log files on that host, the log file designator independent of the operating system corresponding to each host.

16. The server of claim 15 wherein the log collection utility is operable to computing the query type by identifying an operating system for each host.

17. The server of claim 16 further comprising storing each of the log file designators corresponding to each of the hosts in the set of hosts in a designator script file, the designator script file further indicative of, for each host, a category of the log files selected from the group consisting of infrastructure logs, agent logs and third party software logs.

18. A computer program product having a non-transitory computer readable medium operable to store computer program logic embodied in computer program code encoded thereon as an encoded set of processor based instructions for performing a method of identifying log files at a SAN infrastructure server in a heterogeneous storage area network (SAN), comprising:

computer program code for identifying a set of hosts in the SAN, each host having at least one log file and operable for log file collection, each host including an agent configured to manage resources in the SAN;

computer program code for computing, for each host in the identified set of hosts, a query type indicative of the manner of locating the log files stored in various locations on the host according to an application executing on the host or agent that generated the log files depending on the manageable entities responsive to the host;

computer program code for determining a path operable to locate the log files on each of the hosts in the set of hosts according to the identification of each of the hosts in the set of hosts;

computer program code for determining an expression indicative of a file specification according to the identification of each of the hosts in the set of hosts, the file specification denoting the log files;

computer program code for generating a script file at the SAN infrastructure server for discovering the log files on each of the hosts in the set of hosts according to the query type, path and expression, wherein the hosts need not support similar file systems by generating commands for performing the identifying, computing, and determining the path and file specification for the log files and storing the generated commands in the script file for transfer from the SAN infrastructure server to each host in the identified set of hosts for execution of the script file to generate a task list of available log files, wherein computer program code for generating a script file wherein computer program code for generating a script file further comprises computer program code for generating a task definition indicative of operations for gathering the collective set of log files corresponding to the identified set of hosts by;

traversing at least one log file designators;

enumerating, for each log file obtained in the traversal, a task operable to retrieve the corresponding log file; and

storing each of the enumerated tasks in a task definition file;

computer program code for transferring the script file from the SAN infrastructure server to each host in the identified set of hosts for execution at each respective host in the identified set of hosts; and

computer program code for receiving at the SAN infrastructure server, in response to execution of the transferred script on each host in the identified set of hosts, identification of each log file type at each respective host in the identified set of hosts.