Apparatus, system and method for data collection, import and modeling

• US 10,216,579 B2
• Filed: 06/13/2016
• Issued: 02/26/2019
• Est. Priority Date: 08/07/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: McPhail, Iain
• Primary Examiner(s): Beausoliel, Jr., Robert W
• Assistant Examiner(s): Rayyan, Susan F
• Application Number:
  US15/180,169
• Publication Number:
  US 20160292029A1
• Time in Patent Office: 988 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A computer program product for data analysis of a backup system, including interrogating a plurality of computer systems to generate for each computer system at least one dump file, each dump file including configuration and state information; extracting predetermined configuration and state information from the respective dump files; and storing the extracted configuration and state information in a database in a normalized format.

Claims

1. A computer program product for data analysis of a backup system, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media, the program instructions comprising;

program instructions to generate a dump file for each of a plurality of backup servers, each dump file comprising configuration and state information about each of the plurality of backup servers in a native format used by each of the plurality of backup servers on which data is stored, wherein the backup servers backup the data from a primary storage layer to a common media layer;

program instructions to extract a first predetermined configuration and state information from the respective dump files of the plurality of backup servers, the first predetermined configuration and state information being in different formats based on the dump file from which it was extracted;

program instructions to translate the first predetermined configuration and state information from the format used by each of the plurality of backup servers into a normalized format, wherein the translated first configuration and state information comprises configuration and state information irrespective of which of the plurality of backup servers from which it was generated;

program instructions to store the translated first configuration and state information in a single database;

program instructions to generate a dump file for each of a plurality of different computer systems of the primary storage layer, each dump file comprising configuration and state information about each of the plurality of computer systems in a format used by each of the plurality of computer systems on which the data is stored, wherein the plurality of computer systems include server computers, desktop computers, and laptop computers which are physically located across various sites and use hardware and software from different vendors;

program instructions to extract a second predetermined configuration and state information from the respective dump files of the plurality of different computer systems, the second predetermined configuration and state information being in different formats based on the dump file from which it was extracted;

program instructions to translate the second predetermined configuration and state information from the format used by each of the plurality of computer systems into a normalized format, wherein the translated second configuration and state information comprises configuration and state information irrespective of which of the plurality of computer systems from which it was generated;

program instructions to store the translated second configuration and state information in the single database; and

program instructions to determine what components are in the backup system, how the backup system works, how data is stored in the backup system, how efficiently data is stored in the backup system, a total capacity of the backup system, a remaining capacity of the backup system, and an operating cost of the backup system by analyzing the normalized first and second configuration and state information stored in the single database.

2. The computer program product according to claim 1, wherein the format used by each of the plurality of computer systems is at least one of a vendor specific format and a product specific format.

3. The computer program product according to claim 1, wherein the program instructions to generate the dump files for each of the plurality of backup servers and each of the plurality of different computer systems comprises:

program instructions to query each of the plurality of backup servers using an interface native to each of the plurality of backup servers; and

program instructions to query each of the plurality of different computer systems using an interface native to each of the plurality of different computer systems.

4. The computer program product according to claim 3, wherein the program instructions to query each of the plurality of computer systems using the interface native to each of the plurality of computer systems comprises:

program instructions to run a predetermined series of commands, wherein the interface comprises a standard command line interface.

5. The computer program product according to claim 1, wherein the configuration and state information relates to a plurality of objects stored on the plurality of backup servers and the plurality of different computer systems and comprises metadata about the plurality of objects.

6. The computer program product according to claim 1, wherein the configuration and state information comprises metadata which describes a physical storage location of each object and includes customer data, site data, source data, node data, object data, and fragment data, wherein the fragment data describes a physical location of each the object.

7. A computer program product for data analysis of a backup system, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media, the program instructions comprising;

program instructions to generate a dump file for each of a plurality of different computer systems, each dump file comprising product specific configuration and state information about each of the plurality of computer systems in a native format used by each of the plurality of computer systems on which data is stored;

program instructions to extract predetermined configuration and state information from the respective dump files of the plurality of different computer systems, the predetermined configuration and state information being in different native formats, wherein the predetermined configuration and state information comprises metadata which describes a physical storage location of each object and includes customer data, site data, source data, node data, object data, and fragment data, wherein the fragment data describes a physical location of each the object;

program instructions to translate the predetermined configuration and state information from the native format used by each of the plurality of different computer systems into a normalized format, wherein the normalized configuration and state information comprises a format irrespective of which of the plurality of different computer systems from which it was generated; and

program instructions to determine what components are in the backup system, how the backup system works, how data is stored in the backup system, how efficiently data is stored in the backup system, a total capacity of the backup system, a remaining capacity of the backup system, and an operating cost of the backup system by analyzing the normalized predetermined configuration and state information.

8. The computer program product according to claim 7, wherein the native format used by each of the plurality of different computer systems is a vendor specific format.

9. The computer program product according to claim 7, wherein the program instructions to generate the dump file for each of the plurality of different computer systems comprises:

program instructions to query each of the plurality of different computer systems using an interface native to each of the plurality of different computer systems.

10. The computer program product according to claim 9, wherein the program instructions to query each of the plurality of different computer systems using the interface native to each of the plurality of different computer systems comprises:

program instructions to run a predetermined series of commands, wherein the interface comprises a standard command line interface.

11. The computer program product according to claim 7, wherein the configuration and state information relates to a plurality of objects stored on the plurality of different computer systems and comprises metadata about the plurality of objects.

12. The computer program product according to claim 7, wherein the plurality of different computer systems use at least two different proprietary data formats.

13. A computer program product for data analysis of a backup system, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media, the program instructions comprising;

program instructions to generate a dump file for each of a plurality of different computer systems, each dump file comprising configuration and state information about each of the plurality of different computer systems in a native format used by each of the plurality of different computer systems on which data is stored;

program instructions to extract predetermined configuration and state information from the respective dump files of the plurality of different computer systems, the predetermined configuration and state information being in different native formats based on the dump file from which it was extracted;

program instructions to translate the predetermined configuration and state information from the native format used by each of the plurality of different computer systems into a normalized format, wherein the translated configuration and state information comprises configuration and state information irrespective of which of the plurality of different computer systems from which it was generated; and

program instructions to determine what components are in the backup system, how the backup system works, how data is stored in the backup system, how efficiently data is stored in the backup system, a total capacity of the backup system, a remaining capacity of the backup system, and an operating cost of the backup system by analyzing the normalized predetermined configuration and state information.

14. The computer program product according to claim 13, wherein the native format used by each of the plurality of different computer systems is at least one of a vendor specific format and a product specific format.

15. The computer program product according to claim 13, wherein the program instructions to generate the dump file for each of the plurality of different computer systems comprises:

program instructions to query each of the plurality of different computer systems using an interface native to each of the plurality of different computer systems.

16. The computer program product according to claim 15, wherein the program instructions to query each of the plurality of different computer systems using the interface native to each of the plurality of different computer systems comprises:

program instructions to run a predetermined series of commands, wherein the interface comprises a standard command line interface.

17. The computer program product according to claim 13, wherein the configuration and state information relates to a plurality of objects stored on the plurality of different computer systems and comprises metadata about the plurality of objects.

18. The computer program product according to claim 13, wherein the configuration and state information comprises metadata which describes a physical storage location of each object and includes customer data, site data, source data, node data, object data, and fragment data, wherein the fragment data describes a physical location of each the object.

19. The computer program product according to claim 13, wherein the plurality of different computer systems use at least two different proprietary data formats.