System and method to predict reliability of backup software

• US 10,346,237 B1
• Filed: 08/28/2015
• Issued: 07/09/2019
• Est. Priority Date: 08/28/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Surendra Rao, Naveen Kumar, Lohchab, Sanjeev
• Primary Examiner(s): Truong, Loan L. T.
• Application Number:
  US14/838,940
• Time in Patent Office: 1,411 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/0778 Dumping, i.e. gathering error/state information after a fault for later diagnosis
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

Embodiments are directed to method of determining the reliability of a software program by correlating reliability with performance of the system through monitoring the entire system and it components. A component captures memory usage and CPU utilization of all components at regular interval and records failure of services by the system. An analyzer analyzes the events performed to determine which component failed to complete the action and record the failure against that component to enable identification of individual component reliability as well as the product as a whole.

Claims

1. A computer-implemented method of measuring reliability of a deduplication backup program executed by a backup server in an enterprise-level network including a plurality of backup appliances including storage nodes, comprising:

monitoring, in a capture process executed in a processor of a reliability module in the backup server, a plurality of components in the backup appliances and the deduplication backup program for an overall monitoring period;

measuring, in a data analyzer process executed in the processor, a functional performance of each of the plurality of components based on a failure of a storage node due to a failure type of a plurality of failure types, wherein the functional performance is measured through continuous daemon processes on a periodic basis defined by a time period;

comparing the measured functional performance to defined performance values to identify a specific failed component of the storage node to distinguish from failure of the deduplication backup program as a whole; and

deriving a reliability measure by summing a weighted value of a total number of failures of the plurality of failure types per the overall monitoring period through a formula that sums weighted failure types multiplied by the total number of failures per a total time period, and as expressed by;

Reliability factor=Σ

Weighting to a failure*No of failure/Total Time in hour,

to provide a measure of analytical metrics that can be used to analyze overall network performance to identify failure patterns caused by the plurality of components to thereby improve the reliability of the deduplication backup program.

2. The method of claim 1 further comprising assigning different weighting values to each failure type of the plurality of failure types, wherein the storage node is part of a backup appliance executing a client-side deduplication process distributed by the backup server.

3. The method of claim 2 wherein the functional performance comprises at least one of processor usage, memory usage, and network input/output throughput.

4. The method of claim 3 wherein the plurality of components is selected from the group consisting of:

central processing units (CPUs), application programs, storage devices, and network links.

5. The method of claim 4 wherein the types of failures including a program crash, a program hang condition, a memory leak, a performance deviation in excess of a defined threshold, a major functional failure of a core feature of the deduplication backup program, and a minor functional failure of a non-core feature of the deduplication backup program.

6. The method of claim 1 wherein the time period is defined by one of a user or a default system setting.

7. The method of claim 6 wherein the overall monitoring period is on the order of multiple weeks, and wherein the time period is on the order of multiple minutes.

8. The method of claim 1 wherein the defined performance values comprise past performance values compiled or an earlier version of the deduplication backup program.

9. The method of claim 7 wherein the network comprises a multi-tenant network in which the backup server runs a single instance of a program serving multiple clients embodied as the backup appliances, and in which the deduplication backup program virtually partitions its data so that each client works with its own customized virtual application.

10. The method of claim 9 wherein the backup server and deduplication backup program comprise a Data Domain Restorer (DDR)-based deduplication storage system.

11. A system comprising a processor-based executable module configured to derive a reliability metric of a deduplication backup program executed by backup server in an enterprise-level network including a plurality of backup appliances including storage nodes, comprising:

a processor of the backup server having an administrator module executing first program instructions to monitor a plurality of components in the backup appliances and the deduplication backup program during a monitoring time period;

a capture module of the processor executing second program instructions in the processor to capture data and events of the components through continuous daemon processes running on a periodic basis defined a measurement period;

a failure analytics module of the processor executing third program instructions to analyze a log of the data and events to capture failure conditions to determine which process failed an expected operation causing failure of a storage node;

a data analyzer of the processor evaluating performance data of the software program to identify a specific failed component of the storage node to distinguish from failure of the deduplication backup program as a whole; and

and reliability module executed by the processor and deriving a reliability measure using a weighted value formula, and deriving a reliability measure through a formula that sums weighted failure types multiplied by a number of failures per a total time period, and as expressed by;

Reliability factor=Σ

Weighting to a failure*No of failure/Total Time in hour,

to provide a measure of analytical metrics that can be used to analyze overall network performance to identify failure patterns caused by the plurality of components to thereby improve the reliability of the deduplication backup program.

12. The system of claim 11 wherein the data and events comprise CPU utilization, memory utilization, I/O throughput for each component, and wherein the storage node is part of a backup appliance executing a client-side deduplication process distributed by the backup server.

13. The system of claim 12 wherein the failure analytics module captures conditions selected from the group consisting of:

program crashes, program hang conditions, memory leaks, performance deviations in excess of a defined threshold, major functional failures of a core feature of the deduplication backup program, and minor functional failures of a non-core feature of the deduplication backup program.

14. The system of claim 13 wherein the data analyzer benchmarks the deduplication backup program performance against a defined standard, comprising one of an earlier release and a comparative backup program.

15. The system of claim 14 wherein the monitoring time period is on the order of multiple weeks, and wherein the measurement period is on the order of multiple minutes.

16. The system of claim 11 wherein the network comprises a multi-tenant network in which the backup server runs a single instance of a program serving multiple clients embodied as the backup appliances, and in which the deduplication backup program virtually partitions its data so that each client works with its own customized virtual application.

17. The system of claim 16 wherein the measurement period is defined by one of a user or a default system setting.

18. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method for measuring reliability of a deduplication backup program executed by a backup server in an enterprise-level network including a plurality of backup appliances including storage nodes by:

monitoring, in a capture process executed in a processor of a reliability module in the backup server, a plurality of components in the backup appliances and the deduplication backup program for an overall monitoring period;

measuring, in a data analyzer process executed in the processor, a functional performance of each of the plurality of components based on a failure of a storage node due to a failure type of a plurality of failure types, wherein the functional performance is measured through continuous daemon processes on a periodic basis defined by a time period;

comparing the measured functional performance to defined performance values to identify a specific failed component of the storage node to distinguish from failure of the deduplication backup program as a whole; and

deriving a reliability measure by summing a weighted value of a total number of failures of the plurality of failure types per the overall monitoring period through a formula that sums weighted failure types multiplied by the total number of failures per a total time period, and as expressed by;

Reliability factor=Σ

Weighting to a failure*No of failure/Total Time in hour,

to provide a measure of analytical metrics that can be used to analyze overall network performance to identify failure patterns caused by the plurality of components to thereby improve the reliability of the deduplication backup program.