System and method for predicting single-disk failures

• US 9,189,309 B1
• Filed: 09/25/2013
• Issued: 11/17/2015
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ma, Ao, Hsu, Windsor W.
• Primary Examiner(s): Ehne, Charles
• Application Number:
  US14/037,202
• Time in Patent Office: 783 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2221 to test input/output devices or peripheral units
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 2201/81 Threshold

Abstract

Techniques for predicting single-disk failures are described herein. According to one embodiment, first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system are received. An optimal threshold of the received first values of the predetermined diagnostic parameter is determined based on a quantile distribution of the first values against a plurality of threshold candidates. A potential risk of disk failures of one or more disks of a target storage system is determined based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

Claims

1. A computer-implemented method for predicting disk failures, the method comprising:

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system;

generating a quantile distribution graph based on the first values;

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein the quantile distribution of the first values is derived from the quantile distribution graph; and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

2. The method of claim 1, wherein the plurality of threshold candidates is determined based on a range of the values of the predetermined diagnostic parameter of the known failed disks.

3. The method of claim 1, wherein the predetermined diagnostic parameter is one of reallocated sector count, medium error, timeout, pending sector count, uncorrectable sector count, connection error, and data error.

4. A computer-implemented method for predicting disk failures, the method comprising:

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system;

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein determining an optimal threshold comprises;

generating a first set of data points based on values of the predetermined diagnostic parameter of the known failed disks in view of the threshold candidates,

generating a second set of data points based on values of the predetermined diagnostic parameter of the known working disks in view of the threshold candidates, and

comparing corresponding pairs of the first and second sets of data points associated with each of the threshold candidates to select one of the threshold candidates as the optimal threshold; and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

5. The method of claim 4, wherein each of the first set of data points represents a number of the failed disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

6. The method of claim 5, wherein each of the second set of data points represents a number of the working disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

7. The method of claim 4, wherein comparing corresponding pairs of the first and second sets of data points comprises:

determining a difference value between each pair of data points from the first and second sets of data points corresponding to each of the threshold candidates; and

selecting a threshold candidate as the optimal threshold that has a maximum difference value amongst all difference values of all threshold candidates.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform a operations for predicting disk failures, the operations comprising:

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system;

generating a quantile distribution graph based on the first values;

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein the quantile distribution of the first values is derived from the quantile distribution graph; and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

9. The non-transitory machine-readable medium of claim 8, wherein the plurality of threshold candidates is determined based on a range of the values of the predetermined diagnostic parameter of the known failed disks.

10. The non-transitory machine-readable medium of claim 8, wherein the predetermined diagnostic parameter is one of reallocated sector count, medium error, timeout, pending sector count, uncorrectable sector count, connection error, and data error.

11. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for predicting disk failures, the operations comprising:

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system;

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein determining an optimal threshold comprises;

generating a first set of data points based on values of the predetermined diagnostic parameter of the known failed disks in view of the threshold candidates,

generating a second set of data points based on values of the predetermined diagnostic parameter of the known working disks in view of the threshold candidates, and

comparing corresponding pairs of the first and second sets of data points associated with each of the threshold candidates to select one of the threshold candidates as the optimal threshold; and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

12. The non-transitory machine-readable medium of claim 11, wherein each of the first set of data points represents a number of the failed disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

13. The non-transitory machine-readable medium of claim 12, wherein each of the second set of data points represents a number of the working disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

14. The non-transitory machine-readable medium of claim 11, wherein comparing corresponding pairs of the first and second sets of data points comprises:

determining a difference value between each pair of data points from the first and second sets of data points corresponding to each of the threshold candidates; and

selecting a threshold candidate as the optimal threshold that has a maximum difference value amongst all difference values of all threshold candidates.

15. A data processing system, comprising:

a processor; and

a memory coupled to the processor for storing instructions, which when executed from the memory, cause the processor to perform operations, the operations including

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system,

generating a quantile distribution graph based on the first values,

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein the quantile distribution of the first values is derived from the quantile distribution graph, and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

16. The system of claim 15, wherein the plurality of threshold candidates is determined based on a range of the values of the predetermined diagnostic parameter of the known failed disks.

17. The system of claim 15, wherein the predetermined diagnostic parameter is one of reallocated sector count, medium error, timeout, pending sector count, uncorrectable sector count, connection error, and data error.

18. A data processing system, comprising:

a processor; and

a memory coupled to the processor for storing instructions, which when executed from the memory, cause the processor to perform operations, the operations including

receiving first values of a predetermined diagnostic parameter collected from a set of known working disks and a set of known failed disks of a storage system

determining an optimal threshold of the received first values of the predetermined diagnostic parameter based on a quantile distribution of the first values against a plurality of threshold candidates, wherein

determining an optimal threshold comprises;

generating a first set of data points based on values of the predetermined diagnostic parameter of the known failed disks in view of the threshold candidates,

generating a second set of data points based on values of the predetermined diagnostic parameter of the known working disks in view of the threshold candidates, and

comparing corresponding pairs of the first and second sets of data points associated with each of the threshold candidates to select one of the threshold candidates as the optimal threshold, and

determining a potential risk of disk failures of one or more disks of a target storage system based on second values of the predetermined diagnostic parameter collected from the one or more disks of the target storage system in view of the optimal threshold.

19. The system of claim 18, wherein each of the first set of data points represents a number of the failed disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

20. The system of claim 19, wherein each of the second set of data points represents a number of the working disks that have a value of the predetermined diagnostic parameter greater than a corresponding threshold candidate.

21. The system of claim 18, wherein comparing corresponding pairs of the first and second sets of data points comprises:

determining a difference value between each pair of data points from the first and second sets of data points corresponding to each of the threshold candidates; and

selecting a threshold candidate as the optimal threshold that has a maximum difference value amongst all difference values of all threshold candidates.