Analyzing drive errors in data storage systems

• US 8,566,637 B1
• Filed: 12/23/2010
• Issued: 10/22/2013
• Est. Priority Date: 12/23/2010
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Puhov, Peter, Harel, Shay, Chen, Lili, Liu, Zhiqi, Hakimi, Huzefa
• Primary Examiner(s): MCCARTHY, CHRISTOPHER S
• Application Number:
  US12/977,962
• Time in Patent Office: 1,034 Days
• Field of Search: None
• US Class Current: 714/6.32
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40

Abstract

A method is used in analyzing drive errors in data storage systems. An error tag and total good I/O count are maintained for a drive. For each failed I/O, the following is performed. An error weight per error is retrieved. A new error tag is calculated from the error weight, a previous error tag, and the total good I/O count. An error ratio is calculated from the new error tag and a total I/O count. The error ratio is compared with thresholds. If one or more of the thresholds has been crossed, action is taken on the drive. The error tag is reduced as good I/O is produced.

Claims

1. A method for use in analyzing drive errors in data storage systems, the method comprising:

maintaining an error tag and total good I/O count for a drive;

for each failed I/O;

retrieving an error weight per error;

calculating a new error tag from the error weight, a previous error tag, and the total good I/O count; and

calculating an error ratio from the new error tag and a total I/O count;

comparing the error ratio with thresholds;

if one or more of the thresholds has been crossed, taking an action on the drive; and

reducing the error tag as good I/O is produced.

2. The method of claim 1, wherein the error tag is used to help provide a process for detecting drive abnormal conditions with frequency-based accumulative error information.

3. The method of claim 1, wherein a configuration file having definitions of error weights and thresholds is dynamically updated.

4. The method of claim 1, wherein the drive errors may include SCSI errors and FIS errors.

5. The method of claim 1, wherein reducing the error tag is based on I/O based error fading.

6. The method of claim 1, wherein error handling decisions are based on at least one of the following:

specific drive type, vendor, revision, and serial number range.

7. The method of claim 1, wherein errors are divided into categories with common recovery characteristics.

8. The method of claim 1, wherein the more serious an error category is, the lower the error category'"'"'s interval, the higher the error category'"'"'s weight is, and the lower the ratio between the interval and the weight.

9. The method of claim 1, wherein when a drive error happens, the drive error is recorded on an I/O count axis, and it is determined how far the recorded drive error is from a current I/O count.

10. The method of claim 1, wherein the drive error is recorded in internal data structures, the drive is reset, if a reset did not help and a reset threshold is passed, proactive sparing is started, the drive is reset last time, and if the last reset did not help, the drive is deactivated.

11. A system for use in analyzing drive errors in data storage systems, the system comprising:

a processor comprising a program logic for carrying out the steps of;

first logic maintaining an error tag and total good I/O count for a drive;

second logic retrieving an error weight per error for each failed I/O;

third logic calculating a new error tag from the error weight, a previous error tag, and the total good I/O count for each failed I/O; and

fourth logic calculating an error ratio from the new error tag and a total I/O count for each failed I/O;

fifth logic comparing the error ratio with thresholds;

sixth logic taking an action on the drive if one or more of the thresholds has been crossed; and

seventh logic reducing the error tag as good I/O is produced.

12. The system of claim 11, wherein the error tag is used to help provide a process for detecting drive abnormal conditions with frequency-based accumulative error information.

13. The system of claim 11, wherein a configuration file having definitions of error weights and thresholds is dynamically updated.

14. The system of claim 11, wherein the drive errors may include SCSI errors and FIS errors.

15. The system of claim 11, wherein reducing the error tag is based on I/O based error fading.

16. The system of claim 11, wherein error handling decisions are based on at least one of the following:

specific drive type, vendor, revision, and serial number range.

17. The system of claim 11, wherein errors are divided into categories with common recovery characteristics.

18. The system of claim 11, wherein the more serious an error category is, the lower the error category'"'"'s interval, the higher the error category'"'"'s weight is, and the lower the ratio between the interval and the weight.

19. The system of claim 11, wherein when a drive error happens, the drive error is recorded on an I/O count axis, and it is determined how far the recorded drive error is from a current I/O count.

20. The system of claim 11, wherein the drive error is recorded in internal data structures, the drive is reset, if a reset did not help and a reset threshold is passed, proactive sparing is started, the drive is reset last time, and if the last reset did not help, the drive is deactivated.