Method and system for reliability analysis of disk drive failures

• US 7,136,768 B1
• Filed: 03/31/2005
• Issued: 11/14/2006
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Shah, Sandeep, Elerath, Jon G.
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): Huynh, Phuong
• Application Number:
  US11/095,249
• Time in Patent Office: 593 Days
• Field of Search: 702/115, 702182-188, 707 3- 5
• US Class Current: 702/115
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A method and system for performing reliability analysis of disk drive failure mechanisms is provided. The information for performing the analysis is obtained in accordance with the invention from a database generated from identification information stored about individual drives, and drive families, that are deployed in the field. An error database stores error codes that are issued by a disk drive upon a particular event. These error codes are reported to a storage system administrator and recorded in the error database. The disk drive information and the error codes are mapped, and error codes are translated into failure mechanisms for a particular drive family. An analysis is performed whereby a hazard rate plot is provided for either all failure indicators or selected failure indicators or subpopulations for a particular drive family over a given time.

Claims

1. A method of performing reliability analysis in disk drives in an installation base comprising the steps of:

recording identification information about each disk drive in a family of disk drives in the installation base;

recording error codes that are generated by the disk drives operating in the installation base;

creating a reliability database including mapping each recorded error code with a particular drive family;

adding to said reliability database further information from a drive manufacturer about failure mechanisms associated with one or more error codes; and

retrieving one or more error codes reported for a disk drive family during a time period, and matching the retrieved error codes with one or more failure mechanisms for that drive family.

2. The method of performing reliability analysis in disk drives as defined in claim 1 including the further step of calculating rates of change of failure indicators including error codes, failure modes and failure mechanisms and determining whether a rate of change of one or more of said failure indicators is increasing, decreasing or remaining constant.

3. The method of performing reliability analysis in disk drives as defined in claim 1 including the further step of plotting a hazard rate of one or more failure indicators including error codes, failure modes and failure mechanisms for a particular drive family.

4. The method of performing reliability analysis in disk drives as defined in claim 1 including the further step of plotting a hazard rate by first performing one or more data analysis techniques on retrieved error code data, or failure indicators including maximum likelihood estimates, goodness of fit tests, statistical distributions and/or curve fitting techniques to generate a linear plot of hazard rates.

5. The method of performing reliability analysis in disk drives as defined in claim 1 including dividing a disk drive family into subpopulations based upon predetermined conditions or events.

6. The method of performing reliability analysis in disk drives as defined in claim 5, including using drive capacity as a subpopulation.

7. The method of performing reliability analysis in disk drives as defined in claim 6 including the further steps of selecting drives of a low capacity from a particular drive family and analyzing one or more failure indicators for low capacity drives occurring in a given time period, and selecting drives of a high capacity from that particular drive family and analyzing one or more failure indicators for high capacity drives over the same time period.

8. The method of performing reliability analysis in disk drives as defined in claim 5 including the further step of plotting a hazard rate of one or more failure indicators for one or more of said subpopulations of disk drives.

9. A method of performing reliability analysis in disk drives in an installation base comprising the steps of:

recording identification information about each disk drive in a family of disk drives in the installation base;

recording error codes that are generated by the disk drives operating in the installation base;

creating a reliability database including mapping each recorded error code with a particular drive family;

adding to said reliability database further information from a drive manufacturer about failure mechanisms associated with one or more error codes;

retrieving one or more error codes reported for a disk drive family during a time period, and matching the retrieved error codes with one or more failure mechanisms for that drive family; and

dividing a disk drive family into subpopulations based upon predetermined conditions or events using disk drives that have undergone an upgrade or a new firmware version as a subpopulation.

10. A method of performing reliability analysis in disk drives in an installation base comprising the steps of:

recording identification information about each disk drive in a family of disk drives in the installation base;

recording error codes that are generated by the disk drives operating in the installation base;

creating a reliability database including mapping each recorded error code with a particular drive family;

adding to said reliability database further information from a drive manufacturer about failure mechanisms associated with one or more error codes;

retrieving one or more error codes reported for a disk drive family during a time period, and matching the retrieved error codes with one or more failure mechanisms for that drive family; and

dividing a disk drive family into subpopulations based upon predetermined conditions or events using disk drives are operating under a new or upgraded operating system software version as a subpopulation.

11. A system configured to perform reliability analysis on disk drives, comprising:

(A) a reliability database configured to store;

(i) information about each drive in a particular drive family;

(ii) error information including a failure mechanism associated with each error code from each drive family;

(iii) a mapping between individual error codes received, and drives within a drive family; and

(B) a reliability analysis utility configured to extract data from said reliability database and to construct one or more hazard rate plots of failure indicators associated with particular drive families.

12. A system adapted to perform reliability analysis on disk drives, comprising:

(A) a storage system adapted to execute an autosupport utility that is adapted to forward error code information generated about disk drives associated with the storage system to a storage system administrator;

(B) a reliability analysis utility adapted to receive and parse said error code information and to perform a reliability analysis thereupon; and

(C) a graphical user interface adapted to display a representation of said reliability analysis.

13. The system as defined in claim 12 wherein said reliability analysis includes computation of one or more hazard rates for error code information about selected disk drives and said graphical user interface is adapted to display a hazard rate plot for one or more of said hazard rates.

14. The system as defined in claim 12 further comprising an error log associated with said storage system and wherein said autosupport utility periodically retrieves error code information from said error log of said storage system and forwards retrieved error code information to the storage system administrator.

15. The system as defined in claim 12 further comprising an error log associated with said storage system and said autosupport utility is adapted to poll said error log to retrieve updated error code information and said autosupport utility forwards updated error code information to the storage system administrator.

16. The system as defined in claim 12 wherein said reliability analysis utility is further adapted to perform a one-to-one mapping using a disk drive serial number to retrieve information from an associated error code database, and from an associated storage information database, which is unique and common to both databases, and stores this information in a reliability database.

17. The system as defined in claim 16 wherein said reliability analysis utility uses information in said reliability database to analyze failure indicators occurring with respect to particular disk drive families associated with the system.

18. A system configured to perform reliability analysis on disk drive failures indicators comprising:

means for reporting error codes that are generated by disk drives in said field installation;

means for extracting information about an individual disk drive family and error codes reported on drives in that drive family over a time period;

means for analyzing failure indicators associated with said reported error codes about said individual drive family including means for generating a hazard rate plot of one or more failure indicators over a time period.

19. A computer readable medium for performing reliability analysis on disk drive failure indicators, including program instructions for performing the steps of:

generating a reliability database containing information regarding disk drive identity information and error log information stored in an associated computer;

selecting a particular drive family to study;

extracting from said reliability database one or more error codes representing failure indicators about a particular drive or drive family which have been received over a time period.

20. The computer readable medium for performing reliability analysis on disk drive failure indicators as defined in claim 19 including programming instructions for performing the further step of plotting one or more hazard rates for one or more failure indicators that have occurred over a time period for that particular drive family.

21. The computer readable medium for performing reliability analysis on disk drive failure indicators as defined in claim 19 including programming instructions for performing the further step of mapping a particular error code from a disk drive with the drive family to which the particular disk drive belongs by performing a one-to-one mapping using a disk drive serial number which is unique and common to both databases, and stores this information in said reliability database.

22. The computer readable medium for performing reliability analysis on disk drive failure indicators as defined in claim 19 including programming instructions for performing the further step of mapping the particular error code determined to be from a disk drive with a drive family to a specific failure mode or failure mechanism.

23. A method of performing reliability analysis in disk drive failure indicators comprising:

generating a reliability database containing information regarding disk drive identity information and error log information stored in an associated computer;

selecting a particular drive family to study from the reliability database; and

extracting from said reliability database one or more error codes representing failure indicators about the particular drive family which have been received over a time period.

24. The method of performing reliability analysis in disk drive failure indicators as defined in claim 23, further comprising:

dividing a disk drive family into subpopulations based upon predetermined conditions or events.

25. The method of performing reliability analysis in disk drive failure indicators as defined in claim 24, further comprising:

plotting a hazard rate of one or more failure indicators for one or more of said subpopulations of disk drives.

26. A system of performing reliability analysis in disk drive failure indicators comprising:

means for generating a reliability database containing information regarding disk drive identity information and error log information stored in an associated computer;

means for selecting a particular drive family to study from the reliability database; and

means for extracting from said reliability database one or more error codes representing failure indicators about the particular drive family which have been received over a time period.

27. The system of performing reliability analysis in disk drive failure indicators as defined in claim 26 further comprising:

means for dividing a disk drive family into subpopulations based upon predetermined conditions or events.

28. The system of performing reliability analysis in disk drive failure indicators as defined in claim 27, further comprising:

means for plotting a hazard rate of one or more failure indicators for one or more of said subpopulations of disk drives.

29. A system of performing reliability analysis in disk drive failure indicators comprising:

a reliability database containing information regarding disk drive identity information and error log information stored in an associated computer;

a reliability analysis utility configured to select a particular drive family to study from the reliability database; and

the reliability analysis utility further configured to extract from the reliability database one or more error codes representing failure indicators about the particular drive family which have been received over a time period.