Variable drive diagnostics

• US 8,972,799 B1
• Filed: 03/29/2012
• Issued: 03/03/2015
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Brooker, Marc J., Holgers, Tobias L., Wei, Danny, Parthasarathy, Madhuvanesh, Li, Yu
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): KUDIRKA, JOSEPH R
• Application Number:
  US13/434,752
• Time in Patent Office: 1,069 Days
• Field of Search: 714/25, 714/47.1, 714/47.2, 714/47.3, 714/42
• US Class Current: 714/47.2
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/2221 to test input/output devices or peripheral units

Abstract

The relative health of data storage drives may be determined based, at least in some aspects, on data access information and/or other drive operation information. In some examples, upon receiving the operation information from a computing device, a health level of a drive may be determined. The health level determination may be based at least in part on operating information received from a client entity. Additionally, a storage space allocation instruction or operation may be determined for execution. The allocation instruction or operation determined to be performed may be based at least in part on the determined health level.

Claims

1. A computer-implemented method for data storage management, comprising:

under control of one or more computer systems configured with executable instructions,

receiving, from a device of the one or more computer systems, an operating indicator associated with a data storage server, the operating indicator indicating at least one aspect of a level of ability of the data storage server to respond to data requests;

determining, based at least in part on the operating indicator, a health level of the data storage server, the health level selected from a sliding scale of health including at least an intermediate health level between a first health level and a second health level;

determining, based at least in part on the determined health level, a regime for performing one or more diagnostic tests, the regime selected from a plurality of diagnostic regimes, each regime with a varying level of diagnostic test invasiveness;

allocating an amount of storage space of the data storage server for performing the one or more diagnostic tests;

storing the one or more diagnostic tests in the data storage server prior to performing the one or more diagnostic tests; and

performing, based at least in part on the determined regime, the one or more diagnostic tests.

2. The computer-implemented method of claim 1, wherein the sliding scale of health includes at least a first health level that indicates a low probability of failure, a second health level that indicates a medium probability of failure that is higher than that of the first health level, and a third health level that indicates a high probability of failure that is higher than that of the second health level.

3. The computer-implemented method of claim 1, wherein the operating indicator associated with the data storage server is based at least in part on at least one of a spin-up time or a seek time associated with the data storage server.

4. The computer-implemented method of claim 1, further comprising:

determining, based at least in part on a result of the one or more diagnostic tests, a rehabilitation plan for storage space of the data storage server; and

performing, based at least in part on the determined rehabilitation plan, one or more rehabilitation operations on the storage space.

5. A computer-implemented method for data storage management, comprising:

under control of one or more computer systems configured with executable instructions,

receiving, from a device of the one or more computer systems, an operating indicator associated with a data storage server, the operating indicator indicating at least one aspect of a level of ability of the data storage server to respond to data requests;

determining, based at least in part on the operating indicator, a health level of the data storage server, the health level selected from a sliding scale of health including at least an intermediate health level between a first health level and a second health level;

determining, based at least in part on the determined health level, a regime for performing one or more diagnostic tests, the regime selected from a plurality of diagnostic regimes, each regime with a varying level of diagnostic test invasiveness;

performing, based at least in part on the determined regime, the one or more diagnostic tests;

storing data of the data storage server, stored in a particular storage space of the data storage server, in a second data storage server; and

allocating an amount of storage space in the data storage server for storing the one or more diagnostic tests by reallocating the particular storage space after the data of the data storage server is stored in the second data storage server to make available space to store the one or more diagnostic tests.

6. A computer-implemented method for data storage management, comprising:

under control of one or more computer systems configured with executable instructions,

receiving, from a device of the one or more computer systems, an operating indicator associated with a server;

determining, based at least in part on the operating indicator, a health level of the server, the health level selected from a plurality of possible health levels above at least a lowest level of health;

determining, based at least in part on the determined health level, a regime for performing one or more diagnostic tests, wherein the determined regime for performing the one or more diagnostic tests is inversely related to the health level; and

performing, based at least in part on the determined regime, the one or more diagnostic tests in a storage space of the server.

7. The computer-implemented method of claim 6, wherein the health level of the server indicates a probability of failure of the storage space of the server.

8. The computer-implemented method of claim 6, wherein the determined health level is a suspect health level indicating a higher probability of failure than a healthy level and a lower probability of failure than an unhealthy level.

9. The computer-implemented method of claim 6, wherein the operating indicator associated with the server is based at least in part on at least one of a number of attempts to complete a read operation, a number of attempts to complete a write operation, or a temperature associated with the server.

10. The computer-implemented method of claim 6, wherein the operating indicator associated with the server is based at least in part on at least one of a latency of requests or a variance in overall behavior associated with the server.

11. A system for data storage management, comprising:

at least one memory that stores computer-executable instructions;

at least one processor configured to access the at least one memory, wherein the at least one processor is configured to execute the computer-executable instructions to;

receive, from a device of the system, information associated with a physical or software performance indicator associated with a storage device;

determine, based at least in part on the received information, a health level of the storage device;

determine, based at least in part on the determined health level of the storage device, a diagnostic testing plan to be performed on the storage device, the diagnostic testing plan selected from a plurality of diagnostic testing plans, each diagnostic testing plan with a different level of invasiveness;

allocate an amount of storage space of the storage device for performing one or more diagnostic tests;

store the one or more diagnostic tests in the storage device prior to performing the one or more diagnostic tests; and

perform the one or more diagnostic tests on the storage device based at least in part on the diagnostic testing plan corresponding to the determined health level.

12. The system of claim 11, wherein the physical performance indicator associated with the storage device is based at least in part on at least one of a latency for a read request, a latency for a write request, a spin-up time, a seek time, a number of attempts to complete an operation, a temperature, or an overall variance in behavior associated with the storage device.

13. The system of claim 11, wherein the software performance indicator associated with the storage device is based at least in part on at least one of a type of software associated with the storage device or a predefined software detail associated with performance of the storage device.

14. The system of claim 11, wherein the health level is based at least in part on at least a first level indicating that the storage device is healthy, a second level indicating that the storage device is suspect, or a third level indicating that the storage device is unhealthy.

15. The system of claim 11, wherein the at least one processor is further configured to execute the computer-executable instructions to:

determine, based at least in part on a result of the one or more diagnostic tests, a rehabilitation plan for storage space of the storage device; and

perform, based at least in part on the determined rehabilitation plan, one or more rehabilitation operations on the storage device.

16. The system of claim 11, wherein the device of the system comprises a client entity virtually attached to the storage device.

17. One or more non-transitory computer-readable media collectively storing computer-executable instructions for data storage management that, when executed by one or more processors, configures the one or more processors to perform operations comprising:

determining, based at least in part on operation information associated with a storage space of a data server, a health level of the storage space from a sliding scale of health including at least a middle health level between a first health level and a second health level;

determining, based at least in part on the determined health level of the storage space, a diagnostic testing regime to be performed on the storage space corresponding to the determined health level;

storing, at least on the data server, computer executable instructions for performing one or more diagnostic tests;

storing data of the data server in a second data server to generate storage space of the data server for the computer executable instructions for performing the one or more diagnostic tests; and

reallocating the storage space of the data server for storing the computer-executable instructions for performing the one or more diagnostic tests prior to storing, at least on the data server, the computer executable instructions for performing the one or more diagnostic tests.

18. The one or more non-transitory computer-readable media of claim 17, wherein the health level of the storage space is based at least in part on at least the first health level, the middle health level, or the second health level.

19. The one or more non-transitory computer-readable media of claim 18, wherein the sliding scale of health is based at least in part on at least a healthy level indicating a first failure probability, a suspect level that is less healthy than the healthy level, the suspect level indicating a second failure probability higher than the first failure probability, and an unhealthy level that is less healthy than the suspect level, the unhealthy level indicating a third failure probability higher than the second failure probability.

20. The one or more non-transitory computer-readable media of claim 17, wherein the operation information is based at least in part on at least one of a write latency or a read latency associated with the storage space.