High performance pooled hot spares

• US 8,086,893 B1
• Filed: 07/31/2009
• Issued: 12/27/2011
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: MacFarland, Jeffrey S., Fore, R. Christopher
• Primary Examiner(s): Le, Dieu-Minh
• Application Number:
  US12/534,017
• Time in Patent Office: 879 Days
• Field of Search: 714/3, 714/4.11, 714/4.12, 714/6.1, 714/6.11, 714/6.12, 714/6.13, 714/6.2, 714/6.21, 714/6.22, 714/6.23, 714/6.24, 714/6.3, 714/6.32, 714/11.114, 714/132, 714/162, 711/114, 711/132, 711/162
• US Class Current: 714/3
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks

Abstract

A high-performance spare disk pool is created as a logical construct for provisioning and managing striped hot spares. Two or more drives (nonvolatile mass storage devices) are logically grouped together to allow the input/output (I/O) write bandwidth during RAID reconstruction to be increased over that of a single drive. The reconstructed data can be written to the spare disk pool in a striped manner. After the reconstruction is complete, a background process can be executed to copy the reconstructed data on the spare disk pool back to a single physical spare disk drive with minimal performance impact.

Claims

1. A method comprising:

detecting a failure of a particular storage device in a redundant array of non-volatile mass storage devices in a storage system; and

reconstructing data stored in the particular storage device from data stored in at least one other non-volatile mass storage device in the redundant array, by writing to a logical object which collectively represents two or more non-volatile mass storage devices in a hot spare pool of non-volatile mass storage devices, to cause reconstructed data to be written in a distributed manner across the two or more non-volatile mass storage devices represented by the logical object.

2. A method as recited in claim 1, further comprising:

copying the reconstructed data from said two or more non-volatile mass storage devices to a single physical storage device as a background process of the storage system.

3. A method as recited in claim 2, wherein said copying comprises:

copying the reconstructed data to a single non-volatile mass storage device outside the hot spare pool.

4. A method as recited in claim 2, wherein said copying comprises:

collecting the reconstructed data on a single selected non-volatile mass storage device in the hot spare pool; and

then

logically removing the single selected non volatile mass storage device from the hot spare pool.

5. A method as recited in claim 1, further comprising:

creating the logical object in response to the failure of the particular storage device.

6. A method as recited in claim 5, further comprising:

in response to the failure, determining a number of non-volatile mass storage devices from the hot spare pool that are to be represented by the logical object, based on a specified criterion.

7. A method as recited in claim 5, further comprising:

in response to the failure, selecting said two or more non-volatile mass storage devices from among at least three non-volatile mass storage devices assigned to the hot spare pool that are to be represented by the logical object, based on a specified criterion.

8. A method as recited in claim 7, wherein the specified criterion is from the list consisting of:

a number of storage devices assigned to participate in a given reconstruction;

a number of concurrent reconstructions within the storage system;

a number of concurrent reconstructions in which a single storage device can participate;

a service-level agreement for the redundant array;

a number of non-failed storage devices in the redundant array; and

a current host workload for the redundant array.

9. A method as recited in claim 1, further comprising:

logically removing a non-volatile mass storage device from the hot spare pool and assigning the removed non-volatile mass storage device for use other than as a member of the hot spare pool, when said non-volatile mass storage device is not needed for data reconstruction.

10. A method as recited in claim 1, further comprising:

using the hot spare pool as hot spare support for a plurality of redundant arrays of non-volatile mass storage devices in the storage system, such that the hot spare pool is available for reconstruction of data from a failed storage device in any of the plurality of redundant arrays.

11. A method as recited in claim 1, further comprising:

using a solid-state non-volatile memory device as at least part of a read cache of the storage system during a non-failure mode of operation of the storage system; and

in response to the failure of the particular storage device in the redundant array of non-volatile mass storage devices, using the solid-state non-volatile memory device as one of the storage devices represented by the logical object, to reconstruct data stored in the particular storage device.

12. A method as recited in claim 11, further comprising:

copying reconstructed data from the solid-state non-volatile memory device to another hot spare storage device while using the solid-state non-volatile memory device as part of the read cache.

13. A method comprising:

detecting a failure of a particular storage device in a RAID array that includes a plurality of non-volatile mass storage devices of a network storage server;

in response to the failure of the particular storage device, determining which of a plurality of non-volatile mass storage devices in a hot spare pool are to be used for reconstruction of data stored in the particular storage device, based on a specified criterion, and mapping the determined non-volatile mass storage devices in the hot spare pool to a logical storage object;

reconstructing data stored in the particular storage device by using a RAID reconstruction technique, including writing to the logical storage object to cause reconstructed data to be written in a distributed manner across the determined non-volatile mass storage devices in the hot spare pool; and

copying the reconstructed data from the determined non-volatile mass storage devices in the hot spare pool to a single physical storage device as a background process of the network storage server.

14. A method as recited in claim 13, wherein said copying comprises:

copying the reconstructed data to a single non-volatile mass storage device outside the hot spare pool.

15. A method as recited in claim 13, wherein said copying comprises:

collecting the reconstructed data on a single selected non-volatile mass storage device in the hot spare pool; and

then

logically removing the single selected non-volatile mass storage device from the hot spare pool.

16. A method as recited in claim 13, wherein determining the plurality of non-volatile mass storage devices in the hot spare pool to be used for reconstruction of data comprises:

in response to the failure, determining a number of non-volatile mass storage devices in the hot spare pool that are to be mapped to the logical storage object.

17. A method as recited in claim 13, further comprising:

logically removing a non-volatile mass storage device from the hot spare pool and assigning the removed non-volatile mass storage device for use other than as a member of the hot spare pool, when said non-volatile mass storage device is not needed for data reconstruction.

18. A method as recited in claim 13, further comprising:

using the hot spare pool as hot spare support for a plurality of RAID arrays in the network storage server, such that the hot spare pool is available for reconstruction of data from a failed storage device in any of the plurality of RAID arrays.

19. A method as recited in claim 13, further comprising:

using a solid-state non-volatile memory device as at least part of a read cache of the network storage server during a non-failure mode of operation of the network storage server; and

in response to the failure of the particular storage device, using the solid-state non-volatile memory device as one of the storage devices represented by the logical storage object, to reconstruct data stored in the particular storage device.

20. A method as recited in claim 19, further comprising:

copying reconstructed data from the solid-state non-volatile memory device to another hot spare storage device while using the solid-state non-volatile memory device as part of the read cache.

21. A processing system comprising:

a processor;

a storage interface through which to access a first redundant array of non-volatile mass storage devices; and

a storage access module configured to detect a failure of a particular storage device in the first redundant array of non-volatile mass storage devices, and further configured to reconstruct data stored in the particular storage device in response to the failure of the particular storage device, by writing to a logical object which collectively represents two or more non-volatile mass storage devices in a hot spare pool of non-volatile mass storage devices, and further configured to cause reconstructed data to be written in a distributed manner across the two or more non-volatile mass storage devices represented by the logical object.

22. A processing system as recited in claim 21, wherein the storage access module is further configured to copy the reconstructed data from said two or more non-volatile mass storage devices to a single physical hot spare storage device as a background process of the processing system.

23. A processing system as recited in claim 22, wherein copying the reconstructed data from said two or more non-volatile mass storage devices to a single physical storage device comprises:

copying the reconstructed data to a single non-volatile mass storage device outside the hot spare pool.

24. A processing system as recited in claim 22, wherein said copying the reconstructed data from said two or more non-volatile mass storage devices to a single physical storage device comprises:

collecting the reconstructed data on a single selected non-volatile mass storage device in the hot spare pool; and

then

logically removing the single selected non volatile mass storage device from the hot spare pool.

25. A processing system as recited in claim 21, wherein the storage access module is further configured to create the logical object in response to the failure of the particular storage device.

26. A processing system as recited in claim 25, wherein the storage access module is further configured to determine a number of non-volatile mass storage devices from the hot spare pool that are to be represented by the logical object in response to the failure, based on a specified criterion.

27. A processing system as recited in claim 25, wherein the storage access module is further configured to select said two or more non-volatile mass storage devices from among at least three non-volatile mass storage devices assigned to the hot spare pool that are to be represented by the logical object in response to the failure, based on a specified criterion.

28. A processing system as recited in claim 27, wherein the specified criterion is from the list consisting of:

a number of storage devices assigned to participate in a given reconstruction;

a number of concurrent reconstructions within the processing system;

a number of concurrent reconstructions in which a single storage device can participate;

a service-level agreement for the first redundant array;

a number of non-failed storage devices in the first redundant array; and

a current host workload for the first redundant array.

29. A processing system as recited in claim 21, wherein the storage access module is further configured to reassign a non-volatile mass storage device from the hot spare pool to a use other than as a member of the hot spare pool, when said non-volatile mass storage device is not needed for data reconstruction.

30. A processing system as recited in claim 21, wherein the storage access module is further configured to use the hot spare pool as hot spare support for both the first redundant array and a second redundant array of non-volatile mass storage devices, such that the hot spare pool is available for reconstruction of data from a failed storage device in either of the first and second redundant arrays.

31. A processing system as recited in claim 21, further comprising:

a solid-state non-volatile memory device;

logic to use the solid-state non-volatile memory device as at least part of a read cache of the processing system during a non-failure mode of operation of the processing system; and

logic to respond to the failure of the particular storage device in the first redundant array by using the solid-state non-volatile memory device as one of the storage devices represented by the logical object, to reconstruct data stored in the particular storage device.

32. A processing system as recited in claim 31, further comprising:

logic to copy reconstructed data from the solid-state non-volatile memory device to another hot spare storage device while using the solid-state non-volatile memory device as part of the read cache.

33. A network storage server comprising:

a processor;

a network interface through which to receive a data access request from a storage client and through which to send to the storage client a response to the request;

a storage interface through which to access a first redundant array of non-volatile mass storage devices;

logic to detect a failure of a particular storage device in the first redundant array

logic to determine, in response to the failure of the particular storage device, which of a plurality of non-volatile mass storage devices in a hot spare pool are to be used for reconstruction of data stored in the particular storage device based on a specified criterion, and to map the determined non-volatile mass storage devices in the hot spare pool to a logical storage object; and

logic to reconstruct data stored in the particular storage device by writing to the logical storage object, to cause reconstructed data to be written in a distributed manner across the determined non-volatile mass storage devices in the hot spare pool.

34. A network storage server as recited in claim 33, further comprising:

logic to copy the reconstructed data from the determined non-volatile mass storage devices in the hot spare pool to a single physical storage device as a background process of the network storage server.

35. A network storage server as recited in claim 34, wherein copying the reconstructed data from the determined non-volatile mass storage devices in the hot spare pool to a single physical storage device comprises:

copying the reconstructed data to a single non-volatile mass storage device outside the hot spare pool.

36. A network storage server as recited in claim 34, wherein copying the reconstructed data from the determined non-volatile mass storage devices in the hot spare pool to a single physical hot spare storage device comprises:

collecting the reconstructed data on a single selected non-volatile mass storage device in the hot spare pool; and

then

logically removing the single selected non-volatile mass storage device from the hot spare pool.

37. A network storage server as recited in claim 33, wherein the logic to determine which of the plurality of non-volatile mass storage devices in the hot spare pool are to be used for reconstruction of data comprises:

logic to determine a number of non-volatile mass storage devices in the hot spare pool that are to be mapped to the logical storage object.

38. A network storage server as recited in claim 33, further comprising:

logic to reassign a non-volatile mass storage device from the hot spare pool to a use other than as a member of the hot spare pool, when said non-volatile mass storage device is not needed for data reconstruction.

39. A network storage server as recited in claim 33, further comprising:

logic to use the hot spare pool as hot spare support for both the first redundant array and a second redundant array of non-volatile mass storage devices, such that the hot spare pool is available for reconstruction of data from a failed storage device in either of the first and second redundant arrays.

40. A network storage server as recited in claim 33, further comprising:

a solid-state non-volatile memory device;

logic to use the solid-state non-volatile memory device as at least part of a read cache of the network storage server during a non-failure mode of operation of the network storage server; and

logic to respond to the failure of the particular storage device in the first redundant array by using the solid-state non-volatile memory device as one of the storage devices represented by the logical object, to reconstruct data stored in the particular storage device.

41. A network storage server as recited in claim 40, further comprising:

logic to copy reconstructed data from the solid-state non-volatile memory device to another hot spare storage device while using the solid-state non-volatile memory device as part of the read cache.