Early raid rebuild to improve reliability

• US 10,013,321 B1
• Filed: 06/22/2016
• Issued: 07/03/2018
• Est. Priority Date: 06/22/2016
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Stern, Richard M
• Primary Examiner(s): Bryan, Jason B
• Application Number:
  US15/190,056
• Time in Patent Office: 741 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/86 Event-based monitoring

Abstract

A method of minimizing rebuild times within a large-scale data storage system, such as a RAID array by: maintaining a spare disk for a plurality of disks within a disk array; monitoring the plurality of disks for occurrence of one or more pre-failure indicators; maintaining, for each disk, a count of the occurrences of the pre-failure indicators; comparing the count for each disk to a defined threshold value; and copying the first disk and mirroring write operations to the first disk to the spare disk if the count for the first disk exceeds the threshold. The method switches the copying to the spare disk from the first disk to a second disk if the count for the second disk exceeds the count for the first disk. In this manner, certain predictive information can be used to use the spare disk to reduce RAID rebuild times to near instantaneous periods.

Claims

1. A method of minimizing rebuild times within a large-scale data storage system comprising:

maintaining a spare disk for a plurality of disks within a disk array;

monitoring the plurality of disks for occurrence of one or more pre-failure indicators;

maintaining, for each disk, a count of the occurrences of the one or more pre-failure indicators;

comparing the count for each disk to a defined threshold value;

copying a first disk and mirroring write operations to the first disk to the spare disk if the count for the first disk exceeds the threshold; and

switching from copying to the spare disk from the first disk to copying to the spare disk from a second disk if the count for the second disk exceeds the count for the first disk.

2. The method of claim 1 wherein the one or more pre-failure indicators comprise a medium error (ME) event and a re-allocated sector (RAS) occurrence.

3. The method of claim 2 wherein the monitoring step comprises detecting reported ME events as they occur, and checking a smart data area of the first disk storing a count of the RAS occurrences.

4. The method of claim 3 wherein the maintaining step maintains a weighted count of the occurrences of the ME events and the RAS occurrences, and wherein a relative weight of the ME events versus the RAS occurrences is defined by the absolute value of weights w₁ and w₂ assigned respectively to the ME events and RAS occurrences.

5. The method of claim 3 further comprising maintaining for the count for each disk, a rate factor for each of the ME events and RAS occurrences calculated for measured time intervals between each ME event and RAS occurrence.

6. The method of claim 1 wherein the array of storage devices comprises a RAID array.

7. The method of claim 4 wherein the RAID array conforms to one of a RAID5 or a RAID6 system.

8. The method of claim 7 wherein the copying step uses one of a dedicated copy process copying data blocks from the first disk to the spare disk, or a rebuild procedure defined by the RAID system.

9. A system for minimizing rebuild times within a RAID system, comprising:

a spare disk maintained for a plurality of disks within a disk array of the RAID system;

a monitor component monitoring the plurality of disks for occurrence of one or more pre-failure indicators;

a counter maintaining, for each disk, a count of the occurrences of the one or more pre-failure indicators;

a comparator comparing the count for each disk to a defined threshold value; and

a backup component copying a first disk and mirroring write operations to the first disk to the spare disk if the count for the first disk exceeds the threshold and switching from copying to the spare disk from the first disk to copying to the spare disk from a second disk if the count for the second disk exceeds the count for the first disk.

10. The system of claim 9 wherein the one or more pre-failure indicators comprise a medium error (ME) event and a re-allocated sector (RAS) occurrence.

11. The system of claim 10 further comprising a detector component detecting reported ME events as they occur, and checking a smart data area of the first disk storing a count of the RAS occurrences.

12. The system of claim 11 wherein counter maintains a weighted count of the occurrences of the ME events and the RAS occurrences, and wherein a relative weight of the ME events versus the RAS occurrences is defined by the absolute value of weights w₁ and w₂ assigned respectively to the ME events and RAS occurrences.

13. The system of claim 12 wherein the counter maintains for the count for each disk, a rate factor for each of the ME events and RAS occurrences calculated for measured time intervals between each ME event and RAS occurrence.

14. The system of claim 9 wherein RAID system comprises one of a RAID5 system or a RAID6 system.

15. The system of claim 14 wherein the backup component uses one of a dedicated copy process copying data blocks from the first disk to the spare disk, or a rebuild procedure defined by the RAID system.

16. A non-transitory computer-readable medium having stored thereon a program containing executable instructions causing a processor-based computer to perform, within a disk storage system having disk arrays with at least one associated spare disk, a method comprising:

monitoring the plurality of disks for occurrence of one or more pre-failure indicators;

maintaining, for each disk, a count of the occurrences of the one or more pre-failure indicators;

comparing the count for each disk to a defined threshold value;

copying a first disk and mirroring write operations to the first disk to the spare disk if the count for the first disk exceeds the threshold; and

switching from copying to the spare disk from the first disk to copying to the spare disk from a second disk if the count for the second disk exceeds the count for the first disk.