On-line restoration of redundancy information in a redundant array system

• US 5,613,059 A
• Filed: 12/01/1994
• Issued: 03/18/1997
• Est. Priority Date: 12/21/1990
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Gordon, David, Brant, William A., Stallmo, David C.
• Primary Examiner(s): Beausoliel, Jr., Robert W.
• Assistant Examiner(s): Decady, Albert
• Application Number:
  US08/349,766
• Time in Patent Office: 838 Days
• Field of Search: 371/10.2, 371/10.3, 371/40.1, 371/40.2, 371/51.1, 395/182.03, 395/182.04, 395/182.05, 395/182.06
• US Class Current: 714/6.12
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata

Abstract

A method for on-line restoration of redundancy information in a redundant array storage system. The invention provides alternative methods of restoring valid data to a storage unit after a Write failure caused by a temporary storage unit fault. In the first preferred method, a valid redundancy block is generated for the corresponding data blocks on all storage units. Resubmitting the interrupted Write operation causes the old (and potentially corrupted) data block to be "subtracted" out of the re-computed redundancy block. The uncorrupted new data block is written over the old data block, and is "added" into the re-computed redundancy block to create a new, corrected redundancy block. The new, corrected redundancy block is written to the appropriate storage unit. In the second preferred method, a new redundancy block is generated from all valid data blocks and the new data block. The new redundancy block and the new data block are then written to the appropriate storage units. In both cases, the entire method is done on-line, with insignificant interruption of normal operation of the redundant array system, and without requiring added processing during normal operation.

Claims

1. A storage device readable by a programmable control device when coupled to the storage device, the storage device containing a control program tangibly stored thereon, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of a valid data block and at least one associated redundancy block to at least each affected data storage unit stripe after a potential corruption in either type of block during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) accessing all of the data blocks, including the potentially corrupted data block, in each stripe containing the potentially corrupted blocks;

(b) computing at least one redundancy block from the accessed blocks;

(c) saving the at least one computed redundancy block;

(d) resubmitting the valid data block from the data modification operation to the redundant array of storage units for storage;

(e) updating the at least one saved computed redundancy block;

(f) storing the updated at least one redundancy block and the valid data block in the stripe.

2. A storage device as set forth in claim 1, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively-OR'"'"'ing the accessed blocks.

3. A storage device as set forth in claim 1, wherein at least one redundancy block contains parity information, the control program being further configured to perform the step of updating that parity redundancy block by performing the function of:

(a) accessing the potentially corrupted data block;

(b) generating a new parity redundancy block by exclusively-OR'"'"'ing the accessed data block with the saved parity redundancy block and the resubmitted valid data block.

4. A storage device as set forth in claim 1, the control program being further configured to operate the control device to perform restoration as a task concurrently with other input/output tasks.

5. A storage device as set forth in claim 4, the control program being further configured to perform the function of locking each block being read or modified during restoration.

6. A storage device as set forth in claim 1, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array to a non-volatile storage device coupled to the control device at least until the data modification operation is completed.

7. A storage device as set forth in claim 1, the control program being further configured to operate the control device to perform restoration as a task concurrently with other read operations during which data is read from data storage units by the control device, and write operations during which data is written to the data storage units by the control device, such other read and write operations being essentially unrelated to the restoration operation.

8. A storage device readable by a programmable control device when coupled to the storage device, the storage device containing a control program tangibly stored thereon, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of a valid data block and at least one associated redundancy block to at least each affected data storage unit stripe after a potential corruption in either type of block during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) resubmitting the valid data block from the data modification operation to the redundant array of storage units for storage;

(b) accessing all of the uncorrupted data blocks in the stripe containing the potentially corrupted blocks;

(c) computing at least one redundancy block from the accessed blocks and the resubmitted valid data block;

(d) storing the computed at least one redundancy block and the valid data block in the stripe;

(e) performing restoration as a task concurrently with other input/output tasks.

9. A storage device as set forth in claim 8, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively-OR'"'"'ing the accessed blocks with the resubmitted valid data block.

10. A storage device as set forth in claim 8, the control program being further configured to perform the function of locking each block being read or modified during restoration.

11. A storage device as set forth in claim 8, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array in a non-volatile storage device coupled to the control system at least until the data modification operation is completed.

12. A storage device readable by a programmable control device when coupled to the storage device, the storage device containing a control program tangibly stored thereon, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of at least one valid redundancy block to at least each affected data storage unit stripe after a potential corruption in a data block and/or in such redundancy blocks in such stripes during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) accessing all of the data blocks, including the potentially corrupted data block, in each stripe containing the potentially corrupted blocks;

(b) computing at least one redundancy block from the accessed blocks;

(c) saving the at least one computed redundancy block in the stripe.

13. A storage device as set forth in claim 12, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively OR'"'"'ing the accessed blocks.

14. A storage device as set forth in claim 12, the control program being further configured to operate the control device to perform restoration as a task concurrently with other input/output tasks.

15. A storage device as set forth in claim 12, the control program being further configured to perform the function of locking each block being read or modified during restoration.

16. A storage device as set forth in claim 12, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array in a non-volatile storage device coupled to the control system at least until the data modification operation is completed.

17. A storage device as set forth in claim 12, the control program being further configured to operate the control device to perform restoration as a task concurrently with other read operations during which data is read from data storage units by the control device, and write operations during which data is written to the data storage units by the control device, such other read and write operations being essentially unrelated to the restoration operation.

18. A control program tangibly stored on a storage device readable by a programmable control device, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of a valid data block and at least one associated redundancy block to at least each affected data storage unit stripe after a potential corruption in either type of block during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) accessing all of the data blocks, including the potentially corrupted data block, in each stripe containing the potentially corrupted blocks;

(b) computing at least one redundancy block from the accessed blocks;

(c) saving the at least one computed redundancy block;

(d) resubmitting the valid data block from the data modification operation to the redundant array of storage units for storage;

(e) updating the at least one saved computed redundancy block;

(f) storing the updated at least one redundancy block and the valid data block in the stripe.

19. A control program as set forth in claim 18, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively-OR'"'"'ing the accessed blocks.

20. A control program as set forth in claim 18, wherein at least one redundancy block contains parity information, the control program being further configured to perform the step of updating that parity redundancy block by performing the function of:

(a) accessing the potentially corrupted data block;

(b) exclusively-OR'"'"'ing the accessed data block with the saved parity redundancy block and the resubmitted valid data block to generate a new parity redundancy block.

21. A control program as set forth in claim 18, the control program being further configured to operate the control device to perform restoration as a task concurrently with other input/output tasks.

22. A control program as set forth in claim 21, the control program being further configured to perform the function locking each block being read or modified during restoration.

23. A control program as set forth in claim 18, the data storage system further comprising a non-volatile storage device, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array in the non-volatile storage device at least until the data modification operation is completed.

24. A storage device as set forth in claim 18, the control program being further configured to operate the control device to perform restoration as a task concurrently with other read operations during which data is read from data storage units by the control device, and write operations during which data is written to the data storage units by the control device, such other read and write operations being essentially unrelated to the restoration operation.

25. A control program tangibly stored on a storage device readable by a programmable control device, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of a valid data block and at least one associated redundancy block to at least each affected data storage unit stripe after a potential corruption in either type of block during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) resubmitting the valid data block from the data modification operation to the redundant array of storage units for storage;

(b) accessing all of the uncorrupted data blocks in the stripe containing the potentially corrupted blocks;

(c) computing at least one redundancy block from the accessed blocks and the resubmitted valid data block;

(d) storing the computed at least one redundancy block and the valid data block in the stripe;

(e) performing restoration as a task concurrently with other input/output tasks.

26. A control program as set forth in claim 25, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively-OR'"'"'ing the accessed blocks with the resubmitted valid data block.

27. A control program as set forth in claim 25, the control program being further configured to perform the function of locking each block being read or modified during restoration.

28. A control program as set forth in claim 25, the data storage system further comprising a non-volatile storage device, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array in the non-volatile storage device at least until the data modification operation is completed.

29. A control program tangibly stored on a storage device readable by a programmable control device, such that the control device is operated by the control program when the storage device is read by the control device, the control device being operated to control a coupled redundant array of data storage units, the data storage units having at least one stripe containing a plurality of data blocks and at least one associated redundancy block, the control program being configured to operate the control device to perform restoration of at least one valid redundancy block to at least each affected data storage unit stripe after a potential corruption in a data block and/or in such redundancy blocks in such stripes during a data modification operation, such restoration being performed by the combination of the control program and the control device to perform the function of:

(a) accessing all of the data blocks, including the potentially corrupted data block, in each stripe containing the potentially corrupted blocks;

(b) computing at least one redundancy block from the accessed blocks;

(c) saving the at least one computed redundancy block in the stripe.

30. A control program as set forth in claim 29, wherein at least one redundancy block in each stripe contains parity information, the control program being further configured to perform the function of computing the parity redundancy block by exclusively OR'"'"'ing the accessed blocks.

31. A control program as set forth in claim 29, the control program being further configured to operate the control device to perform restoration as a task concurrently with other input/output tasks.

32. A control program as set forth in claim 29, the control program being further configured to perform the function of locking each block being read or modified during restoration.

33. A control program as set forth in claim 29, the data storage system further including a non-volatile storage device, the control program being further configured to perform the function of storing each data modification operation submitted to the redundant array in the non-volatile storage device at least until the data modification operation is completed.

34. A storage device as set forth in claim 29, the control program being further configured to operate the control device to perform restoration as a task concurrently with other read operations during which data is read from data storage units by the control device, and write operations during which data is written to the data storage units by the control device, such other read and write operations being essentially unrelated to the restoration operation.

35. A method for on-line restoration of potentially invalid data stored in one or more physical blocks of storage comprising a portion of a plurality of physical blocks of storage and wherein at least one of said physical blocks of storage in said plurality of physical blocks of storage contains redundancy information associated with data stored in said plurality of physical blocks of storage in a computer system including a computer storage subsystem comprising an array of data storage units and wherein said plurality of physical blocks of storage containing data and redundancy information are arranged at a first set of predetermined positions in at least one first storage stripe across a subset of said data storage units in said array, said method comprising the steps of:

accessing data corresponding to data stored in said plurality of physical blocks of storage in said at least one first storage stripe;

computing an interim value for said associated redundancy information for said accessed data;

resubmitting valid data corresponding to said potentially invalid data;

recomputing an updated value for said associated redundancy information from said potentially invalid data, said interim value for said associated redundancy information, and said valid data; and

storing data corresponding to said valid data and data corresponding to said updated value for said associated redundancy information to storage locations in said computer system corresponding to a second set of predetermined positions in at least one second storage stripe across a subset of said data storage units in said array.

36. The method of claim 35 wherein said step of accessing is carried out by the step of:

reading said data from the plurality of storage blocks in said data storage units in said array.

37. The method of claim 35 wherein said step of accessing comprises the step of:

determining at least one alternative non-volatile storage location within said computer system wherein a portion of said data corresponding to data stored in said plurality of physical blocks of storage in said at least one first storage stripe is maintained; and

reading said portion of said data from said at least one alternative non-volatile storage location.

38. The method of claim 35 wherein said step of accessing comprises the step of:

determining at least one alternative volatile storage location within said computer system wherein a portion of said data corresponding to data stored in said plurality of physical blocks of storage in said at least one first storage stripe is maintained; and

reading said portion of said data from said at least one alternative volatile storage location.

39. The method of claim 35 wherein said step of computing is carried out by the step of:

XORing blocks of data corresponding to data stored in said first storage stripe to provide said interim value for said associated redundancy information.

40. The method of claim 35 wherein said step of resubmitting is carried out by the step of:

reattempting a previous data modification operation which resulted in said potentially invalid data; and

completing said previous data modification operation to provide said valid data.

41. The method of claim 40 further comprising the step of:

storing data indicative of said previous data modification operation in non-volatile storage.

42. The method of claim 35 wherein said step of recomputing is carried out by the step of:

XORing said potentially invalid data, said interim value for said associated redundancy information, and said valid data, to provide said updated value for said associated redundancy information.

43. The method of claim 42 further comprising the step of:

storing said updated value for said associated redundancy information in non-volatile storage.

44. The method of claim 35 wherein said step of storing is carried out by the step of:

writing said valid data and said updated value for said associated redundancy information to said data storage units at said second set of predetermined positions in said at least one second storage stripe.

45. The method of claim 35 wherein said steps of computing and recomputing are carried out by a computer mass storage device controller operatively coupling said array of data storage units to a central processor unit of said computer system.

46. The method of claim 45 wherein said steps of accessing and storing are further carried out by said computer mass storage device controller and said array of data storage units.

47. The method of claim 35 wherein said steps of accessing, computing, resubmitting, recomputing and storing are carried out substantially concurrently with other input/output tasks of said computer mass storage device subsystem.

48. The method of claim 35 further comprising the step of:

preventing access by other input/output processes to each storage block being accessed during said on-line restoration process.

49. The method of claim 35 wherein said first set of predetermined positions in said at least one first storage stripe across a subset of said data storage units is identical to said second set of predetermined positions in said at least one second storage stripe across a subset of said data storage units.

50. A method for on-line restoration of potentially invalid data stored in one or more physical blocks of storage comprising a portion of a plurality of physical blocks of storage and wherein at least one physical block of storage in said plurality of physical blocks of storage contains redundancy information associated with the data stored in said plurality of physical blocks of storage in a computer mass storage subsystem comprising an array of data storage units and wherein said plurality of physical blocks of storage containing data and redundancy information are arranged at a first set of predetermined positions in at least one first storage stripe across a subset of said data storage units in said array, said method comprising the steps of:

accessing data corresponding to known valid data stored in said plurality of physical blocks of storage in said at least one first storage stripe;

resubmitting valid data corresponding to said potentially invalid data;

computing an updated value for said associated redundancy information from said accessed data and said valid data; and

storing data corresponding to said valid data and data corresponding to said updated value for said associated redundancy information to storage locations in said computer mass storage subsystem corresponding to a second set of predetermined positions in at least one second storage stripe across a subset of said data storage units in said array.

51. The method of claim 50 wherein said step of accessing is carried out by the step of:

reading said data from the plurality of storage blocks in said data storage units in said array.

52. The method of claim 50 wherein said step of accessing includes the step of:

determining at least one alternative non-volatile storage location within said computer mass storage subsystem wherein a portion of said data corresponding to said known valid data stored in said plurality of physical storage blocks in said at least one first storage stripe is maintained; and

reading said portion of said data from said at least one alternative non-volatile storage location.

53. The method of claim 50 wherein said step of accessing includes the step of:

determining at least one alternative volatile storage location within said computer mass storage subsystem wherein a portion of said data corresponding to said known Valid data stored in said plurality of physical storage blocks in said at least one first storage stripe is maintained; and

reading said portion of said data from said at least one alternative volatile storage location.

54. The method of claim 50 wherein said step of computing is carried out by the step of:

XORing blocks of data corresponding to said accessed data and data corresponding to said resubmitted valid data to provide said updated value for said associated redundancy information.

55. The method of claim 50 wherein said step of resubmitting is carried out by the step of:

reattempting a previous data modification operation which resulted in said potentially invalid data; and

completing said previous data modification operation to provide said valid data.

56. The method of claim 55 further comprising the step of:

storing data indicative of said previous data modification operation in non-volatile storage.

57. The method of claim 50 wherein said step of storing is carried out by the step of:

writing said valid resubmitted data and said updated value for said associated redundancy information to said data storage units at said second set of predetermined positions in said at least one second storage stripe.

58. The method of claim 50 wherein said step of computing is carried out by a computer mass storage device controller operatively coupling said array of data storage units to a central processor unit.

59. The method of claim 58 wherein said steps of accessing and storing are further carried out by said computer mass storage device controller and said array of data storage units.

60. The method of claim 50 wherein said steps of accessing, resubmitting, computing and storing are carried out substantially concurrently with other input/output tasks of said computer mass storage device subsystem.

61. The method of claim 50 further comprising the step of:

preventing access by other input/output processes to each storage block being accessed during said on-line restoration process.

62. The method of claim 50 wherein said first set of predetermined positions in said at least one first storage stripe across a subset of said data storage units is identical to said second set of predetermined positions in said at least one second storage stripe across a subset of said data storage units.