Method for on-line reorganization of the data on a RAID-4 or RAID-5 array in the absence of one disk and the on-line restoration of a replacement disk

• US 5,390,327 A
• Filed: 06/29/1993
• Issued: 02/14/1995
• Est. Priority Date: 06/29/1993
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Lubbers, Clark E., Thiel, David W.
• Primary Examiner(s): ATKINSON, CHARLES
• Application Number:
  US08/085,021
• Time in Patent Office: 595 Days
• Field of Search: 371/10.1, 371/40.1, 395/575
• US Class Current: 714/6.32
• CPC Class Codes: G06F 11/1084 Degraded mode, e.g. caused by single or multiple storage removals or disk failures
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1004 Adaptive RAID, i.e. RAID system adapts to changing circumstances, e.g. RAID1 becomes RAID5 as disks fill up
• CPC Class Codes: G11B 20/1833 by adding special lists or symbols to the coded information G11B20/1806, G11B20/1866 take precedence

Abstract

In a storage system having a plurality of disks arranged in a RAID-4 or RAID-5 array, a method of improving the performance and reliability of the array in the absence of a member. The method re-organizes the array into the equivalent of a higher performance and reliability RAID-0 organization while allowing concurrent high performance application access to the array and includes a deliberate reorganization activity concurrent with application access. The method also restores the RAID-4 or RAID-5 organization subsequent to the failure of a member using a replacement member while continuing to allow concurrent high performance application access to the array. In order to perform this reorganization on-line state information is maintained for each parity block, each data block and the array itself. A recently removed disk may be reinserted using an expedited replacement process.

Claims

1. A method of on-line reorganization of data in a storage system having n+1 disks arranged in a RAID array when one of said disks falls while preserving concurrent access to said array by a user application, each of said disks in said storage system having a plurality of data blocks, a plurality of parity blocks, each parity block associated with n data blocks, a plurality of groups of parity block state bits, each group of parity block state bits indicating that said associated parity block contains parity information or data or the contents are undefined, and a group of array state bits indicating the condition of the array as "normal," "folding," "fully folded" or "unfolding," said method comprising the steps of:

(a) receiving a read request from said user application to read a targeted data block;

(b) interrogating said array state bits to determine the condition of said array;

(c) if said array state bits indicate said array is in said "normal" state, then proceeding with a read operation;

(d) if said array state bits indicate said array is in said "folding" state, then proceeding with a "folding" read operation;

(e) if said array state bits indicate said array is in said "fully folded" state, then proceeding with a "fully folded" read operation; and

(f) if said array state bits indicate said array is in said "folding" state, then also proceeding with a deliberate process to complete said reorganization concurrently with user application access to said array.

2. The method of on-line reorganization of data in a storage system as set forth in claim 1 wherein said "folding" read operation comprises the following steps:

(g) determining if said targeted data block of said read request is on said failed disk;

(h) if said targeted data block is not on said failed disk, then proceeding with a read operation;

(i) if said targeted data block is on said failed disk, then interrogating the state bits of the parity block associated with said targeted data block;

(j) if said parity block state bits indicate said parity block is in said data state, then reading data from said parity block and send said read data to said user application;

(k) if said parity block state bits indicate said parity block is in said undefined state, then sending a read failure signal to said user application; and

(l) if said parity block state bits indicate said party block is in said parity state, then regenerating said targeted data block, sending said regenerated data to said user application, setting said parity block state bits to said undefined state, writing said regenerated data to said parity block and setting said parity block state bits to said data state.

3. The method of on-line reorganization of data in a storage system as set forth in claim 2 wherein said deliberate process comprises the following steps:

(m) locating a parity block with state bits indicating said parity state;

(n) regenerating said data block on said failed disk associated with said located parity block;

(o) setting said state bits of said located parity block to said undefined state;

(p) writing said regenerated data block to said located parity block;

(q) setting said state bits of said located parity block to said data state;

(r) repeating steps (m) through (q) until all parity blocks with said state bits indicating said parity state are processed; and

then

(s) setting said array state bits to indicate said array is in said "fully folded" state.

4. A method of on-line reorganization of data in a storage system having n+1 disks arranged in a RAID array when one of said disks fails while preserving concurrent access to said array by a user application, each of said disks in said storage system having a plurality of data blocks, a plurality of data block state bits, each data block state bit associated with one of said data blocks, and having a first state indicating that said associated data block may contain invalid information and a second state indicating that said associated data block contains valid information, a plurality of parity blocks, each parity block associated with n data blocks, a plurality of groups of parity block state bits, each group of parity block state bits indicating that said associated parity block contains parity information or data or the contents are undefined, and a group of array state bits indicating the condition of the array as "normal," "folding," "fully folded" or "unfolding," said method comprising the steps of:

(a) receiving a read request from said user application to read a targeted data block;

(b) inserting a replacement disk;

(c) interrogating said array state bits to determine the state of said array;

(d) if said array state bits indicate said array is in said "unfolding" state, then proceeding with an "unfolding" read operation and proceeding with a deliberate process to complete said reorganization concurrently with user application access to said array.

5. The method of on-line reorganization of data in a storage system as set forth in claim 4, wherein said "unfolding" read operation comprises the following steps:

(e) determining if said targeted data block of said read request is on said failed disk;

(f) if said targeted data block is not on said failed disk, then proceeding with a read operation;

(g) if said targeted data block is on said failed disk, then interrogating said parity block state bits for the associated parity block;

(h) if said parity block state bits indicate said parity block is in said data state, then,

(1) reading data from said parity block and sending said read data to said user application;

(2) writing said read data to said targeted data block on said replacement disk;

(3) setting said data block state bit for said target data block to said valid state;

(4) calculating parity information associated with said targeted data block;

(5) setting said parity block state bits of said associated parity block to said undefined state;

(6) writing said calculated parity information to said associated parity block; and

(7) setting said parity block state bits of said associated parity block to said parity state;

(i) if said parity block state bits indicate said parity block is in said undefined state, then,

(1) interrogating said targeted data block on said replacement disk;

(2) if said state bits of said targeted data block indicate said targeted data block is in said invalid state, then sending a read failure signal to said user application;

(3) if said state bits of said targeted data block indicate said targeted data block is in said valid state, then reading said targeted data block from said replacement disk and sending said read data to said user application;

(j) if said parity block state bits indicate said parity block is in said parity state, then,

(1) interrogating said state bits of said targeted data block on said replacement disk;

(2) if said state bit of said targeted data block indicates said target data block is in said valid state, then reading said targeted data block from said replacement disk and sending said read data to said user application;

(3) if said state bit of said targeted data block indicates said target data block is in said invalid state, then regenerating said data block, sending said regenerated data block to said user application, writing said regenerated data block to said replacement disk and setting said state bit of said targeted data block to said valid state.

6. The method of on-line reorganization of data in a storage system as set forth in claim 5, wherein said deliberate process comprises the following steps:

(k) setting said state bits for every one of said data blocks on said replacement disk to said invalid state;

(l) setting said state bits for every one of said parity blocks on said replacement disk to said undefined state;

(m) setting said array state bits to said "unfolding" state;

(n) interrogating the parity block state bits of one of said parity blocks;

(o) if said parity block state bits indicate said parity block is in said parity state and said corresponding n-1 data block state bits indicate said n-1 data blocks are in said valid state, then regenerating said targeted data block, writing said regenerated data block to said replacement disk and setting said data block state bits for said data block to said valid state;

(p) if said parity block state bits indicate said parity block is in said data state and the corresponding data block state bit indicates said data block is in said invalid state, then copying the contents of said parity block to the corresponding data block on said replacement disk, and setting said data block state bit to said valid state; and

(1) if said state bits for all corresponding data blocks are in the valid state, then calculating the associated parity, writing said calculated parity to said parity block, setting said parity block state bits to said parity state;

(2) if said state bits for all corresponding data blocks are not in said valid state, then setting said parity block state bits to said undefined state;

(q) if said parity block state bits indicate said parity block is in said undefined state and said corresponding data block state bits are in said valid state, then calculating the associate parity, writing said calculated parity to said parity block, and setting said parity block state bits to said parity state; and

(r) determining if all parity blocks have been processed, if not, repeating steps (n) through (q) and, if so, setting said array state bits to said "normal" state.

7. A method of on-line reorganization of data in a storage system having n+1 disks arranged in a RAID array when one of said disks fails while preserving concurrent access to said array by a user application, each of said disks in said storage system having a plurality of data blocks, a plurality of parity blocks, each parity block associated with n data blocks, a plurality of groups of parity block state bits, each group of parity block state bits indicating that said associated parity block contains parity information or data or the contents are undefined, and a group of array state bits indicating the condition of the array as "normal," "folding," "fully folded" or "unfolding," said method comprising the steps of:

(a) receiving a write request from said user application to write to a targeted data block;

(b) interrogating said array state bits to determine the condition of said array;

(c) if said array state bits indicate said array is in said "normal" state, then proceeding with a write operation;

(d) if said array state bits indicate said array is in said "folding" state, then proceeding with a "folding" write operation;

(e) if said array state bits indicate said array is in said "fully folded" state, then proceeding with a "fully folded" write operation; and

(f) if said array state bits indicate said array is in said "folding" state, then also proceeding with a deliberate process to complete said reorganization concurrently with user application access to said array.

8. The method of on-line reorganization of data in a storage system as set forth in claim 7 wherein said "folding" write operation comprises the following steps:

(g) determining if said failed disk contains the parity data block associated with said targeted data block;

(h) if said failed disk contained said associated parity block, then proceeding with a write operation;

(i) if said failed disk did not contain said associated parity block, then interrogating said state bits of said associated parity block;

(j) if said parity block state bits indicate said parity block is in said data state or said undefined state, then,

(1) determining if said target data block is on said failed disk;

(2) if said target data block is not on said failed disk, then proceeding with a NRW write operation;

(3) if said target data block is on said failed disk, then writing to said parity block and setting said parity block state bits to said data state;

(k) if said parity block state bits indicate said parity block is in said parity state, then,

(1) determining if said target data block is on said failed disk;

(2) if said target data block is not on said failed disk, then proceeding with a RMW write operation;

(3) if said target data block is on said failed disk, then setting said parity block state bits to said undefined state, writing to said parity block, and setting said parity block state bits to said data state.

9. The method of on-line reorganization of data in a storage system as set forth in claim 7 wherein said deliberate process comprises the following steps:

(l) locating a parity block with state bits indicating said parity state;

(m) regenerating said data block on said failed disk associated with said located parity block;

(n) setting said state bits of said located parity block to said undefined state;

(o) writing said regenerated data block to said located parity block;

(p) setting said state bits of said located parity block to said data state;

(q) repeating steps (l) through (p) until all parity blocks with said state bits indicating said parity state are processed; and

then

(r) setting said array state bits to indicate said array is in said "fully folded" state.

10. A method of on-line reorganization of data in a storage system having n+1 disks arranged in a RAID array when one of said disks fails while preserving concurrent access to said array by a user application, each of said disks in said storage system having a plurality of data blocks, a plurality of data block state bits, each data block state bit associated with one of said data blocks, each data block state bit having a first state indicating that said associated data block may contain invalid information and a second state indicating that said associated data block contains valid information, a plurality of parity blocks, each parity block associated with n data blocks, a plurality of groups of parity block state bits, each group of parity block state bits indicating that said associated parity block contains parity information or data or the contents are undefined, and a group of array state bits indicating the condition of the array as "normal," "folding," "fully folded" or "unfolding," said method comprising the steps of:

(a) receiving a write request from said user application to write to a targeted data block;

(b) inserting a replacement disk;

(c) interrogating said array state bits to determine the state of said array;

(d) if said array state bits indicate said array is in said "unfolding" state, then proceeding with an "unfolding" write operation and proceeding with a deliberate process to complete said reorganization concurrently with user application access to said array.

11. The method of on-line reorganization of data in a storage system as set forth in claim 10, wherein said "unfolding" write operation comprises the following steps:

(e) determining if said targeted data block of said write request is on said failed disk;

(f) if said targeted data block is on said failed disk, then,

(1) interrogating said state bits of said parity block associated with said targeted data block;

(2) if said state bits of said associated parity block are in said data state or said undefined state, then proceed with a MRW write operation to said data block;

(3) if said state bits of said associated parity block are in said parity state, then interrogating said data block state bit and, if invalid, performing a MRW write operation to said data block and, if valid, performing a MRMW write operation to said data block;

(g) if said targeted data block is not on said failed disk, then determining if said replacement disk normally contains parity;

(h) if said replacement disk does not normally contain parity, then,

(1) interrogating said state bits of said parity block associated with said targeted data block;

(2) if said state bits of said associated parity block are in said data state or said undefined state, then proceeding with a NRW write operation to said data block;

(3) if said state bits of said parity block are in said parity state, then proceeding with a MRMW write operation to said data block;

(i) if said replacement disk does normally contain parity, then,

(1) interrogating said state bits of said parity block associated with said target data block;

(2) if said state bits are in said parity state, then interrogating said state bits of said data block and, if valid, proceeding with a MRMW write operation to said data block and, if invalid, proceeding with a MRW write operation to said data block;

(3) if said state bits are in said undefined state, then proceeding with a MRW write operation to said data block and, if unsuccessful, then proceeding with a MRMW write operation to said data block.

12. The method of on-line reorganization of data in a storage system as set forth in claim 11, wherein said deliberate process comprises the following steps:

(j) setting said state bits for every one of said data blocks on said replacement disk to said invalid state;

(k) setting said state bits for every one of said parity blocks on said replacement disk to said undefined state;

(l) setting said array state bits to said "unfolding" state;

(m) interrogating the parity block state bits of one of said parity blocks;

(n) if said parity block state bits indicate said parity block is in said parity state and said corresponding n-1 data block state bits indicate said n-1 data blocks are in said valid state, then regenerating said targeted data block, writing said regenerated data block to said replacement disk and setting said data block state bits for said data block to said valid state;

(o) if said parity block state bits indicate said parity block is in said data state and the corresponding data block state bit indicates said data block is in said invalid state, then copying the contents of said parity block to the corresponding data block on said replacement disk, and setting said data block state bit to said valid state; and

(1) if said state bits for all corresponding data blocks are in the valid state, then calculating the associated parity, writing said calculated parity to said parity block, setting said parity block state bits to said parity state;

(2) if said state bits for all corresponding data blocks are not in said valid state, then setting said parity block state bits to said undefined state;

(p) if said parity block state bits indicate said parity block is in said undefined state and said corresponding data block state bits are in said valid state, then calculating the associate parity, writing said calculated parity to said parity block, setting said parity block state bits to said parity state; and

(q) determining if all parity blocks have been processed, if not, repeating steps (m) through (p) and, if so, set said array state bits to said "normal" state.

13. A method for the replacement of a recently failed disk in a storage system having n+1 disks arranged in a RAID array, each of said disks in said storage system having a plurality of data blocks, a plurality of data block state bits, each data block state bit associated with one of said data blocks, and having a first state indicating that said associated data block may contain invalid information and a second state indicating that said associated data block contains valid information, a plurality of parity blocks, each parity block associated with n data block, a plurality of groups of parity block state bits, each group of parity block state bits indicating that said associated parity block contains parity information or data or the contents are undefined, and a group of array state bits indicating the condition of the array as "normal," "folding," "fully folded" or "unfolding," said method comprising the steps of:

(a) detecting a failed disk;

(b) removing said failed disk from the array;

(c) setting all parity block status bits to said undefined state on said failed disk;

(d) setting to said invalid state all status bits of data blocks having associated parity block status bits in said data or undefined state on said failed disk;

(e) inserting said failed disk into said array; and

(f) setting said array state bits from folding to unfolding state.