Method and apparatus for rebuilding data in a dispersed data storage network

• US 8,464,096 B2
• Filed: 07/25/2012
• Issued: 06/11/2013
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Thornton, Vance T., Bellanca, Jamie, Hendrickson, Dustin M., Mark, Zachary J., Volvovski, Ilya
• Primary Examiner(s): LOHN, JOSHUA A
• Application Number:
  US13/558,008
• Publication Number:
  US 20120290873A1
• Time in Patent Office: 321 Days
• Field of Search: 714/6.22, 714/6.24
• US Class Current: 714/6.24
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2211/104 Metadata, i.e. metadata associated with RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A method begins by identifying a data slice requiring rebuilding to produce an identified data slice, wherein the identified data slice is one of a plurality of data slices that constitute a data segment. The method continues by retrieving at least m number of data slices, wherein m data slices of the plurality of data slices enable reconstruction of the data segment, and wherein the at least m number of data slices does not include the identified data slice. The method continues by reconstructing the identified data slice from the at least m number of data slices to produce a rebuilt data slice. The method continues by writing the rebuilt data slice to one of a plurality of data slice servers.

Claims

1. A method for execution by one or more computers associated with a dispersed data storage network, the method comprises:

identifying a data slice requiring rebuilding to produce an identified data slice, wherein the identified data slice is one of a set of data slices that constitute a data segment, wherein data words of the data segment are arranged into a data matrix that is encoding using an encoding matrix function in accordance an information dispersal algorithm to produce a coded matrix of coded values that is arranged into the set of data slices;

retrieving at least m number of data slices for the set of data slices, wherein n represents the number of data slices in the set of data slices and m is less than or equal to n-2;

decoding the retrieved at least m number of data slices by arranging coded values of the retrieved at least m number of data slices into a reconstructed coded matrix and decoding the reconstructed coded matrix using a decoding matrix function in accordance with the information dispersal algorithm to reconstruct the data segment;

encoding the reconstructed data segment by arranging data words of the reconstructed data segment into the data matrix and encoding the data matrix using the encoding matrix function in accordance with the information dispersal algorithm to produce the coded matrix that is arranged into a new set of data slices;

selecting a data slice of the new set of data slices as a rebuilt data slices to replace the identified data slice; and

outputting the rebuilt data slice for storage in one of a plurality of data slice servers.

2. The method of claim 1, wherein the retrieving further comprises:

transmitting a rebuild message to the plurality of data slice servers, wherein the rebuild message identifies at least one of;

a corresponding data slice of the set of data slices; and

the data segment;

receiving the at least m number of data slices from at least some of the plurality of data slice servers.

3. The method of claim 1, wherein the outputting the rebuilt data slice further comprises:

transmitting a write command to the one of the plurality of data slice servers, wherein the write command instructs the one of the plurality of data slice servers to overwrite the identified data slices with the rebuilt data slice.

4. The method of claim 1 further comprises:

updating a transaction identification value of the rebuilt data slice.

5. The method of claim 1 further comprises:

determining a cause for the identified data slice requiring rebuilding;

when the cause for the identified data slice requiring rebuilding is one of corruption or out-of-date, outputting the rebuilt data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers stored the identified data slice; and

when the cause for the identified data slice requiring rebuilding is a missing data slice;

outputting the rebuild data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers is a new data slice server of the plurality of data slice servers than the one of the plurality of data slice servers stored the identified data slice; and

updating a list of data slice servers to include the new data slice server and to remove the data slice server corresponding to the identified data slice.

6. A computer for use within a dispersed data storage network, the computer comprises:

a network interface for interfacing with a network, wherein a plurality of data slice servers is operably coupled to the network; and

a central processing unit, wherein the central processing unit is operably coupled to;

identify a data slice requiring rebuilding to produce an identified data slice, wherein the identified data slice is one of a set of data slices that constitute a data segment and wherein data words of the data segment are arranged into a data matrix that is encoded by an encoding matrix function in accordance an information dispersal algorithm to produce a coded matrix of coded values that is arranged into the set of data slices;

retrieve at least m number of data slices from the set of data slices, wherein n represents the number of data slices in the set of data slices and m is less than or equal to n-2;

decode the retrieved at least m number of data slices by arranging coded values of the retrieved at least m number of data slices into a reconstructed coded matrix and decoding the reconstructed coded matrix by a decoding matrix function in accordance with the information dispersal algorithm to reconstruct the data segment;

encode the reconstructed data segment by arranging the data words of the reconstructed data segment into the data matrix and encoding the data matrix by the encoding matrix function in accordance with the information dispersal algorithm to produce the coded matrix that is arranged into a new set of data slices;

select one of the new set of data slices as a rebuilt data slices to replace the identified data slice; and

output the rebuilt data slice to one of a plurality of data slice servers.

7. The computer of claim 6, wherein the central processing unit further functions to retrieve the at least m number of data slices by:

transmitting, via the network interface, a rebuild message to at least some of the plurality of data slice servers, wherein the rebuild message identifies at least one of;

a corresponding data slice of the plurality of data slices; and

the data segment;

receiving, via the network interface, the at least m number of data slices from the some of the plurality of data slice servers.

8. The computer of claim 6, wherein the central processing unit further functions to the output the rebuilt data slice by at least one of:

transmitting, via the network interface, a write command to one of the plurality of data slice servers, wherein the write command instructs the one of the plurality of data slice servers to overwrite the identified data slices with the rebuilt data slice.

9. The computer of claim 6, wherein the central processing unit further functions to:

update a transaction identification value of the rebuilt data slice.

10. The computer of claim 6, wherein the central processing unit further functions to:

determine a cause for the identified data slice requiring rebuilding; and

when the cause for the identified data slice requiring rebuilding is one of corruption or out-of-date, output the rebuilt data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers stored the identified data slice; and

when the cause for the identified data slice requiring rebuilding is a missing data slice;

output the rebuild data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers is a new data slice server of the plurality of data slice servers than the one of the plurality of data slice servers stored the identified data slice; and

updating a list of data slice servers to include the new data slice server and to remove the data slice server corresponding to the identified data slice.

11. A computer readable storage memory that stores operational instructions, which, when executed by a computing device, causes the computing device to:

identify a data slice requiring rebuilding to produce an identified data slice, wherein the identified data slice is one of a set of data slices that constitute a data segment, wherein data words of the data segment are arranged into a data matrix that is encoding using an encoding matrix function in accordance an information dispersal algorithm to produce a coded matrix of coded values that is arranged into the set of data slices;

retrieve at least m number of data slices for the set of data slices, wherein n represents the number of data slices in the set of data slices and m is less than or equal to n-2;

decode the retrieved at least m number of data slices by arranging coded values of the retrieved at least m number of data slices into a reconstructed coded matrix and decoding the reconstructed coded matrix using a decoding matrix function in accordance with the information dispersal algorithm to reconstruct the data segment;

encode the reconstructed data segment by arranging data words of the reconstructed data segment into the data matrix and encoding the data matrix using the encoding matrix function in accordance with the information dispersal algorithm to produce the coded matrix that is arranged into a new set of data slices;

select a data slice of the new set of data slices as a rebuilt data slices to replace the identified data slice; and

output the rebuilt data slice for storage in one of a plurality of data slice servers.

12. The computer readable storage memory of claim 11, wherein the retrieving further comprises:

transmitting a rebuild message to the plurality of data slice servers, wherein the rebuild message identifies at least one of;

a corresponding data slice of the set of data slices; and

the data segment;

receiving the at least m number of data slices from at least some of the plurality of data slice servers.

13. The computer readable storage memory of claim 11, wherein the outputting the rebuilt data slice further comprises:

transmitting a write command to the one of the plurality of data slice servers, wherein the write command instructs the one of the plurality of data slice servers to overwrite the identified data slices with the rebuilt data slice.

14. The computer readable storage memory of claim 11 further stores operational instructions that cause the computing device to:

update a transaction identification value of the rebuilt data slice.

15. The computer readable storage memory of claim 11 further stores operational instructions that cause the computing device to:

determine a cause for the identified data slice requiring rebuilding;

when the cause for the identified data slice requiring rebuilding is one of corruption or out-of-date, output the rebuilt data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers stored the identified data slice; and

when the cause for the identified data slice requiring rebuilding is a missing data slice;

output the rebuild data slice to the one of the plurality of data slice servers, wherein the one of the plurality of data slice servers is a new data slice server of the plurality of data slice servers than the one of the plurality of data slice servers stored the identified data slice; and

update a list of data slice servers to include the new data slice server and to remove the data slice server corresponding to the identified data slice.