System for rebuilding dispersed data

• US 7,818,518 B2
• Filed: 04/28/2009
• Issued: 10/19/2010
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gopala Krishna Kapila Lakshmana Harsha, Dhavi, Mark, Zachary J., Gladwin, S. Christopher, England, Matthew M., Thornton, Vance T.
• Primary Examiner(s): LANE, JOHN A
• Application Number:
  US12/431,166
• Publication Number:
  US 20090254720A1
• Time in Patent Office: 539 Days
• Field of Search: 711/154
• US Class Current: 711/154
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems

Abstract

A digital data file storage system is disclosed in which original data files to be stored are dispersed using some form of information dispersal algorithm into a number of file “slices” or subsets in such a manner that the data in each file share is less usable or less recognizable or completely unusable or completely unrecognizable by itself except when combined with some or all of the other file shares. These file shares are stored on separate digital data storage devices as a way of increasing privacy and security. As dispersed file shares are being transferred to or stored on a grid of distributed storage locations, various grid resources may become non-operational or may operate below at a less than optimal level. When dispersed file shares are being written to a dispersed storage grid which not available, the grid clients designates the dispersed data shares that could not be written at that time on a Rebuild List. In addition when grid resources already storing dispersed data become non-available, a process within the dispersed storage grid designates the dispersed data shares that need to be recreated on the Rebuild List. At other points in time a separate process reads the set of Rebuild Lists used to create the corresponding dispersed data and stores that data on available grid resources.

Claims

1. A method operating on a distributed computer system, the method comprising the steps of:

creating a plurality of data slices from a data file;

determining a plurality of storage nodes for storing the plurality of data slices;

determining that less than a threshold number of the plurality of storage nodes are unavailable to produce at least one unavailable storage node and a set of available storage nodes of the plurality of storage nodes;

outputting a set of the plurality of data slices to the set of available storage nodes for storage therein; and

adding at least one of the plurality of data slices to a rebuild list, wherein the at lest one of the plurality of data slices is targeted for storage in the at least one unavailable storage node.

2. The method of claim 1, wherein the number of data slices created from said data file and the threshold number corresponds to the desired level of reliability and security.

3. The method of claim 1, wherein the step of outputting said data slices includes storing metadata associated with said data file.

4. The method of claim 1 further comprises:

determining that the at least one unavailable storage node has become available to produce at least one currently available storage node;

when the at least one unavailable storage node has become available, rebuilding the at least one data slice identified in the rebuild list to produce a rebuilt data slice; and

outputting the rebuilt data slice for storing in the at least one currently available storage node.

5. The method of claim 1, further comprises mapping the storage location of each of the plurality of data slices to corresponding ones of the plurality of storage nodes.

6. A computer comprises:

an interface; and

a central processing unit operable to;

create a plurality of data slices from a data file;

determine a plurality of storage nodes for storing the plurality of data slices;

determine that less than a threshold number of the plurality of storage nodes are unavailable to produce at least one unavailable storage node and a set of available storage nodes of the plurality of storage nodes;

output, via the interface, a set of the plurality of data slices to the set of available storage nodes for storage therein; and

facilitate adding at least one of the plurality of data slices to a rebuild list, wherein the at lest one of the plurality of data slices is targeted for storage in the at least one unavailable storage node.

7. The computer of claim 6, wherein the number of data slices created from said data file and the threshold number corresponds to the desired level of reliability and security.

8. The computer of claim 6, wherein the central processing unit is further operable to output, via the interface, metadata of the data file with the plurality of data slices.

9. The computer of claim 6, wherein the central processing unit is further operable to:

determine that the at least one unavailable storage node has become available to produce at least one currently available storage node;

when the at least one unavailable storage node has become available, initiate rebuilding the at least one data slice identified in the rebuild list to produce a rebuilt data slice; and

facilitate outputting the rebuilt data slice for storing in the at least one currently available storage node.

10. The computer of claim 6, wherein the central processing unit is further operable to map storage location of each of the plurality of data slices to corresponding ones of the plurality of storage nodes.