Rebuilding a data segment in a dispersed storage network

• US 9,152,514 B2
• Filed: 04/19/2013
• Issued: 10/06/2015
• Est. Priority Date: 11/24/2009
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Baptist, Andrew, Markison, Timothy W., Grube, Gary W.
• Primary Examiner(s): Gelagay, Shewaye
• Assistant Examiner(s): NGUYEN, TRONG H
• Application Number:
  US13/866,224
• Publication Number:
  US 20130232378A1
• Time in Patent Office: 900 Days
• Field of Search: 726/4, 726/17, 726/21, 726 27- 30, 713/165, 713/167, 713193-194, 714/2, 714/6.2, 714/6.22, 714/6.24, 714/6.32, 714/48, 714/54, 714/710, 714/746, 714/747, 714/755, 714/758, 714/760, 714/764, 714/799
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 21/80 in storage media based on magnetic or optical technology, e.g. disks with sectors preventing unauthorised reproduction or copying of disc-type recordable media G11B20/00

Abstract

A method begins with a processing module storing a set of encoded data slices in memory of a dispersed storage network (DSN), where a data segment is encoded using an error coding dispersal storage function to produce the set of encoded data slices. The method continues with the processing module adding the data segment to a rebuilding list, where encoded data slices of data segments identified in the rebuilding list are checked via a rebuilding process to detect errors and, when one of the encoded data slices has an error, the rebuilding process rebuilds the one of the encoded data slices. The method continues with the processing module, in response to a condition, removing the data segment from the rebuilding list.

Claims

1. A method for execution by a computing device within a dispersed storage network (DSN), the method comprises:

storing a set of encoded data slices in memory of the DSN, wherein a data segment is encoded using an error coding dispersal storage function to produce the set of encoded data slices;

adding the data segment to a rebuilding list, wherein encoded data slices of data segments identified in the rebuilding list are checked via a rebuilding process to detect errors and, when one of the encoded data slices has an error, the rebuilding process rebuilds the one of the encoded data slices;

in response to a condition, removing the data segment from the rebuilding list;

when the data segment has been removed from the rebuilding list;

identifying other data segments of a plurality of data segments, wherein the plurality of data segments includes the data segment and corresponds to at least one data object;

determining whether at least some of the other data segments have been removed from the rebuilding list;

when at least a predetermined number of other data segments has been removed from the rebuilding list, determining whether the at least the predetermined number of other data segments have at least a threshold number of encoded data slices available; and

when the at least the predetermined number of other data segments does not have at least the threshold number of encoded data slices available, deleting the at least one data object from the memory of the DSN.

2. The method of claim 1, wherein the condition comprises at least one of:

elapse of time;

a user command;

a system administrator directive; and

a passive expiration subscription.

3. The method of claim 1 further comprises, when the data segment has been removed from the rebuilding list:

determining when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, deleting the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment.

4. The method of claim 1 further comprises, when the data segment has been removed from the rebuilding list:

determining when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, sending a query regarding re-adding the data segment to the rebuilding list.

5. The method of claim 1 further comprises, when the data segment has been removed from the rebuilding list:

determining when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, confirming non-rebuilding of encoded data slices of the set of encoded data slices.

6. The method of claim 1 further comprises, when the data segment has been removed from the rebuilding list:

determining when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN;

detecting when less than the threshold number of error-free encoded data slices are available for the data segment; and

when the less than the threshold number of error-free encoded data slices are available for the data segment, deleting the less than the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment.

7. The method of claim 1 further comprises, when the data segment has been removed from the rebuilding list:

determining when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN;

detecting when less than the threshold number of encoded data slices are available for the data segment; and

when the less than the threshold number of encoded data slices are available for the data segment, replacing the data segment with null data.

8. A method for execution by a computing device within a dispersed storage network (DSN), the method comprises:

storing a set of encoded data slices in memory of the DSN, wherein a data segment is encoded using an error coding dispersal storage function to produce the set of encoded data slices;

determining a rebuilding option regarding the data segment is to not include the data segment on a rebuilding list, wherein encoded data slices of data segments identified in the rebuilding list are checked via a rebuilding process to detect errors and, when one of the encoded data slices has an error, the rebuilding process rebuilds the one of the encoded data slices;

determining disposition of the data segment when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN;

identifying other data segments of a plurality of data segments, wherein the plurality of data segments includes the data segment and corresponds to at least one data object;

determining whether at least some of the other data segments are not in the rebuilding list;

when at least a predetermined number of other data segments is not in the rebuilding list, determining whether the at least the predetermined number of other data segments has at least a threshold number of encoded data slices available; and

when the at least the predetermined number of other data segments does not have at least the threshold number of encoded data slices available, deleting the at least one data object from the memory of the DSN.

9. The method of claim 8, wherein the determining the disposition of the data segment comprises one of:

deleting the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment;

sending a query regarding adding the data segment to the rebuilding list;

confirming non-rebuilding of encoded data slices of the set of encoded data slices;

detecting when less than the threshold number of error-free encoded data slices are available for the data segment and, when the less than the threshold number of error-free encoded data slices are available for the data segment, deleting the less than the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment; and

detecting when less than the threshold number of error-free encoded data slices are available for the data segment, and, when the less than the threshold number of error-free encoded data slices are available for the data segment, replacing the data segment with null data.

10. A dispersed storage (DS) module of a computing device within a dispersed storage network (DSN), the DS module comprises:

a first module, when operable within the computing device, causes the computing device to;

store a set of encoded data slices in memory of the DSN, wherein a data segment is encoded using an error coding dispersal storage function to produce the set of encoded data slices;

a second module, when operable within the computing device, causes the computing device to;

add the data segment to a rebuilding list, wherein encoded data slices of data segments identified in the rebuilding list are checked via a rebuilding process to detect errors and, when one of the encoded data slices has an error, the rebuilding process rebuilds the one of the encoded data slices; and

a third module, when operable within the computing device, causes the computing device to;

in response to a condition, remove the data segment from the rebuilding list;

when the data segment has been removed from the rebuilding list;

identify other data segments of a plurality of data segments, wherein the plurality of data segments includes the data segment and corresponds to at least one data object;

determine whether at least some of the other data segments have been removed from the rebuilding list;

when at least a predetermined number of other data segments have been removed from the rebuilding list, determine whether the at least the predetermined number of other data segments has at least a threshold number of encoded data slices available; and

when the at least the predetermined number of other data segments does not have at least the threshold number of encoded data slices available, delete the at least one data object from the memory of the DSN.

11. The DS module of claim 10, wherein the condition comprises at least one of:

elapse of time;

a user command;

a system administrator directive; and

a passive expiration subscription.

12. The DS module of claim 10 further comprises, when the data segment has been removed from the rebuilding list:

the third module further functions to;

determine when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, delete the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment.

13. The DS module of claim 10 further comprises, when the data segment has been removed from the rebuilding list:

the third module further functions to;

determine when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, send a query regarding re-adding the data segment to the rebuilding list.

14. The DS module of claim 10 further comprises, when the data segment has been removed from the rebuilding list:

the third module further functions to;

determine when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN, confirm non-rebuilding of encoded data slices of the set of encoded data slices.

15. The DS module of claim 10 further comprises, when the data segment has been removed from the rebuilding list:

the third module further functions to;

determine when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN;

detect when less than the threshold number of error-free encoded data slices are available for the data segment; and

when the less than the threshold number of error-free encoded data slices are available for the data segment, delete the less than the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment.

16. The DS module of claim 10 further comprises, when the data segment has been removed from the rebuilding list:

the third module further functions to;

determine when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN; and

when the set of encoded data slices includes the threshold number of error-free encoded data slices remaining in the memory of the DSN;

detect when less than the threshold number of error-free encoded data slices are available for the data segment; and

when the less than the threshold number of error-free encoded data slices are available for the data segment, replace the data segment with null data.

17. A dispersed storage (DS) module of a computing device within a dispersed storage network (DSN), the DS module comprises:

a first module, when operable within the computing device, causes the computing device to;

store a set of encoded data slices in memory of the DSN, wherein a data segment is encoded using an error coding dispersal storage function to produce the set of encoded data slices;

a second module, when operable within the computing device, causes the computing device to;

determine a rebuilding option regarding the data segment is to not include the data segment on a rebuilding list, wherein encoded data slices of data segments identified in the rebuilding list are checked via a rebuilding process to detect errors and, when one of the encoded data slices has an error, the rebuilding process rebuilds the one of the encoded data slices; and

a third module, when operable within the computing device, causes the computing device to;

determine disposition of the data segment when the set of encoded data slices includes a threshold number of error-free encoded data slices remaining in the memory of the DSN, wherein

the second module further functions to;

identify other data segments of a plurality of data segments, wherein the plurality of data segments includes the data segment and corresponds to at least one data object; and

determine whether at least some of the other data segments are not in the rebuilding list; and

the third module further functions to;

when at least a predetermined number of other data segments is not in the rebuilding list, determine whether the at least the predetermined number of other data segments has at least a threshold number of encoded data slices available; and

when the at least the predetermined number of other data segments does not have at least the threshold number of encoded data slices available, delete the at least one data object from the memory of the DSN.

18. The DS module of claim 17, wherein the third module functions to determine the disposition of the data segment by one of:

deleting the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment;

sending a query regarding adding the data segment to the rebuilding list;

confirming non-rebuilding of encoded data slices of the set of encoded data slices;

detecting when less than the threshold number of error-free encoded data slices are available for the data segment and, when the less than the threshold number of error-free encoded data slices are available for the data segment, deleting the less than the threshold number of error-free encoded data slices and deleting addressing information regarding the data segment; and

detecting when less than the threshold number of encoded data slices are available for the data segment, and, when the less than the threshold number of encoded data slices are available for the data segment, replacing the data segment with null data.