Resolution of a storage error in a dispersed storage network

• US 9,164,841 B2
• Filed: 04/19/2013
• Issued: 10/20/2015
• Est. Priority Date: 06/05/2012
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K.
• Primary Examiner(s): KUDIRKA, JOSEPH R
• Application Number:
  US13/866,631
• Publication Number:
  US 20130326264A1
• Time in Patent Office: 914 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16

Abstract

A method begins by a dispersed storage (DS) processing module identifying an encoded data slice having an error, where a storage unit of a dispersed storage network (DSN) stores the encoded data slice. The method continues with the DS processing module sending a lock command to the storage unit. The method continues with the DS processing module determining resolution for the error of the encoded data slice, where the resolution includes one or more of: rebuilding the encoded data slice, issuing a set of delete requests to storage units of the DSN regarding a set of encoded data slices, issuing a set of undo write requests to the storage units of the DSN regarding the set of encoded data slices, and issuing a set of roll-back write requests to the storage units of the DSN regarding the set of encoded data slices.

Claims

1. A method for execution by a rebuilding module of a computing device within a dispersed storage network (DSN), the method comprising:

identifying an encoded data slice having an error, wherein the encoded data slice has a slice name and a revision number and wherein a storage unit of the DSN stores the encoded data slice;

sending a lock command to the storage unit, wherein the lock command instructs the storage unit to ignore access requests regarding the encoded data slice from other entities of the DSN and to ignore write-related requests and delete-related requests regarding other encoded data slices from the other entities of the DSN and wherein the other encoded data slices have the slice name but different revision numbers than the encoded data slice; and

in response to a lock acknowledgement from the storage unit, selecting a resolution from a set of resolutions for the error of the encoded data slice, wherein the set of resolutions includes;

rebuilding the encoded data slice, issuing a set of delete requests to storage units of the DSN regarding a set of encoded data slices, issuing a set of undo requests to the storage units of the DSN regarding the set of encoded data slices, and issuing a set of roll-back requests to the storage units of the DSN regarding the set of encoded data slices, wherein the set of encoded data slices includes the encoded data slice and the storage units of the DSN include the storage unit.

2. The method of claim 1 further comprising:

when the resolution for the error of the encoded data slice has been executed, sending an unlock command to the storage unit.

3. The method of claim 1, wherein the identifying the encoded data slice having the error comprises at least one of:

identifying the encoded data slice via a rebuilding detection process;

receiving a message indicating the error; and

receiving a response to a query regarding the encoded data slice.

4. The method of claim 1, wherein determining to rebuild the encoded data slice comprises:

determining whether at least a threshold number of encoded data slices of the set of encoded data slices is stored in the DSN, wherein a data segment is dispersed storage error encoded to produce the set of encoded data slices, and wherein the set of encoded data slices includes the encoded data slice; and

when the at least a threshold number of encoded data slices is stored in the DSN, rebuilding the encoded data slice using at least one of a partial rebuilding process and a rebuilding process.

5. The method of claim 1, wherein determining to issue the set of delete requests comprises:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is complete, issuing the set of delete requests to the storage units of the DSN to delete the set of encoded data slices, which includes the encoded data slice.

6. The method of claim 1, wherein determining to issue the set of undo requests comprises:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is at a commit phase, issuing the set of undo requests to the storage units of the DSN to undo storing the set of encoded data slices, which includes the encoded data slice.

7. The method of claim 1, wherein determining to issue the set of roll-back requests comprises:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is at an initial write phase, issuing the set of roll-back requests to the storage units of the DSN to retract the initial write phase regarding storing the set of encoded data slices, which includes the encoded data slice.

8. The method of claim 1, wherein the selecting the resolution further comprises:

requesting re-issuance of a write command regarding the encoded data slice.

9. The method of claim 1 further comprising:

sending a resolution request regarding the resolution to at least one of the other entities of the DSN for execution of the resolution.

10. A dispersed storage (DS) module of a computing device within a dispersed storage network (DSN), the DS module comprising:

a first module, when operable within the computing device, causing the computing device to;

identify an encoded data slice having an error, wherein the encoded data slice has a slice name and a revision number and wherein a storage unit of the DSN stores the encoded data slice;

a second module, when operable within the computing device, causing the computing device to;

send a lock command to the storage unit, wherein the lock command instructs the storage unit to ignore access requests regarding the encoded data slice from other entities of the DSN and to ignore write-related requests and delete-related requests regarding other encoded data slices from the other entities of the DSN and wherein the other encoded data slices have the slice name but different revision numbers than the encoded data slice; and

a third module, when operable within the computing device, causing the computing device to;

in response to a lock acknowledgement from the storage unit, select a resolution from a set of resolutions for the error of the encoded data slice, wherein the set of resolutions includes;

rebuilding the encoded data slice, issuing a set of delete requests to storage units of the DSN regarding a set of encoded data slices, issuing a set of undo write requests to the storage units of the DSN regarding the set of encoded data slices, and issuing a set of roll-back write requests to the storage units of the DSN regarding the set of encoded data slices, wherein the set of encoded data slices includes the encoded data slice and the storage units of the DSN include the storage unit.

11. The DS module of claim 10 further comprising:

when the resolution for the error of the encoded data slice has been executed, the third module further functions to send an unlock command to the storage unit.

12. The DS module of claim 10, wherein the first module functions to identify the encoded data slice having the error by at least one of:

identifying the encoded data slice via a rebuilding detection process;

receiving a message indicating the error; and

receiving a response to a query regarding the encoded data slice.

13. The DS module of claim 10, wherein the third module further functions to determine to rebuild the encoded data slice by:

determining whether at least a threshold number of encoded data slices of the set of encoded data slices is stored in the DSN, wherein a data segment is dispersed storage error encoded to produce the set of encoded data slices, and wherein the set of encoded data slices includes the encoded data slice; and

when the at least a threshold number of encoded data slices is stored in the DSN, rebuilding the encoded data slice using at least one of a partial rebuilding process and a rebuilding process.

14. The DS module of claim 10, wherein the third module further functions to determine to issue the set of delete requests by:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is complete, issuing the set of delete requests to the storage units of the DSN to delete the set of encoded data slices, which includes the encoded data slice.

15. The DS module of claim 10, wherein the third module further functions to determine to issue the set of undo requests by:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is at a commit phase, issuing the set of undo requests to the storage units of the DSN to undo storing the set of encoded data slices, which includes the encoded data slice.

16. The DS module of claim 10, wherein the third module further functions to determine to issue the set of roll-back requests by:

when a determination is made not to rebuild the encoded data slice, determining status of a write process; and

when the status of the write process is at an initial write phase, issuing the set of roll-back requests to the storage units of the DSN to retract the initial write phase regarding storing the set of encoded data slices, which includes the encoded data slice.

17. The DS module of claim 10, wherein the third module further functions to determine the resolution further by:

requesting re-issuance of a write command regarding the encoded data slice.

18. The DS module of claim 10 further comprising:

the third module further functions to send a resolution request regarding the resolution to at least one of the other entities of the DSN for execution of the resolution.