Multi option rebuilding in a dispersed storage network

• US 10,133,631 B2
• Filed: 09/13/2016
• Issued: 11/20/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Leggette, Wesley B., Resch, Jason K., Vossberg, Trevor J.
• Primary Examiner(s): Decady, Albert
• Assistant Examiner(s): Alshack, Osman M
• Application Number:
  US15/264,160
• Publication Number:
  US 20170123916A1
• Time in Patent Office: 798 Days
• Field of Search: 714752, 714763, 714764
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 12/0646 Configuration or reconfiguration
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2212/1008 Correctness of operation, e.g. memory ordering
• CPC Class Codes: G06F 2212/657 Virtual address space management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 41/0816 : the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 47/803 : Application aware
• CPC Class Codes: H04L 63/0428 : wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/0457 : wherein the sending and receiving network entities apply dynamic encryption, e.g. stream encryption cryptographic mechanisms or cryptographic arrangements for stream encryption H04L9/065
• CPC Class Codes: H04L 63/06 : for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/068 : using time-dependent keys, e.g. periodically changing keys cryptographic mechanisms or cryptographic arrangements for controlling usage of secret information H04L9/088
• CPC Class Codes: H04L 63/08 : for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/101 : Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/0861 : Generation of secret information including derivation or calculation of cryptographic keys or passwords
• CPC Class Codes: H04L 9/14 : using a plurality of keys or algorithms

Abstract

A method includes identifying an encoded slice for rebuilding. The method further includes determining whether the set of encoded slices is stored in an encrypted section of a vault or within an unencrypted section of the vault. The method further includes, when the set of encoded slices is stored in the unencrypted section of the vault, determining whether the set of storage units have viewing rights. The method further includes, when the set of storage units does not have the viewing rights, enabling a restricted rebuilding process to rebuild the encoded slice. The method further includes, when the set or storage units does have the viewing rights, enable an unrestricted rebuilding process to rebuild the encoded slice.

Claims

1. A method comprises:

identifying, by a rebuilding module that is implemented based on at least one processing circuit within a device of a dispersed storage network (DSN), an encoded slice for rebuilding, wherein a data element is dispersed storage error encoded into a set of encoded slices, which includes the encoded slice, wherein a set of storage units of the DSN stores the set of encoded slices, and wherein a first storage unit of the set of storage units is assigned to store the encoded slice;

determining, by the rebuilding module, whether the set of encoded slices is stored in an encrypted section of a vault or within an unencrypted section of the vault;

when the set of encoded slices is stored in the unencrypted section of the vault, determining, by the rebuilding module, whether the set of storage units have viewing rights;

when the set of storage units does not have the viewing rights indicating that the set of storage units is unauthorized to access any useful information corresponding to the set of encoded slices, enabling, by the rebuilding module, a restricted rebuilding process to rebuild the encoded slice; and

when the set of storage units does have the viewing rights indicating that the set of storage units is authorized to access the useful information corresponding to the set of encoded slices, enabling, by the rebuilding module, an unrestricted rebuilding process to rebuild the encoded slice.

2. The method of claim 1 further comprises:

when the set of encoded slices is stored in the encrypted section of the vault, enabling, by the rebuilding module, the unrestricted rebuilding process for rebuilding the encoded slice.

3. The method of claim 1, wherein the data element comprises:

an encryption key that is used to encrypt data stored in the encrypted section of the vault, wherein the encryption key is dispersed storage error encoded into a set of encoded key slices that is stored in the unencrypted section of the vault supported by the set of storage units.

4. The method of claim 1, wherein the data element comprises:

a data segment of a data object, wherein the data segment is dispersed storage error encoded into a set of encoded data slices, wherein the set of encoded data slices is encrypted using an encryption key to produce an encrypted set of encoded data slices that is stored in the encrypted section of the vault supported by the set of storage units.

5. The method of claim 1, wherein the data element comprises:

a data segment of a data object, wherein the data segment is dispersed storage error encoded into a set of encoded data slices that is stored in the unencrypted section of the vault supported by the set of storage units.

6. The method of claim 1, wherein the enabling the restricted rebuilding process comprises:

sending, by the rebuilding module, a partial rebuilding request to a subset of storage units of the storage units, wherein the subset of storage units does not include the first storage unit;

executing, by each storage unit in the subset of storage units, a partial rebuilding function for the encoded slice to produce partial slice rebuilding data, wherein the subset of storage units produces a set of partial slice rebuilding data; and

generating, by the rebuilding module, a rebuilt encoded slice based on the set of partial slice rebuilding data.

7. The method of claim 1, wherein the enabling the unrestricted rebuilding process comprises:

sending, by the rebuilding module, slice retrieval requests to a subset of storage units of the storage units, wherein the subset of storage units does not include the first storage unit;

in response to the slice retrieval requests, receiving, by the rebuilding module, a subset of encoded slices of the set of encoded slices from the subset of storage units;

reconstructing, by the rebuilding module, the data element from the subset of encoded slices;

dispersed storage error encoding, by the rebuilding module, the reconstructed data element to produce a new set of encoded slices; and

selecting one of the new set of encoded slices as rebuilt encoded slice for the encoded slice.

8. The method of claim 1 further comprises:

identifying, by the rebuilding module, a second encoded slice of the set of encoded slices for rebuilding;

when the set of storage units does not have the viewing rights;

enabling, by the rebuilding module, the restricted rebuilding process to rebuild the encoded slice; and

when the rebuilding of the encoded slice is complete, enabling, by the rebuilding module, the restricted rebuilding process to rebuild the second encoded slice.

9. The method of claim 1 further comprises:

identifying, by the rebuilding module, a second encoded slice of the set of encoded slices for rebuilding; and

when the set of storage units does have the viewing rights, enabling, by the rebuilding module, the unrestricted rebuilding process to rebuild the encoded slice and the second encoded slice substantially concurrently.

10. A rebuilding module comprises:

a first module that is implemented based on at least one processing circuit and that, when operable within a device of a dispersed storage network (DSN), causes the device to;

identify an encoded slice for rebuilding, wherein a data element is dispersed storage error encoded into a set of encoded slices, which includes the encoded slice, wherein a set of storage units of the DSN stores the set of encoded slices, and wherein a first storage unit of the set of storage units is assigned to store the encoded slice;

a second module that is implemented based on the at least one processing circuit and that, when operable within the device, causes the device to;

determine whether the set of encoded slices is stored in an encrypted section of a vault or within an unencrypted section of the vault; and

when the set of encoded slices is stored in the unencrypted section of the vault, determine whether the set of storage units have viewing rights; and

a third module that is implemented based on the at least one processing circuit and that, when operable within the device, causes the device to;

when the set of storage units does not have the viewing rights indicating that the set of storage units is unauthorized to access any useful information corresponding to the set of encoded slices, enable a restricted rebuilding process to rebuild the encoded slice; and

when the set of storage units does have the viewing rights indicating that the set of storage units is authorized to access the useful information corresponding to the set of encoded slices, enable an unrestricted rebuilding process to rebuild the encoded slice.

11. The rebuilding module of claim 10 further comprises:

a fourth module that is implemented based on the at least one processing circuit and that, when operable within the device, causes the device to;

when the set of encoded slices is stored in the encrypted section of the vault, enable the unrestricted rebuilding process for rebuilding the encoded slice.

12. The rebuilding module of claim 10, wherein the data element comprises:

an encryption key that is used to encrypt data stored in the encrypted section of the vault, wherein the encryption key is dispersed storage error encoded into a set of encoded key slices that is stored in the unencrypted section of the vault supported by the set of storage units.

13. The rebuilding module of claim 10, wherein the data element comprises:

a data segment of a data object, wherein the data segment is dispersed storage error encoded into a set of encoded data slices, wherein the set of encoded data slices is encrypted using an encryption key to produce an encrypted set of encoded data slices that is stored in the encrypted section of the vault supported by the set of storage units.

14. The rebuilding module of claim 10, wherein the data element comprises:

a data segment of a data object, wherein the data segment is dispersed storage error encoded into a set of encoded data slices that is stored in the unencrypted section of the vault supported by the set of storage units.

15. The rebuilding module of claim 10, wherein the third module, when operable within the device, further causes the device to enable the restricted rebuilding process by:

sending a partial rebuilding request to a subset of storage units of the storage units, wherein the subset of storage units does not include the first storage unit;

receiving, from each storage unit in the subset of storage units, a partial rebuilding function for the encoded slice to produce partial slice rebuilding data, wherein the subset of storage units produces a set of partial slice rebuilding data; and

generating, by the rebuilding module, a rebuilt encoded slice based on the set of partial slice rebuilding data.

16. The rebuilding module of claim 10, wherein the third module, when operable within the device, further causes the device to enable the unrestricted rebuilding process by:

send slice retrieval requests to a subset of storage units of the storage units, wherein the subset of storage units does not include the first storage unit;

in response to the slice retrieval requests, receive a subset of encoded slices of the set of encoded slices from the subset of storage units;

reconstruct the data element from the subset of encoded slices;

dispersed storage error encode the reconstructed data element to produce a new set of encoded slices; and

select one of the new set of encoded slices as rebuilt encoded slice for the encoded slice.

17. The rebuilding module of claim 10 further comprises:

the first module, when operable within the device, further causes the device to;

identify a second encoded slice of the set of encoded slices for rebuilding; and

the third module, when operable within the device, further causes the device to;

when the set of storage units does not have the viewing rights;

enable the restricted rebuilding process to rebuild the encoded slice; and

when the rebuilding of the encoded slice is complete, enable the restricted rebuilding process to rebuild the second encoded slice.

18. The rebuilding module of claim 10 further comprises:

the first module, when operable within the device, further causes the device to;

identify a second encoded slice of the set of encoded slices for rebuilding; and

the third module, when operable within the device, further causes the device to;

when the set of storage units does have the viewing rights, enable the unrestricted rebuilding process to rebuild the encoded slice and the second encoded slice substantially concurrently.