Adaptive dispersed storage network (DSN) and system

• US 9,823,845 B2
• Filed: 06/02/2017
• Issued: 11/21/2017
• Est. Priority Date: 11/25/2009
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Abhijeet, Kumar, Dhuse, Greg R., Gladwin, S. Christopher, Grube, Gary W., Markison, Timothy W., Resch, Jason K.
• Primary Examiner(s): Eskandarnia, Arvin
• Application Number:
  US15/612,750
• Publication Number:
  US 20170269842A1
• Time in Patent Office: 172 Days
• Field of Search: 713193
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/08 : for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/101 : Access control lists [ACL]
• CPC Class Codes: H04L 63/102 : Entity profiles
• CPC Class Codes: H04L 69/04 : Protocols for data compression, e.g. ROHC

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and to the memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. When additional unique pillar combinations of at least read threshold number of encoded data slices (EDSs) supported by EDSs may be needed, the computing device dispersed error encodes each data segment in accordance to generate additional pluralities of EDSs and distributedly stores the additional pluralities of EDSs associated respectively with the data object across the plurality of SUs within the DSN to support the additional unique pillar combinations of the at least read threshold number of EDSs.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

transmit, via the DSN, a read request for a data object to another computing device, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs) that is stored within the DSN, wherein unique pillar combinations of at least a decode threshold number of EDSs of the set of EDSs provide for unique recovery of the data segment; and

when no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receive another unique pillar combination of at least the decode threshold number of EDSs of another set of EDSs from the another computing device via the DSN, wherein the another set of EDS are based on dispersed error encoding of the data segment in accordance with other dispersed error encoding parameters to generate the another set of EDS.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

when a unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receive the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the another computing device via the DSN.

3. The computing device of claim 2, wherein:

the set of EDSs are distributedly stored among a plurality of storage units (SUs) within the DSN; and

the another computing device is configured to;

assign the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs to the computing device;

retrieve the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the plurality of SUs; and

transmit the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs that is assigned to the computing device to the computing device via the DSN.

4. The computing device of claim 1, wherein, when the no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, the another computing device is configured to:

dispersed error encode the data segment in accordance with the other dispersed error encoding parameters to generate the another set of EDSs, wherein other unique pillar combinations of at least the decode threshold number of EDSs of the another set of EDSs provide for recovery of the data segment; and

distributedly store the another set of EDSs among a plurality of storage units (SUs) within the DSN.

5. The computing device of claim 1, wherein:

the data segment of the plurality of data segments is dispersed error encoded in accordance with the dispersed error encoding parameters to produce the set of EDSs that is of a first pillar width; and

the data segment of the plurality of data segments is dispersed error encoded in accordance with the other dispersed error encoding parameters to produce the another set of EDSs that is of a second pillar width that is greater than the first pillar width.

6. The computing device of claim 1, wherein:

at least one of the set of EDSs or the another set of EDSs is distributedly stored among a plurality of storage units (SUs) within the DSN;

a read threshold number of EDSs of the at least one of the set of EDSs or the another set of EDSs provides for reconstruction of the data segment; and

a write threshold number of EDSs of the at least one of the set of EDSs or the another set of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN.

7. The computing device of claim 1 further comprising:

a SU of a plurality of storage units (SUs) within the DSN that distributedly store at least one of the set of EDSs or the another set of EDSs, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

transmit, via the DSN, a read request for a data object to another computing device, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs) that is stored within the DSN, wherein unique pillar combinations of at least a decode threshold number of EDSs of the set of EDSs provide for unique recovery of the data segment;

when a unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receive the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the another computing device via the DSN; and

when no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receive another unique pillar combination of at least the decode threshold number of EDSs of another set of EDSs from the another computing device via the DSN, wherein the another set of EDS is generated by the another computing device that is configured to dispersed error encode the data segment in accordance with other dispersed error encoding parameters to generate the another set of EDSs, wherein other unique pillar combinations of at least the decode threshold number of EDSs of the another set of EDSs provide for recovery of the data segment.

10. The computing device of claim 9, wherein:

the set of EDSs are distributedly stored among a plurality of storage units (SUs) within the DSN; and

the another computing device is configured to;

assign the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs to the computing device;

retrieve the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the plurality of SUs; and

transmit the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs that is assigned to the computing device to the computing device via the DSN.

11. The computing device of claim 9, wherein, when the no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, the another computing device is configured to:

dispersed error encode the data segment in accordance with the other dispersed error encoding parameters to generate the another set of EDSs, wherein other unique pillar combinations of at least the decode threshold number of EDSs of the another set of EDSs provide for recovery of the data segment; and

distributedly store the another set of EDSs among a plurality of storage units (SUs) within the DSN, wherein;

the data segment of the plurality of data segments is dispersed error encoded in accordance with the dispersed error encoding parameters to produce the set of EDSs that is of a first pillar width; and

the data segment of the plurality of data segments is dispersed error encoded in accordance with the other dispersed error encoding parameters to produce the another set of EDSs that is of a second pillar width that is greater than the first pillar width.

12. The computing device of claim 9 further comprising:

a SU of a plurality of storage units (SUs) within the DSN that distributedly store at least one of the set of EDSs or the another set of EDSs, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

transmitting, via an interface of the computing device configured to interface and communicate with a dispersed or distributed storage network (DSN) and via the DSN, a read request for a data object to another computing device, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs) that is stored within the DSN, wherein unique pillar combinations of at least a decode threshold number of EDSs of the set of EDSs provide for unique recovery of the data segment; and

when no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receiving, via the interface, another unique pillar combination of at least the decode threshold number of EDSs of another set of EDSs from the another computing device via the DSN, wherein the another set of EDS are based on dispersed error encoding of the data segment in accordance with other dispersed error encoding parameters to generate the another set of EDS.

15. The method of claim 14 further comprising:

when a unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, receiving, via the interface, the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the another computing device via the DSN.

16. The method of claim 14, wherein the set of EDSs are distributedly stored among a plurality of storage units (SUs) within the DSN;

and the another computing device operating by;

assigning the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs to the computing device;

retrieving the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs from the plurality of SUs; and

transmitting the unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs that is assigned to the computing device to the computing device via the DSN.

17. The method of claim 14, wherein, when the no unique pillar combination of the at least the decode threshold number of EDSs of the set of EDSs is determined to be available within the DSN by the another computing device, the another computing device operating by:

dispersed error encoding the data segment in accordance with the other dispersed error encoding parameters to generate the another set of EDSs, wherein other unique pillar combinations of at least the decode threshold number of EDSs of the another set of EDSs provide for recovery of the data segment; and

distributedly storing the another set of EDSs among a plurality of storage units (SUs) within the DSN, wherein;

the data segment of the plurality of data segments is dispersed error encoded in accordance with the dispersed error encoding parameters to produce the set of EDSs that is of a first pillar width; and

the data segment of the plurality of data segments is dispersed error encoded in accordance with the other dispersed error encoding parameters to produce the another set of EDSs that is of a second pillar width that is greater than the first pillar width.

18. The method of claim 14, wherein:

at least one of the set of EDSs or the another set of EDSs is distributedly stored among a plurality of storage units (SUs) within the DSN;

a read threshold number of EDSs of the at least one of the set of EDSs or the another set of EDSs provides for reconstruction of the data segment; and

a write threshold number of EDSs of the at least one of the set of EDSs or the another set of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN.

19. The method of claim 14, wherein the computing device includes a SU of a plurality of storage units (SUs) within the DSN that distributedly store at least one of the set of EDSs or the another set of EDSs, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).