Distributed storage network with client subsets and methods for use therewith

• US 9,894,157 B2
• Filed: 05/27/2014
• Issued: 02/13/2018
• Est. Priority Date: 07/31/2013
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: McShane, Niall John, Kazi, Asimuddin, Gladwin, S. Christopher
• Primary Examiner(s): Avellino, Joseph E
• Assistant Examiner(s): McLeod, Marshall
• Application Number:
  US14/287,534
• Publication Number:
  US 20150039666A1
• Time in Patent Office: 1,358 Days
• Field of Search: 709201, 709212, 709213, 709246
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0668 adopting a particular infrastructure
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes identifying a plurality of DST client modules affiliated with data for storage in the DST network. A corresponding subset of a plurality of DST execution units are identified for each of the plurality of DST client modules. The data is encoded into a plurality of slices based on at least one dispersal parameter, the number of the plurality of slices corresponding to a number of the plurality of DST execution units included in a superset formed from the union of each subset of a plurality of DST execution units corresponding to each of the plurality of DST client modules. The plurality of slices are sent for storage in the superset formed from the union of each subset of a plurality of DST execution units.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage and task (DST) network, the method comprises:

encoding a data object into a decode threshold number of dispersed error encoded data slices;

sending a set of write slice requests for the decode threshold number of dispersed error encoded data slices to a first plurality of distributed storage and execution units, wherein the first plurality of distributed storage and execution units are located at a first plurality of corresponding locations;

receiving write slice data from one or more storage and execution units of the first plurality of storage and execution units, wherein the write slice data includes one or more write slice responses from the one or more storage and execution units of the first plurality of storage and execution units, and wherein the write slice data also includes one or more read slice responses from the one or more storage and execution units of the first plurality of storage and execution units;

based on at least the ratio between the write slice responses from the one or more storage and execution units of the first plurality of storage and execution units, and the read slice responses from the one or more storage and execution units of the first plurality of storage and execution units, determining whether to replicate a plurality of dispersed error encoded data slices of the decode threshold number of dispersed error encoded data slices;

when it is determined to replicate the plurality of dispersed error encoded data slices, generating a plurality of replicated dispersed error encoded data slices; and

sending a set of write slice requests for the plurality of replicated dispersed error encoded data slices to a predetermined second plurality of distributed storage and execution units, wherein the predetermined second plurality of distributed storage and execution units are at a second plurality of corresponding locations.

2. The method of claim 1, wherein the first plurality of corresponding locations are different from the locations of any of the second plurality of corresponding locations.

3. The method of claim 1, wherein all of the first plurality of corresponding locations is at a particular site and none of the predetermined second plurality of corresponding locations is at the particular site.

4. The method of claim 1, wherein the determining whether to replicate the plurality of dispersed error encoded data slices is also based on at least one of analyzing the write slice responses, a write slice response time, a requester identification, a data identification, a data type indication, a vault identification, or a replication mode predetermination.

5. The method of claim 1, wherein the determining whether to replicate the plurality of dispersed error encoded data slices is further based on analyzing the one or more write slice responses to determine when a number of storage errors compares unfavorably to a replication threshold.

6. The method of claim 1, wherein the determining whether to replicate the plurality of dispersed error encoded data slices is further based on analyzing the one or more write slice responses to determine when a number of favorable write slice responses compares unfavorably to a replication threshold.

7. The method of claim 1, further comprises:

receiving read slice response data from one or more storage and execution units of the first plurality of storage and execution units, wherein the read slice response data includes a read slice response time and the determining whether to replicate the plurality of dispersed error encoded data slices is further based on a ratio between the write slice response time and the read slice response time.

8. The method of claim 1, wherein the plurality of replicated dispersed error encoded data slices is at least a decode threshold number of dispersed error encoded data slices for the data object.

9. A dispersed storage and task (DST) network comprises:

at least one module, when operable within a computing device, that causes the computing device to;

encode a data object into a decode threshold number of dispersed error encoded data slices;

send a set of write slice requests for the decode threshold number of dispersed error encoded data slices to a first plurality of distributed storage and execution units, wherein the first plurality of distributed storage and execution units are located at a first plurality of corresponding locations;

receive write slice data from one or more storage and execution units of the first plurality of storage and execution units, wherein the write slice data includes one or more write slice responses from the one or more storage and execution units of the first plurality of storage and execution units, and wherein the write slice data also includes one or more read slice responses from the one or more storage and execution units of the first plurality of storage and execution units;

based on at least the ratio between the write slice responses from the one or more storage and execution units of the first plurality of storage and execution units and the read slice responses from the one or more storage and execution units of the first plurality of storage and execution units, determine whether to replicate a plurality of dispersed error encoded data slices of the decode threshold number of dispersed error encoded data slices;

when it is determined to replicate the plurality of dispersed error encoded data slices, generate a plurality of replicated dispersed error encoded data slices; and

send a set of write slice requests for the plurality of replicated dispersed error encoded data slices to a predetermined second plurality of distributed storage and execution units, wherein the predetermined second plurality of distributed storage and execution units are at a second plurality of corresponding locations.

10. The DST network of claim 9, wherein the first plurality of corresponding locations are different from the second plurality of corresponding locations.

11. The DST network of claim 9, wherein all of the first plurality of corresponding locations is at a particular site and none of the second plurality of corresponding locations is at the particular site.

12. The DST network of claim 9, wherein the determine whether to replicate the plurality of dispersed error encoded data slices is also based on at least one of analyzing the write slice responses, a write slice response time, a requester identification, a data identification, a data type indication, a vault identification, or a replication mode predetermination.

13. The DST network of claim 9, wherein the determine whether to replicate the plurality of dispersed error encoded data slices is further based on analyzing the one or more write slice responses to determine when a number of storage errors compares unfavorably to a replication threshold.

14. The DST network of claim 9, wherein the determine whether to replicate the plurality of dispersed error encoded data slices is further based on analyzing the one or more write slice responses to determine when the number of favorable write slice responses compares unfavorably to a replication threshold.

15. The DST network of claim 9, wherein the at least one module, when operable within a computing device, causes the computing device to:

receive read slice response data from one or more storage and execution units of the first plurality of storage and execution units, wherein the read slice response data includes a read slice response time and the determine whether to replicate the plurality of dispersed error encoded data slices is further based on a ratio between the write slice response time and the read slice response time.

16. The DST network of claim 9, wherein the plurality of replicated dispersed error encoded data slices is at least a decode threshold number of dispersed error encoded data slices for the data object.