Time-based storage within a dispersed storage network

• US 9,639,298 B2
• Filed: 06/18/2014
• Issued: 05/02/2017
• Est. Priority Date: 07/31/2013
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin, Cocagne, Thomas Darrel, Leggette, Wesley
• Primary Examiner(s): Simonetti, Nicholas
• Application Number:
  US14/307,672
• Publication Number:
  US 20150039828A1
• Time in Patent Office: 1,049 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/306 User profiles

Abstract

A method begins by a dispersed storage (DS) processing obtaining estimated future availability information for storage units and organizing a plurality of sets of encoded data slices into a plurality of group-sets of encoded data slices. For each of the plurality of group-sets of encoded data slices, the method continues with the DS processing module estimating an approximate storage completion time to produce a plurality of approximate storage completion times. The method continues with the DS processing module establishing a time-availability pattern for writing the plurality of group-sets of encoded data slices to the storage units based on the estimated future availability information and the plurality of approximate storage completion times. The method continues with the DS processing module sending the plurality of group-sets of encoded data slices to at least some of the storage units for storage therein in accordance with the time-availability pattern.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

obtaining estimated future availability information for storage units of the DSN;

organizing a plurality of sets of encoded data slices into a plurality of group-sets of encoded data slices, wherein a group-set of encoded data slices of the plurality of group-sets of encoded data slices includes multiple sets of encoded data slices of the plurality of sets of encoded data slices, wherein data is encoded in accordance with a dispersed storage error coding function to produce the plurality of sets of encoded data slices;

for each of the plurality of group-sets of encoded data slices, estimating an approximate storage completion time to produce a plurality of approximate storage completion times;

obtaining a write threshold number;

establishing a time-availability pattern for writing the plurality of group-sets of encoded data slices to the storage units based on the estimated future availability information, the plurality of approximate storage completion times, and the write threshold number; and

sending the plurality of group-sets of encoded data slices to at least some of the storage units for storage therein in accordance with the time-availability pattern.

2. The method of claim 1 further comprises:

comparing, as time passes, actual availability information of the storage units with corresponding time portions of the estimated future availability information; and

when the actual availability information does not substantially match the estimated future availability information for the corresponding time portions, adjusting the time-availability pattern based on a difference between the actual availability information and estimated future availability information for the corresponding time portions.

3. The method of claim 1 further comprises:

for each of the plurality of group-sets of encoded data slices, assigning a transaction number; and

wherein the sending the plurality of group-sets of encoded data slices to at least some of the storage units for storage therein in accordance with the time-availability pattern includes;

generating a write request for each available storage unit per the time-availability pattern to produce a set of write requests, wherein each write request of the set of write requests includes the transaction number.

4. The method of claim 1, wherein the time-availability pattern comprises:

a plurality of time intervals, wherein storing a group-set of the plurality of group-sets of encoded data slices spans at least one time interval of the plurality of time intervals; and

an availability indication for each of the storage units in each time interval of the plurality of time intervals.

5. The method of claim 1 further comprises:

when, for the sending of a given group-set of the plurality of group-sets of encoded data slices, one of the storage units was listed as unavailable, queuing sending an encoded data slice of each set of the given group-set of encoded data slices; and

when the one of the storage units is available and when another given group-set of the plurality of group-sets of encoded data slices is being sent to the storage units, sending the encoded data slice of each set of the given group-set of encoded data slices to the one of the storage units.

6. The method of claim 1 further comprises:

when, during the sending a given group-set of the plurality of group-sets of encoded data slices, less than a write threshold number of storage units are available;

ceasing the sending of the given group-set of the plurality of group-sets of encoded data slices; and

queuing the given group-set of the plurality of group-sets of encoded data slices for sending to the at least some of the storage units when at least the write threshold number of storage units are available.

7. A dispersed storage (DS) module of a dispersed storage network (DSN), the DS module comprises:

a first module, when operable within a computing device, causes the computing device to;

obtain estimated future availability information for storage units of the DSN;

a second module, when operable within the computing device, causes the computing device to;

organize a plurality of sets of encoded data slices into a plurality of group-sets of encoded data slices, wherein a group-set of encoded data slices of the plurality of group-sets of encoded data slices includes multiple sets of encoded data slices of the plurality of sets of encoded data slices, wherein data is encoded in accordance with a dispersed storage error coding function to produce the plurality of sets of encoded data slices;

a third module, when operable within the computing device, causes the computing device to;

for each of the plurality of group-sets of encoded data slices, estimate an approximate storage completion time to produce a plurality of approximate storage completion times;

obtain a write threshold number; and

establish a time-availability pattern for writing the plurality of group-sets of encoded data slices to the storage units based on the estimated future availability information, the plurality of approximate storage completion times, and the write threshold number; and

a fourth module, when operable within the computing device, causes the computing device to;

send the plurality of group-sets of encoded data slices to at least some of the storage units for storage therein in accordance with the time-availability pattern.

8. The DS module of claim 7 further comprises:

the third module, when operable within the computing device, further causes the computing device to;

compare, as time passes, actual availability information of the storage units with corresponding time portions of the estimated future availability information; and

when the actual availability information does not substantially match the estimated future availability information for the corresponding time portions, adjust the time-availability pattern based on a difference between the actual availability information and estimated future availability information for the corresponding time portions.

9. The DS module of claim 7 further comprises:

the fourth module, when operable within the computing device, further causes the computing device to;

for each of the plurality of group-sets of encoded data slices, assign a transaction number; and

wherein the fourth module functions to cause the computing device to send the plurality of group-sets of encoded data slices to at least some of the storage units for storage therein in accordance with the time-availability pattern by;

generating a write request for each available storage unit per the time-availability pattern to produce a set of write requests, wherein each write request of the set of write requests includes the transaction number.

10. The DS module of claim 7, wherein the time-availability pattern comprises:

a plurality of time intervals, wherein storing a group-set of the plurality of group-sets of encoded data slices spans at least one time interval of the plurality of time intervals; and

an availability indication for each of the storage units in each time interval of the plurality of time intervals.

11. The DS module of claim 7 further comprises:

the fourth module, when operable within the computing device, further causes the computing device to;

when, for the sending of a given group-set of the plurality of group-sets of encoded data slices, one of the storage units was listed as unavailable, queue sending an encoded data slice of each set of the given group-set of encoded data slices; and

when the one of the storage units is available and when another given group-set of the plurality of group-sets of encoded data slices is being sent to the storage units, send the encoded data slice of each set of the given group-set of encoded data slices to the one of the storage units.

12. DS module of claim 7 further comprises:

the fourth module, when operable within the computing device, further causes the computing device to;

when, during the sending a given group-set of the plurality of group-sets of encoded data slices, less than a write threshold number of storage units are available;

cease the sending of the given group-set of the plurality of group-sets of encoded data slices; and

queue the given group-set of the plurality of group-sets of encoded data slices for sending to the at least some of the storage units when at least the write threshold number of storage units are available.