Dispersed storage based on estimated life and methods for use therewith

• US 9,781,207 B2
• Filed: 06/26/2014
• Issued: 10/03/2017
• Est. Priority Date: 08/29/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Shirley, Jr., Thomas Franklin, Gladwin, S. Christopher
• Primary Examiner(s): Taylor, Nicholas
• Assistant Examiner(s): Shiu, Ho
• Application Number:
  US14/315,925
• Publication Number:
  US 20150067101A1
• Time in Patent Office: 1,195 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G11C 2029/4402 Internal storage of test result, quality data, chip identification, repair information
• CPC Class Codes: G11C 29/44 Indication or identification of errors, e.g. for repair
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A dispersed storage and task (DST) processing unit receives a data access request. An estimated end of life is determined for a plurality of DST execution units. A subset of the plurality of DST execution units is selected, based on a threshold number associated with the data access request and further based on the estimated end of life for the subset of the plurality of DST execution units. The data access request is executed via the subset of the plurality of DST execution units.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage and task (DST) network, the method comprises:

receiving, via the DST network, a data access request from another computing device, wherein the data access request corresponds to a data object;

determining an estimated end of life for a plurality of DST execution units that store a plurality of encoded data slices (EDSs) associated with the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage error encoding function parameters to produce a set of encoded data slices (EDSs) of the plurality of EDSs, wherein the set of EDSs are distributedly stored among at least some of the plurality of DST execution units, wherein a decode threshold number of EDSs within the set of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs within the set of EDSs provides for retrieval of the data segment, and wherein a write threshold number of EDSs within the set of EDSs provides for storage of the data segment;

selecting a subset of the plurality of DST execution units, based on a threshold number associated with the data access request and further based on the estimated end of life for the subset of the plurality of DST execution units;

wherein the threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number; and

executing the data access request via the subset of the plurality of DST execution units including transmitting, via the DST network, the data access request to the subset of the plurality of DST execution units to be processed by the subset of the plurality of DST execution units and receiving responses from the plurality of DST execution units based on the subset of the plurality of DST execution units processing the data access request.

2. The method of claim 1 wherein determining the estimated end of life for the plurality of DST execution units includes:

obtaining usage information for the plurality of DST execution units to determine an amount of operation time used for the plurality of DST execution units; and

determining the estimated end of life for the plurality of DST execution units based on a difference between an estimated service life and the amount of operation time used for the plurality of DST execution units.

3. The method of claim 1 wherein the estimated end of life for the plurality of DST execution units is determined based on historical records.

4. The method of claim 1 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units.

5. The method of claim 1 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units; and

wherein the estimate life data includes at least one of;

usage information, or an estimated amount of operation time to end of life.

6. The method of claim 1 wherein selecting the subset of the plurality of DST execution units includes determining ones of the plurality of DST execution units having a highest amount of time remaining until the estimated end of life.

7. The method of claim 1 wherein selecting the subset of the plurality of DST execution units includes selecting ones of the plurality of DST execution units in proportion to an amount of time remaining until the estimated end of life.

8. A dispersed storage and task (DST) processing unit comprises:

at least one module, when operable within a computing device, that causes the computing device to;

receive, via a dispersed storage and task (DST) network, a data access request from a computing device, wherein the data access request corresponds to a data object;

determine an estimated end of life for a plurality of DST execution units that store a plurality of encoded data slices (EDSs) associated with the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage error encoding function parameters to produce a set of encoded data slices (EDSs) of the plurality of EDSs, wherein the set of EDSs are distributedly stored among at least some of the plurality of DST execution units, wherein a decode threshold number of EDSs within the set of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs within the set of EDSs provides for retrieval of the data segment, and wherein a write threshold number of EDSs within the set of EDSs provides for storage of the data segment;

select a subset of the plurality of DST execution units, based on a threshold number associated with the data access request and further based on the estimated end of life for the subset of the plurality of DST execution units;

wherein the threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number; and

execute the data access request via the subset of the plurality of DST execution units including to transmit, via the DST network, the data access request to the subset of the plurality of DST execution units to be processed by the subset of the plurality of DST execution units and receive responses from the plurality of DST execution units based on the subset of the plurality of DST execution units processing the data access request.

9. The DST processing unit of claim 8 wherein determining the estimated end of life for the plurality of DST execution units includes:

obtaining usage information for the plurality of DST execution units to determine an amount of operation time used for the plurality of DST execution units; and

determining the estimated end of life for the plurality of DST execution units based on a difference between an estimated service life and the amount of operation time used for the plurality of DST execution units.

10. The DST processing unit of claim 8 wherein the estimated end of life for the plurality of DST execution units is determined based on historical records.

11. The DST processing unit of claim 8 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units.

12. The DST processing unit of claim 8 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units; and

wherein the estimate life data includes at least one of;

usage information, or an estimated amount of operation time to end of life.

13. The DST processing unit of claim 8 wherein selecting the subset of the plurality of DST execution units includes determining ones of the plurality of DST execution units having a highest amount of time remaining until the estimated end of life.

14. The DST processing unit of claim 8 wherein selecting the subset of the plurality of DST execution units includes selecting ones of the plurality of DST execution units in proportion to an amount of time remaining until the estimated end of life.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage and task (DST) network, causes the one or more computing devices to;

receive, via the DST network, receive a data access request from another computing device, wherein the data access request corresponds to a data object;

determine an estimated end of life for a plurality of DST execution units that store a plurality of encoded data slices (EDSs) associated with the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed storage error encoding function parameters to produce a set of encoded data slices (EDSs) of the plurality of EDSs, wherein the set of EDSs are distributedly stored among at least some of the plurality of DST execution units, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for a read favorable number of EDSs for retrieval of the data segment, wherein a write threshold number of EDSs provides for a write favorable number of the set of EDSs;

select a subset of the plurality of DST execution units, based on a threshold number associated with the data access request and further based on the estimated end of life for the subset of the plurality of DST execution units;

wherein the threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number; and

execute the data access request via the subset of the plurality of DST execution units including to transmit, via the DST network, the data access request to the subset of the plurality of DST execution units to be processed by the subset of the plurality of DST execution units and to receive responses from the plurality of DST execution units based on the subset of the plurality of DST execution units processing the data access request.

16. The non-transitory computer readable storage medium of claim 15 wherein determining the estimated end of life for the plurality of DST execution units includes:

obtaining usage information for the plurality of DST execution units to determine an amount of operation time used for the plurality of DST execution units; and

determining the estimated end of life for the plurality of DST execution units based on a difference between an estimated service life and the amount of operation time used for the plurality of DST execution units.

17. The non-transitory computer readable storage medium of claim 15 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units.

18. The non-transitory computer readable storage medium of claim 15 wherein determining the estimated end of life for the plurality of DST execution units includes:

sending a query to the plurality of DST execution units; and

receiving estimated life data from the plurality of DST execution units; and

wherein the estimate life data includes at least one of;

usage information, or an estimated amount of operation time to end of life.

19. The non-transitory computer readable storage medium of claim 15 wherein selecting the subset of the plurality of DST execution units includes determining ones of the plurality of DST execution units having a highest amount of time remaining until the estimated end of life.

20. The non-transitory computer readable storage medium of claim 15 wherein selecting the subset of the plurality of DST execution units includes selecting ones of the plurality of DST execution units in proportion to an amount of time remaining until the estimated end of life.