Allocating distributed storage and task execution resources
First Claim
1. A distributed storage and/or task (DST) client module comprisingan outbound DST processing module, when operable within a computing device, causes the computing device to:
- allocate execution of a plurality of partial tasks of a distributed computing function to a set of distributed storage and task execution (DSTE) units to produce a plurality of partial results in accordance with a desired executing efficiency based on respective executing resource capabilities of respective DSTE units within the set of DSTE units; and
allocate processing of a plurality of data access requests for data to the set of DSTE units in accordance with a desired data access efficiency based on respective data access capabilities of respective DSTE units within the set of DSTE units, wherein the data is dispersed storage error encoded to produce one or more sets of encoded data slices;
an efficiency module, when operable within the computing device, causes the computing device to;
establish the desired executing efficiency and the desired data access efficiency to obtain a desired utilization of the set of DSTE units based on both the respective executing resource capabilities and the respective data access capabilities of the respective DSTE units within the set of DSTE units; and
an inbound DST processing module, when operable within the computing device, causes the computing device to;
receive the plurality of partial results from the set of DSTE units; and
aggregate the plurality of partial results to generate a result.
5 Assignments
0 Petitions
Accused Products
Abstract
A method begins by a dispersed storage (DS) processing module obtaining a plurality of data access requests and a request to execute a distributed computing function that includes a plurality of partial tasks. The method continues with the DS processing module allocating execution of the plurality of partial tasks to a set of distributed storage and task execution (DSTE) units in accordance with a desired executing efficiency and allocating processing of the plurality of data access requests in accordance with a desired data access efficiency. The method continues with the DS processing module establishing the desired executing efficiency and the desired data access efficiency to obtain a desired utilization of the set of DSTE units.
84 Citations
20 Claims
-
1. A distributed storage and/or task (DST) client module comprising
an outbound DST processing module, when operable within a computing device, causes the computing device to: -
allocate execution of a plurality of partial tasks of a distributed computing function to a set of distributed storage and task execution (DSTE) units to produce a plurality of partial results in accordance with a desired executing efficiency based on respective executing resource capabilities of respective DSTE units within the set of DSTE units; and allocate processing of a plurality of data access requests for data to the set of DSTE units in accordance with a desired data access efficiency based on respective data access capabilities of respective DSTE units within the set of DSTE units, wherein the data is dispersed storage error encoded to produce one or more sets of encoded data slices; an efficiency module, when operable within the computing device, causes the computing device to; establish the desired executing efficiency and the desired data access efficiency to obtain a desired utilization of the set of DSTE units based on both the respective executing resource capabilities and the respective data access capabilities of the respective DSTE units within the set of DSTE units; and an inbound DST processing module, when operable within the computing device, causes the computing device to; receive the plurality of partial results from the set of DSTE units; and aggregate the plurality of partial results to generate a result. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A distributed storage and/or task (DST) client module comprising
an outbound DST processing module, when operable within a computing device, causes the computing device to: -
allocate, in an imbalanced manner, execution of a plurality of partial tasks of a distributed computing function to a set of distributed storage and task execution (DSTE) units to produce a plurality of partial results in accordance with a desired executing efficiency based on respective executing resource capabilities of respective DSTE units within the set of DSTE units such that, from time-to-time, differing DSTE units of the set of DSTE units are not assigned execution of a partial task of the plurality of partial tasks; and allocate, in an imbalanced manner, processing of a plurality of data access requests for data to the set of DSTE units in accordance with a desired data access efficiency based on respective data access capabilities of respective DSTE units within the set of DSTE units, wherein the data is dispersed storage error encoded to produce one or more sets of encoded data slices such that, from time-to-time, the differing DSTE units of the set of DSTE units are not processing a data access request of the plurality of data access requests; an efficiency module, when operable within the computing device, causes the computing device to; establish the desired executing efficiency and the desired data access efficiency to obtain a desired utilization of the set of DSTE units based on both the respective executing resource capabilities and the respective data access capabilities of the respective DSTE units within the set of DSTE units; and an inbound DST processing module, when operable within the computing device, causes the computing device to; receive the plurality of partial results from the set of DSTE units; and aggregate the plurality of partial results to generate a result. - View Dependent Claims (10, 11, 12, 13)
-
-
14. A method for execution by one or more processing modules of one or more computing devices, the method comprising:
-
allocating, by the one or more processing modules, execution of a plurality of partial tasks of a distributed computing function to a set of distributed storage and task execution (DSTE) units to produce a plurality of partial results in accordance with a desired executing efficiency based on respective executing resource capabilities of respective DSTE units within the set of DSTE units; allocating, by the one or more processing modules, processing of a plurality of data access requests for data to the set of DSTE units in accordance with a desired data access efficiency based on respective data access capabilities of respective DSTE units within the set of DSTE units, wherein the data is dispersed storage error encoded to produce one or more sets of encoded data slices; establishing, by the one or more processing modules, the desired executing efficiency and the desired data access efficiency to obtain a desired utilization of the set of DSTE units based on both the respective executing resource capabilities and the respective data access capabilities of the respective DSTE units within the set of DSTE units; receiving, by the one or more processing modules, the plurality of partial results from the set of DSTE units; and aggregating, by the one or more processing modules, the plurality of partial results to generate a result. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification