Transferring a partial task in a distributed computing system
First Claim
Patent Images
1. A method comprises:
- determining, by a first dispersed storage and task (DST) execution unit, whether partial task processing resources of the first DST execution unit are projected to be available, wherein the determining is based on;
comparing a number of partial tasks in a current snapshot of a first local task queue to a current queue threshold, anddetermining a number of partial tasks in a projected snapshot of the first local task queue, and a projected queue threshold, wherein the determining is based at least partially on a first expected partial task performance factor and a first expected partial task allocation factor,wherein the first expected partial task performance factor is based on either when the first DST execution unit will perform one or more of the partial tasks in the projected snapshot of the first local task queue or how long the first DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the first local task queue, and wherein the first expected partial task allocation factor is based on the number of partial tasks in the first local task queue and the types of partial tasks in the first local task queue;
when partial task processing resources of the first DST execution unit are projected to be available, determining, whether partial task processing resources of a second DST execution unit are projected to be overburdened, wherein the determining is based on;
comparing a number of partial tasks in a current snapshot of a second local task queue to a current queue threshold associated with the second local task queue, anddetermining a number of partial tasks in a projected snapshot of the second local task queue, and a projected queue threshold for the second local task queue, wherein the determining is based at least partially on a second expected partial task performance factor and a second expected partial task allocation factor,wherein the second expected partial task performance factor is based on when the second DST execution unit will perform one or more of the partial tasks in the projected snapshot of the second local task queue or how long the second DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the second local task queue, and wherein the second expected partial task allocation factor is based on the number of partial tasks in the second local task queue and the types of partial tasks in the second local task queue;
when partial task processing resources of the second DST execution unit are projected to be overburdened, receiving, by the first DST execution unit, a partial task of a set of partial tasks from the second DST execution unit, wherein the partial task is associated with a data portion, wherein the data portion includes a plurality of data partitions, wherein each of the plurality of data partitions comprises a group of encoded data slices and further wherein the first DST execution unit and the second DST execution unit are each assigned at least one partial task of the set of partial tasks to perform on the group of encoded data slices; and
executing, by the first DST execution unit, the partial task to produce a partial resultant.
5 Assignments
0 Petitions
Accused Products
Abstract
A method begins by a dispersed storage (DS) processing module determining that partial task processing resources of a first DST execution unit are projected to be available. The method continues with the DS processing module ascertaining that partial task processing resources of a second DST execution unit are projected to be overburdened. The method continues with the DS processing module receiving, from the second DST execution unit, a partial task assigned to the second DST execution unit in accordance with a partial task allocation transfer policy to produce an allocated partial task and executing the allocated partial task.
-
Citations
13 Claims
-
1. A method comprises:
-
determining, by a first dispersed storage and task (DST) execution unit, whether partial task processing resources of the first DST execution unit are projected to be available, wherein the determining is based on; comparing a number of partial tasks in a current snapshot of a first local task queue to a current queue threshold, and determining a number of partial tasks in a projected snapshot of the first local task queue, and a projected queue threshold, wherein the determining is based at least partially on a first expected partial task performance factor and a first expected partial task allocation factor, wherein the first expected partial task performance factor is based on either when the first DST execution unit will perform one or more of the partial tasks in the projected snapshot of the first local task queue or how long the first DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the first local task queue, and wherein the first expected partial task allocation factor is based on the number of partial tasks in the first local task queue and the types of partial tasks in the first local task queue; when partial task processing resources of the first DST execution unit are projected to be available, determining, whether partial task processing resources of a second DST execution unit are projected to be overburdened, wherein the determining is based on; comparing a number of partial tasks in a current snapshot of a second local task queue to a current queue threshold associated with the second local task queue, and determining a number of partial tasks in a projected snapshot of the second local task queue, and a projected queue threshold for the second local task queue, wherein the determining is based at least partially on a second expected partial task performance factor and a second expected partial task allocation factor, wherein the second expected partial task performance factor is based on when the second DST execution unit will perform one or more of the partial tasks in the projected snapshot of the second local task queue or how long the second DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the second local task queue, and wherein the second expected partial task allocation factor is based on the number of partial tasks in the second local task queue and the types of partial tasks in the second local task queue; when partial task processing resources of the second DST execution unit are projected to be overburdened, receiving, by the first DST execution unit, a partial task of a set of partial tasks from the second DST execution unit, wherein the partial task is associated with a data portion, wherein the data portion includes a plurality of data partitions, wherein each of the plurality of data partitions comprises a group of encoded data slices and further wherein the first DST execution unit and the second DST execution unit are each assigned at least one partial task of the set of partial tasks to perform on the group of encoded data slices; and executing, by the first DST execution unit, the partial task to produce a partial resultant. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A first dispersed storage and task (DST) execution unit comprises:
-
a communications interface; a memory; and a processing circuitry;
where the memory includes instructions for causing the processing circuitry to;determine whether partial task processing resources of the first DST execution unit are projected to be available, wherein the partial task processing resources of the first DST execution unit are projected to be available when a number of partial tasks in a current snapshot of a first local task queue does not exceed a current queue threshold, and a projected queue threshold based on a first expected partial task performance factor and a first expected partial task allocation factor indicates that a projected number of partial tasks in a snapshot of the first local task queue would not exceed the projected queue threshold, wherein the first expected partial task performance factor is based on when the first DST execution unit will perform one or more of the partial tasks in the projected snapshot of the first local task queue or how long the first DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the first local task queue, and wherein the first expected partial task allocation factor is based on the number of partial tasks in the first local task queue and the types of partial tasks in the first local task queue; when partial task processing resources of the first DST execution unit are projected to be available, determine whether partial task processing resources of a second DST execution unit are projected to be overburdened, wherein the partial task processing resources of the second DST execution unit are projected to be available when a number of partial tasks in a current snapshot of a second local task queue does not exceed a current queue threshold, and a projected queue threshold based on a second expected partial task performance factor and a second expected partial task allocation factor indicates that a number of partial tasks in a projected snapshot of the first local task queue will exceed the projected queue threshold, wherein the second expected partial task performance factor is based on when the second DST execution unit will perform one or more of the partial tasks in the projected snapshot of the second local task queue or how long the second DST execution unit will take to perform one or more of the partial tasks in the projected snapshot of the second local task queue, and wherein the second expected partial task allocation factor is based on the number of partial tasks in the second local task queue and the types of partial tasks in the second local task queue; when partial task processing resources of the second DST execution unit are projected to be overburdened, receive a partial task of a set of partial tasks from the second DST execution unit, wherein the partial task is associated with a data portion, wherein the data portion includes a plurality of data partitions, wherein each of the plurality of data partitions includes a group of encoded data slices and further wherein the first DST execution unit and the second DST execution unit are each assigned at least one partial task of the set of partial tasks to perform on the group of encoded data slices; and execute the partial task to produce a partial resultant. - View Dependent Claims (8, 9, 10, 11, 12, 13)
-
Specification