Managing distributed system performance using accelerated data retrieval operations
First Claim
1. A computer-implemented method for managing performance of a distributed system, the method comprising:
- storing, in a plurality of storage devices of a distributed storage, a plurality of stripes associated with a data item, the plurality of stripes generated according to a coding scheme, wherein the coding scheme generates a number of stripes associated with the data item that is more than a minimum number of stripes needed to assemble the data item, and wherein the plurality of stripes includes at least one redundancy stripe;
performing a distributed process including a task that requires retrieval of the data item from the distributed storage;
determining a processing speed associated with the task; and
responsive to determining the processing speed does not meet a threshold, performing an accelerated data retrieval operation byrequesting more than the minimum number of stripes needed to assemble the data item from at least two of the plurality of storage devices of the distributed storage;
receiving at least the minimum number of stripes; and
reconstructing the data item from the minimum number of stripes according to the coding scheme.
11 Assignments
0 Petitions
Accused Products
Abstract
A distributed system is adapted to manage the performance of distributed processes. In one aspect, multiple stripes associated with a data item are stored in a distributed storage. The stored stripes include one or more stripes of redundancy information for the data item. A distributed process including at least one task is performed. During performance of the distributed process, a determination is made as to whether to perform an accelerated data retrieval operation. Responsive to a determination to perform an accelerated data retrieval operation, at least one of the one or more stripes of redundancy information for the data item is requested from the distributed storage. Other stripes associated with the data item may also be requested from the distributed storage. After a sufficient subset of stripes associated with the data item is received, the data item is reconstructed using the subset.
61 Citations
11 Claims
-
1. A computer-implemented method for managing performance of a distributed system, the method comprising:
-
storing, in a plurality of storage devices of a distributed storage, a plurality of stripes associated with a data item, the plurality of stripes generated according to a coding scheme, wherein the coding scheme generates a number of stripes associated with the data item that is more than a minimum number of stripes needed to assemble the data item, and wherein the plurality of stripes includes at least one redundancy stripe; performing a distributed process including a task that requires retrieval of the data item from the distributed storage; determining a processing speed associated with the task; and responsive to determining the processing speed does not meet a threshold, performing an accelerated data retrieval operation by requesting more than the minimum number of stripes needed to assemble the data item from at least two of the plurality of storage devices of the distributed storage; receiving at least the minimum number of stripes; and reconstructing the data item from the minimum number of stripes according to the coding scheme. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A non-transitory computer readable storage medium executing computer program instructions for managing performance of a distributed system, the computer program instructions comprising instructions for:
-
storing, in a plurality of storage devices of a distributed storage, a plurality of stripes associated with a data item, the plurality of stripes generated according to a coding scheme, wherein the coding scheme generates a number of stripes associated with the data item that is more than a minimum number of stripes needed to assemble the data item, and wherein the plurality of stripes includes at least one redundancy stripe; performing a distributed process including a task that requires retrieval of the data item from the distributed storage; determining a processing speed associated with the task; and responsive to determining the processing speed does not meet a threshold, performing an accelerated data retrieval operation by requesting more than the minimum number of stripes needed to assemble the data item from at least two of the plurality of storage devices of the distributed storage; receiving at least the minimum number of stripes; and reconstructing the data item from the minimum number of stripes according to the coding scheme. - View Dependent Claims (7, 8)
-
-
9. A system comprising:
-
a computer readable storage medium storing processor-executable computer program instructions for managing performance of a distributed system, the instructions comprising instructions for; storing, in a plurality of storage devices of a distributed storage, a plurality of stripes associated with a data item, the plurality of stripes generated according to a coding scheme, wherein the coding scheme generates a number of stripes associated with the data item that is more than a minimum number of stripes needed to assemble the data item, and wherein the plurality of stripes includes at least one redundancy stripe; performing a distributed process including a task that requires retrieval of the data item from the distributed storage; determining a processing speed associated with the task; and responsive to determining the processing speed does not meet a threshold, performing an accelerated data retrieval operation by requesting more than the minimum number of stripes needed to assemble the data item from at least two of the plurality of storage devices of the distributed storage; receiving at least the minimum number of stripes; and reconstructing the data item from the minimum number of stripes according to the coding scheme; and a processor for executing the computer program instructions. - View Dependent Claims (10, 11)
-
Specification