Deduplicated data processing rate control
First Claim
1. A method for deduplicated data processing rate control using at least one processor device in a computing environment, the method comprising:
- configuring a plurality of workers for parallel processing of deduplicated data entities in a plurality of chunks; and
regulating a deduplicated data processing rate using a rate control mechanism, the rate control mechanism incorporating a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times;
wherein;
the rate control mechanism is adapted to limit a data flow rate based on a penalty acquired during a last processing of one of the plurality of chunks in a retroactive manner, andthe rate control mechanism is further adapted to operate on at least one vector representation of at least one limit specification to accommodate a variety of available dimensions corresponding to the at least one limit specification.
1 Assignment
0 Petitions
Accused Products
Abstract
A plurality of workers is configured for parallel processing of deduplicated data entities in a plurality of chunks. The deduplicated data processing rate is regulated using a rate control mechanism. The rate control mechanism incorporates a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times. The rate control mechanism limits a data flow rate based on a penalty acquired during a last processing of one of the plurality of chunks in a retroactive manner, and operates on at least one vector representation of at least one limit specification to accommodate a variety of available dimensions corresponding to the at least one limit specification.
-
Citations
23 Claims
-
1. A method for deduplicated data processing rate control using at least one processor device in a computing environment, the method comprising:
-
configuring a plurality of workers for parallel processing of deduplicated data entities in a plurality of chunks; and regulating a deduplicated data processing rate using a rate control mechanism, the rate control mechanism incorporating a debt/credit algorithm specifying which of the plurality of workers processing the deduplicated data entities must wait for each of a plurality of calculated required sleep times;
wherein;the rate control mechanism is adapted to limit a data flow rate based on a penalty acquired during a last processing of one of the plurality of chunks in a retroactive manner, and the rate control mechanism is further adapted to operate on at least one vector representation of at least one limit specification to accommodate a variety of available dimensions corresponding to the at least one limit specification. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method for deduplicated data processing rate control using at least one processor device in a computing environment, the method comprising:
regulating a deduplicated data processing rate using a rate control mechanism, the rate control mechanism incorporating a debt/credit algorithm specifying which of a plurality of workers processing deduplicated data entities must wait for each of a plurality of calculated required sleep times. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
Specification