Rate matching technique for balancing segment cleaning and I/O workload
First Claim
1. A system comprising:
- a processor of a storage system coupled to a plurality of storage devices; and
a memory coupled to the processor and configured to store a storage input/output (I/O) stack executable by the processor, the storage I/O stack including a process configured to segment clean a segment spanning the storage devices, the memory further configured to store a feedback control loop having a plurality of accounting data structures used to balance a rate of the segment cleaning with a rate of incoming I/O workload of read and write operations directed to the storage devices, the feedback control loop further including a write weight scheduler and a plurality of read weight schedulers, the write weight scheduler provided for all storage devices of the segment to perform a full stripe write operation across the storage devices and a read weight scheduler provided for each storage device of the segment to perform a read operation to one storage device at a time.
0 Assignments
0 Petitions
Accused Products
Abstract
A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.
-
Citations
20 Claims
-
1. A system comprising:
-
a processor of a storage system coupled to a plurality of storage devices; and a memory coupled to the processor and configured to store a storage input/output (I/O) stack executable by the processor, the storage I/O stack including a process configured to segment clean a segment spanning the storage devices, the memory further configured to store a feedback control loop having a plurality of accounting data structures used to balance a rate of the segment cleaning with a rate of incoming I/O workload of read and write operations directed to the storage devices, the feedback control loop further including a write weight scheduler and a plurality of read weight schedulers, the write weight scheduler provided for all storage devices of the segment to perform a full stripe write operation across the storage devices and a read weight scheduler provided for each storage device of the segment to perform a read operation to one storage device at a time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A method comprising:
-
executing, by a processor coupled to a plurality of storage devices, a storage input/output (I/O) stack including a process configured to segment clean a segment spanning the storage devices; storing a feedback control loop in a memory coupled to the processor, the feedback control loop having a plurality of accounting data structures, a write weight scheduler and a plurality of read weight schedulers; balancing a rate of the segment cleaning with a rate of incoming I/O workload of read and write operations directed to the storage devices using the feedback control loop; performing a full stripe write operation across the storage devices using the write weight scheduler provided for all storage devices of the segment; and performing a read operation to one storage device at a time using a read weight scheduler provided for each storage device of the segment. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19)
-
-
20. A non-transitory computer readable medium including program instructions for execution on one or more processors, the program instructions configured to:
-
implement a storage input/output (I/O) stack including a process configured to segment clean a segment spanning a plurality of storage devices coupled to a processor; store a feedback control loop in a memory coupled to the processor, the feedback control loop having a plurality of accounting data structures, a write weight scheduler and a plurality of read weight schedulers; balance a rate of the segment cleaning with a rate of incoming I/O workload of read and write operations directed to the storage devices using the feedback control loop; perform a full stripe write operation across the storage devices using the write weight scheduler provided for all storage devices of the segment; and perform a read operation to one storage device at a time using a read weight scheduler provided for each storage device of the segment.
-
Specification