Bandwidth throttling of virtual disks

• US 9,069,616 B2
• Filed: 09/23/2011
• Issued: 06/30/2015
• Est. Priority Date: 09/23/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kadatch, Andrew, Khorun, Sergey
• Primary Examiner(s): Puente, Emerson
• Assistant Examiner(s): DO, STEVEN M
• Application Number:
  US13/243,752
• Publication Number:
  US 20130081014A1
• Time in Patent Office: 1,376 Days
• Field of Search: 718/1
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing resources in a computing system. For virtual hard disk drives supported by multiple physic hard disk drives over a network, artificial throttling of the disk access bandwidth is implemented, such that the resulting latency behavior of each virtual hard disk drive resembles the latency behavior of a corresponding physical hard disk drive emulated by the virtual hard disk drive. In various implementations, the artificial throttling simulates both characteristics of sequential access latencies and random access latencies on a physical hard drive.

Claims

1. A computer-implemented method, the method comprising:

receiving a first disk access request for disk input/output (I/O) access on a virtual disk of a virtual machine, the virtual disk emulating a local physical disk for the virtual machine, wherein the virtual disk includes logical data blocks that map to physical data blocks distributed among multiple physical storage devices, the access by the virtual disk to the multiple physical storage devices being in parallel;

deriving a simulated latency cost for the requested disk I/O access according to one or more latency and throughput characteristics of the local physical disk emulated by the virtual disk, the simulated latency cost including a simulated data transfer latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk and a simulated random access latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk; and

determining whether to delay access to one or more of the multiple physical storage devices based on a comparison between the simulated latency cost and a current value of a disk access service quota allotted to the virtual disk, wherein the current value of the disk access service quota is replenished over time, is capped at a predetermined cap value, and is decreased by a respective simulated latency cost of each previous disk access request for the virtual disk when respective requested disk I/O access of the previous disk access request was carried out.

2. The method of claim 1, further comprising:

carrying out requested I/O accesses for multiple disk access requests in parallel on the multiple physical storage devices, provided that the current value of the disk access quota is above a sum of respective simulated latency costs of the multiple disk access requests.

3. The method of claim 1, wherein the virtual disk is one of a plurality of virtual disks hosted on the multiple physical storage devices, and the method further comprises:

for each of the plurality of virtual disks;

incrementing a respective disk access service quota of the virtual disk according to a predetermined replenish rate associated with the virtual disk until a current value of the respective disk access service quota reaches a respective predetermined cap value associated with the virtual disk; and

temporarily pausing the incrementing until the current value of the respective disk access service quota of the virtual disk falls below the respective predetermined cap value associated with the virtual disk.

4. The method of claim 1, further comprising:

keeping a record of respective end positions of one or more previous disk access requests for the virtual disk that were received during a predetermined time interval immediately prior to a current time; and

calculating the simulated random access latency for the requested disk I/O access of the first disk access request based at least on a seek distance between a start position of the requested disk I/O access and a nearest end position among the respective end positions of the one or more previous disk access requests.

5. The method of claim 1, further comprising:

upon determination that the simulated latency cost of the first disk access request is at or below the current value of the disk access service quota allotted to the virtual disk;

carrying out the requested disk I/O access of the first disk access request; and

deducting the simulated latency cost of the requested I/O access from the current value of the disk service access quota allotted to the virtual disk; and

upon determination that the simulated latency cost of the first disk access request exceeds the current value of the disk access service quota allotted to the virtual disk;

queuing the first disk access request until the current value of the disk service access quota has reached at or above the simulated latency cost of the first disk access request through scheduled replenishing.

6. The method of claim 1, further comprising:

monitoring response time for respective disk access requests for a plurality of virtual disks that share a plurality of underlying physical storage devices;

monitoring utilization level of a disk access capacity of the plurality of underlying physical storage devices; and

adjusting one or more of a group of predetermined parameters, the group of predetermined parameters including replenish rates and cap values of respective disk access service quotas of the plurality of virtual disks and one or more predetermined values used in calculating the simulated latency cost of the requested disk I/O access.

7. A computer-implemented method, the method comprising:

providing access to a plurality of virtual disks, the plurality of virtual disks being hosted on a plurality of underlying physical storage devices managed by one or more servers, the access by each of the plurality of virtual disks to the plurality of underlying physical storage devices being in parallel, wherein each virtual disk emulates a local physical disk, and wherein the virtual disk includes logical data blocks that map to physical data blocks distributed among multiple physical storage devices;

deriving a simulated latency cost for each disk access request received for each of the plurality of virtual disks according to one or more latency and throughput characteristics of a local physical disk emulated by the virtual disk, the simulated latency cost including a simulated data transfer latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk and a simulated random access latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk; and

controlling a promptness by which respective requested disk accesses for each of the plurality of virtual disks are executed on the plurality of underlying physical storage devices according to a comparison between the simulated latency costs of the respective requested disk accesses and a simulated disk access capacity of the local physical disk emulated by the virtual disk.

8. A computer storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

receiving a first disk access request for disk input/output (I/O) access on a virtual disk of a virtual machine, the virtual disk emulating a local physical disk for the virtual machine, wherein the virtual disk includes logical data blocks that map to physical data blocks distributed among multiple physical storage devices, the access by the virtual disk to the multiple physical storage devices being in parallel;

deriving a simulated latency cost for the requested disk I/O access according to one or more latency and throughput characteristics of the local physical disk emulated by the virtual disk, the simulated latency cost including a simulated data transfer latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk and a simulated random access latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk; and

determining whether to delay access to one or more of the multiple physical storage devices based on a comparison between the simulated latency cost and a current value of a disk access service quota allotted to the virtual disk, wherein the current value of the disk access service quota is replenished over time, is capped at a predetermined cap value, and is decreased by a respective simulated latency cost of each previous disk access request for the virtual disk when respective requested disk I/O access of the previous disk access request was carried out.

9. The computer storage medium of claim 8, wherein the operations further comprise:

carrying out requested I/O accesses for multiple disk access requests in parallel on the multiple physical storage devices, provided that the current value of the disk access quota is above a sum of respective simulated latency costs of the multiple disk access requests.

10. The computer storage medium of claim 8, wherein the virtual disk is one of a plurality of virtual disks hosted on the multiple physical storage devices, and the operations further comprise:

for each of the plurality of virtual disks;

incrementing a respective disk access service quota of the virtual disk according to a predetermined replenish rate associated with the virtual disk until a current value of the respective disk access service quota reaches a respective predetermined cap value associated with the virtual disk; and

temporarily pausing the incrementing until the current value of the respective disk access service quota of the virtual disk falls below the respective predetermined cap value associated with the virtual disk.

11. The computer storage medium of claim 8, wherein the operations further comprise:

keeping a record of respective end positions of one or more previous disk access requests for the virtual disk that were received during a predetermined time interval immediately prior to a current time; and

calculating the simulated random access latency for the requested disk I/O access of the first disk access request based at least on a seek distance between a start position of the requested disk I/O access and a nearest end position among the respective end positions of the one or more previous disk access requests.

12. The computer storage medium of claim 8, wherein the operations further comprise:

upon determination that the simulated latency cost of the first disk access request is at or below the current value of the disk access service quota allotted to the virtual disk;

carrying out the requested disk I/O access of the first disk access request; and

deducting the simulated latency cost of the requested I/O access from the current value of the disk service access quota allotted to the virtual disk; and

upon determination that the simulated latency cost of the first disk access request exceeds the current value of the disk access service quota allotted to the virtual disk;

queuing the first disk access request until the current value of the disk service access quota has reached at or above the simulated latency cost of the first disk access request through scheduled replenishing.

13. The computer storage medium of claim 8, wherein the operations further comprise:

monitoring response time for respective disk access requests for a plurality of virtual disks that share a plurality of underlying physical storage devices;

monitoring utilization level of a disk access capacity of the plurality of underlying physical storage devices; and

adjusting one or more of a group of predetermined parameters, the group of predetermined parameters including replenish rates and cap values of respective disk access service quotas of the plurality of virtual disks and one or more predetermined values used in calculating the simulated latency cost of the requested disk I/O access.

14. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising;

receiving a first disk access request for disk input/output (I/O) access on a virtual disk of a virtual machine, the virtual disk emulating a local physical disk for the virtual machine, wherein the virtual disk includes logical data blocks that map to physical data blocks distributed among multiple physical storage devices, the access by the virtual disk to the multiple physical storage devices being in parallel;

deriving a simulated latency cost for the requested disk I/O access according to one or more latency and throughput characteristics of the local physical disk emulated by the virtual disk, the simulated latency cost including a simulated data transfer latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk and a simulated random access latency for carrying out the requested disk I/O access on the local physical disk emulated by the virtual disk; and

determining whether to delay access to one or more of the multiple physical storage devices based on a comparison between the simulated latency cost and a current value of a disk access service quota allotted to the virtual disk, wherein the current value of the disk access service quota is replenished over time, is capped at a predetermined cap value, and is decreased by a respective simulated latency cost of each previous disk access request for the virtual disk when respective requested disk I/O access of the previous disk access request was carried out.

15. The system of claim 14, wherein the operations further comprise:

carrying out requested I/O accesses for multiple disk access requests in parallel on the multiple physical storage devices, provided that the current value of the disk access quota is above a sum of respective simulated latency costs of the multiple disk access requests.

16. The system of claim 14, wherein the virtual disk is one of a plurality of virtual disks hosted on the multiple physical storage devices, and the operations further comprise:

for each of the plurality of virtual disks;

incrementing a respective disk access service quota of the virtual disk according to a predetermined replenish rate associated with the virtual disk until a current value of the respective disk access service quota reaches a respective predetermined cap value associated with the virtual disk; and

temporarily pausing the incrementing until the current value of the respective disk access service quota of the virtual disk falls below the respective predetermined cap value associated with the virtual disk.

17. The system of claim 14, wherein the operations further comprise:

keeping a record of respective end positions of one or more previous disk access requests for the virtual disk that were received during a predetermined time interval immediately prior to a current time; and

calculating the simulated random access latency for the requested disk I/O access of the first disk access request based at least on a seek distance between a start position of the requested disk I/O access and a nearest end position among the respective end positions of the one or more previous disk access requests.

18. The system of claim 14, wherein the operations further comprise:

upon determination that the simulated latency cost of the first disk access request is at or below the current value of the disk access service quota allotted to the virtual disk;

carrying out the requested disk I/O access of the first disk access request; and

deducting the simulated latency cost of the requested I/O access from the current value of the disk service access quota allotted to the virtual disk; and

upon determination that the simulated latency cost of the first disk access request exceeds the current value of the disk access service quota allotted to the virtual disk;

queuing the first disk access request until the current value of the disk service access quota has reached at or above the simulated latency cost of the first disk access request through scheduled replenishing.

19. The system of claim 14, wherein the operations further comprise:

monitoring response time for respective disk access requests for a plurality of virtual disks that share a plurality of underlying physical storage devices;

monitoring utilization level of a disk access capacity of the plurality of underlying physical storage devices; and

adjusting one or more of a group of predetermined parameters, the group of predetermined parameters including replenish rates and cap values of respective disk access service quotas of the plurality of virtual disks and one or more predetermined values used in calculating the simulated latency cost of the requested disk I/O access.