Load rebalancing for shared resource

• US 8,539,197 B1
• Filed: 06/29/2010
• Issued: 09/17/2013
• Est. Priority Date: 06/29/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Marshall, Bradley E., Sivasubramanian, Swaminathan, Certain, Tate Andrew, Maniscalco, Nicholas J.
• Primary Examiner(s): Bertram, Ryan
• Application Number:
  US12/826,576
• Time in Patent Office: 1,176 Days
• Field of Search: 711/112, 711/148, 711/165, 711/170, 711/173, 718104-105
• US Class Current: 711/173
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

Various aspects of a data volume or other shared resource are determined and updated dynamically for purposes such as to provide guaranteed qualities of service. For example, the number of partitions in a data volume and/or the way in which data is stored across those partitions can be updated dynamically without significantly impacting the customer using the volume. The data stored to the volume can be striped or otherwise distributed across a number of logical areas, which then can be distributed across the partitions. Separate mappings can be used for the data in each logical area, and the logical areas in each partition, such that when moving a logical area only a single mapping has to be updated, regardless of the amount of data in that logical area. Further, logical areas can be moved between partitions without the need to repartition or redistributed the data in the data volume.

Claims

1. A computer-implemented method of rebalancing a load across a number of partitions of a data volume, comprising:

under control of one or more computer systems configured with executable instructions,

monitoring a usage pattern for a customer associated with a data volume in a data environment using at least one monitoring component of a control environment, the data volume including a number of partitions each containing at least one logical area and corresponding to a separate physical storage device, data stored to the data volume being distributed across the logical areas using a data striping algorithm;

determining whether the usage pattern provides an uneven usage across the number of partitions with respect to at least one functional aspect of the data volume; and

in response to determining that the usage pattern provides an uneven usage across the number of partitions,

determining a number of logical areas to be moved from at least one relatively heavily loaded partition to at least one relatively lightly loaded partition with respect to the at least one functional aspect;

moving the determined number of logical areas; and

updating a first set of mappings to indicate a current partition associated with each logical area as a result of moving the determined number of logical areas,

wherein the determined number of logical areas is capable of being moved without updating a second set of mappings indicating the data stored in each logical area.

2. The computer-implemented method of claim 1, wherein the at least one functional aspect includes at least one of a storage capacity, a rate of input/output operations per second (IOPS), a latency, and a bandwidth.

3. The computer-implemented method of claim 1, further comprising:

receiving a request to update at least one functional aspect of the data volume, and

determining whether the update causes the at least one functional aspect to fall outside an allowable range, whereby the system is operable to determine a number of logical areas to be moved from at least one relatively heavily loaded partition to at least one relatively lightly loaded partition with respect to the at least one functional aspect.

4. A computer-implemented method of dynamically rebalancing a data volume, comprising:

under control of one or more computer systems configured with executable instructions,

monitoring at least one functional aspect of a data volume, the data volume including a number of partitions each containing at least one logical area, data stored to the data volume being distributed across the logical areas;

if at least one monitored functional aspect is determined to fall outside an allowable range with respect to at least one partition, determining at least one logical area to be moved between partitions in the data volume in order to bring the at least one monitored aspect back within the allowable range; and

for each determined logical area determined to be moved between partitions, moving the logical area and updating a first mapping to indicate a current partition associated with the moved logical area,

wherein each logical area is capable of being moved without updating a second set of mappings indicating the data stored in the logical area.

5. The computer-implemented method of claim 4, wherein each partition is capable of storing a different number of logical areas, the different number of logical areas being based at least in part upon a usage of each logical area.

6. The computer-implemented method of claim 4, wherein determining at least one logical area to be moved between partitions further comprises selecting specific logical areas to be moved based at least in part upon the usage of each logical area.

7. The computer-implemented method of claim 4, wherein the at least one functional aspect of a data volume cannot be brought within the allowable range using a current number of partitions in the data volume, further comprising:

increasing the number of partitions in the data volume,

wherein determining the at least one logical area to be moved between partitions in the data volume in order to bring the at least one monitored aspect back within the allowable range is based at least in part upon the increased number of partitions.

8. The computer-implemented method of claim 4, wherein the at least one functional aspect can be maintained in the allowable range using a fewer number of partitions in the data volume, further comprising:

decreasing the number of partitions in the data volume,

wherein determining the at least one logical area to be moved between partitions is based at least in part upon the decreased number of partitions.

9. The computer-implemented method of claim 4, wherein each partition corresponds to a separate physical storage device.

10. The computer-implemented method of claim 4, wherein each partition corresponds to a dedicated storage spindle or flash drive.

11. The computer-implemented method of claim 4, wherein logical areas are further capable of being moved in order to substantially evenly spread the logical areas across the number of partitions while still keeping the at least one functional aspect within the allowable range.

12. The computer-implemented method of claim 4, wherein at least one of the first and second mappings comprises a lookup table.

13. The computer-implemented method of claim 4, wherein the data is distributed across the logical areas using at least one of a striping or a hashing algorithm.

14. The computer-implemented method of claim 4, wherein the at least one functional aspect includes a rate of input/output operations per second (IOPS).

15. The computer-implemented method of claim 4, wherein the monitoring is performed by at least one component of a control environment operable to monitor usage of the data volume in a data environment.

16. The computer-implemented method of claim 4, wherein the allowable range for at least one functional aspect is determined based at least in part upon a rate guarantee provided for the data volume.

17. A system for dynamically rebalancing a data volume, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the system to;

monitor at least one functional aspect of a data volume, the data volume including a number of partitions each containing at least one logical area, data stored to the data volume being distributed across the logical areas;

if at least one monitored functional aspect is determined to fall outside an allowable range with respect to at least one partition, determine at least one logical area to be moved between partitions in the data volume in order to bring the at least one monitored aspect back within the allowable range; and

for each determined logical area determined to be moved between partitions, move the logical area and updating a first mapping to indicate a current partition associated with the moved logical area,

wherein each logical area is capable of being moved without updating a second set of mappings indicating the data stored in the logical area.

18. The system of claim 17, wherein each partition is capable of storing a different number of logical areas, the different number of logical areas being based at least in part upon a usage of each logical area.

19. The system of claim 17, wherein determining at least one logical area to be moved between partitions further comprises selecting specific logical areas to be moved based at least in part upon the usage of that logical area.

20. The system of claim 17, wherein each partition corresponds to a separate data storage device.

21. The system of claim 17, wherein logical areas are further capable of being moved in order to substantially evenly spread the logical areas across the number of partitions while still keeping the at least one functional aspect within the allowable range.

22. The system of claim 17, wherein the data is distributed across the logical areas using at least one of a striping or a hashing algorithm.

23. The system of claim 17, wherein the at least one functional aspect includes at least one of a storage capacity, a rate of input/output operations per second (IOPS), and a bandwidth.

24. A non-transitory computer-readable storage medium including instructions for dynamically rebalancing a data volume, the instructions when executed by a processor causing the processor to:

monitor at least one functional aspect of a data volume, the data volume including a number of partitions each containing at least one logical area, data stored to the data volume being distributed across the logical areas;

if at least one monitored functional aspect is determined to fall outside an allowable range with respect to at least one partition, determine at least one logical area to be moved between partitions in the data volume in order to bring the at least one monitored aspect back within the allowable range; and

for each determined logical area determined to be moved between partitions, move the logical area and updating a first mapping to indicate a current partition associated with the moved logical area,

wherein each logical area is capable of being moved without updating a second set of mappings indicating the data stored in the logical area.

25. The non-transitory computer-readable storage medium of claim 24, wherein determining at least one logical area to be moved between partitions further comprises selecting specific logical areas to be moved based at least in part upon the usage of each logical area.

26. The non-transitory computer-readable storage medium of claim 24, wherein logical areas are further capable of being moved in order to substantially evenly spread the logical areas across the number of partitions while still keeping the at least one functional aspect within the allowable range.

27. The non-transitory computer-readable storage medium of claim 24, wherein the data is distributed across the logical areas using at least one of a striping or a hashing algorithm.

28. The non-transitory computer-readable storage medium of claim 24, wherein the at least one functional aspect includes at least one of a storage capacity, a rate of input/output operations per second (IOPS), and a bandwidth.

29. A computer-implemented method of dynamically rebalancing a data volume, comprising:

under control of one or more computer systems configured with executable instructions,

monitoring at least one functional aspect of a data volume, the data volume including a number of partitions each containing at least one piece of data;

if at least one monitored functional aspect is determined to fall outside an allowable range with respect to at least one partition, determining at least one piece of data to be moved between partitions in the data volume in order to bring the at least one monitored aspect back within the allowable range; and

for each determined piece of data determined to be moved between partitions, moving the piece of data and updating a first table of mappings to indicate a current partition associated with a logical area that includes the moved piece of data,

wherein piece of data is capable of being moved without updating a second table of mappings indicating the piece of data included in the logical area.

30. The computer-implemented method of claim 29, wherein the at least one piece of data includes at least one of a data stripe, data chunk, or data block.

31. The computer-implemented method of claim 29, wherein at least the first table of mappings is an override table.

32. The computer-implemented method of claim 29, wherein at least one piece of data is moved between logical areas stored across the number partitions.