Monitoring and automatic scaling of data volumes

• US 10,282,231 B1
• Filed: 11/05/2015
• Issued: 05/07/2019
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sivasubramanian, Swaminathan, McAlister, Grant Alexander MacDonald, Milovanovic, Milovan
• Primary Examiner(s): Sison, June Y
• Application Number:
  US14/933,894
• Time in Patent Office: 1,279 Days
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory

Abstract

Aspects of a data environment, such as various capacities of data stores and instances, can be managed using a separate control environment. A monitoring component of the control environment can periodically communicate with the data environment to obtain performance information. The information is analyzed, using algorithms such as trending and extrapolation algorithms, to determine any recommended scaling of resources in the data environment. The scaling can be performed automatically, or as authorized by a customer. A workflow can be instantiated that includes tasks necessary to perform the scaling. The scaling of storage capacity can be performed without affecting the availability of the data store.

Claims

1. A method, comprising:

performing, by one or more computing devices;

receiving performance information for a resource of a provisioned database, wherein one or more physical devices provide a capacity of the resource;

predicting, based at least in part on analysis of the performance information for the resource, a first anticipated usage value for the resource,

determining that the first anticipated usage value for the resource will fall outside of a range within a period of time;

determining, based on at least two criteria comprising;

the first anticipated usage value, and a second factor distinct from the first anticipated usage value, a scaling action for adjusting the capacity of the resource such that a second anticipated usage value determined for the resource falls within the range;

determining that authorization is granted for the determined scaling action;

in response to determining that the authorization is granted, executing the determined scaling action to adjust the capacity of the resource, comprising;

adjusting a number of the one or more physical devices that provide the capacity; and

reallocating the capacity of the resource among the adjusted number of physical devices.

2. The method of claim 1, wherein the resource is a logical data volume, wherein the one or more physical devices are one or more storage devices storing respective data volumes that together implement the logical data volume, and wherein adjusting the number of the one or more physical devices that provide the capacity comprises:

provisioning another data volume at another storage device; and

extending the logical data volume to include the other data volume.

3. The method of claim 2, wherein reallocating the capacity of the resource among the adjusted number of physical devices comprises, moving one or more portions of at least one of the respective data volumes to be included as part of the respective data volume at a different one of the storage devices.

4. The method of claim 1, wherein the resource includes at least one of a processing component, a data storage component, a memory component, a communications component, a network I/O (input/output) component, or a data I/O component.

5. The method of claim 1, wherein determining that authorization is granted for the scaling action comprises accessing stored authorization information for the provisioned database that indicates authorization is granted for the scaling action to be automatically performed.

6. The method of claim 1, wherein the provisioned database is available for servicing client access requests during the executing of the scaling action to adjust the capacity of the resource.

7. The method of claim 1, wherein the provisioned database is hosted as part of a network-based database service, wherein the receiving the performance information, the determining that the first anticipated usage value for the resource will fall outside of the range within the period of time, the determining the scaling action, the determining that authorization is granted, and the executing the scaling action are performed as part of a control plane for the network-based database service, wherein the provisioned database is stored within a data plane for the network-based database service, and wherein a client of the provisioned database accesses the provisioned database directly in the data plane without accessing the control plane.

8. A system, comprising:

at least one processor;

a memory, that stores program instructions that when executed by the at least one processor cause the at least one processor to;

receive performance information for a resource of a provisioned database, wherein one or more physical devices provide a capacity of the resource;

predict, based at least in part on analysis of the performance information for the resource, a first anticipated usage value for the resource;

determine that the first anticipated usage value for the resource will fall outside of a range within a period of time;

determine, based on at least two criteria comprising the first anticipated usage value and a second factor distinct from the first anticipated usage value, a scaling action, for adjusting the capacity of the resource such that a second anticipated usage value determined for the resource will fall within the range;

determine that authorization is granted for the determined scaling action;

in response to the determination that the authorization is granted, execute the determined scaling action to adjust the capacity of the resource, wherein to execute the scaling action the program instructions cause the at least one processor to;

adjust a number of the one or more physical devices that provide the capacity; and

reallocate the capacity of the resource among the adjusted number of physical devices.

9. The system of claim 8, wherein the resource is a logical data volume, wherein the one or more physical devices are one or more storage devices storing respective data volumes that together implement the logical data volume, and wherein to adjust the number of the one or more physical devices that provide the capacity, the program instructions cause the at least one processor to:

provision another data volume at another storage device; and

extend the logical data volume to include the other data volume.

10. The system of claim 9, wherein to reallocate the capacity of the resource among the adjusted number of physical devices, the program instructions cause the at least one processor to move one or more portions of at least one of the respective data volumes to be included as part of the respective data volume at a different one of the storage devices.

11. The system of claim 8, wherein the resource includes at least one of a processing component, a data storage component, a memory component, a communications component, a network I/O (input/output) component, or a data I/O component.

12. The system of claim 8, wherein to determine that authorization is granted for the scaling action, the program instructions cause the at least one processor to access stored authorization information for the provisioned database that indicates authorization is granted for the scaling action to be automatically performed.

13. The system of claim 8, wherein the provisioned database is available for servicing client access requests during the execution of the scaling action to adjust the capacity of the resource.

14. The system of claim 8, wherein the provisioned database is hosted as part of a network-based database service, wherein the receipt of the performance information, the determination that the first anticipated usage value for the resource will fall outside of the range within the period of time, the determination of the scaling action, the determination that authorization is granted, and the execution of the scaling action are performed as part of a control plane for the network-based database service, wherein the provisioned database is stored within a data plane for the network-based database service, and wherein a client of the provisioned database accesses the provisioned database directly in the data plane without accessing the control plane.

15. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:

receiving performance information for a resource of a provisioned database, wherein one or more physical devices provide a capacity of the resource;

predict, based at least in part on analysis of the performance information for the resource, a first anticipate usage value for the resource;

determining that the first anticipated usage value for the resource will fall outside of a range within a period of time;

determining, based on at least two criteria comprising;

the first anticipated usage value and a second factor distinct from the first anticipated usage value, a scaling action, for adjusting the capacity of the resource for the resource falls within the range;

determining that authorization is granted for the determined scaling action;

in response to determining that the authorization is granted, executing the determined scaling action to adjust the capacity of the resource, wherein, in executing the scaling action, the program instructions cause the one or more computing devices to implement;

adjusting a number of the one or more physical devices that provide the capacity; and

reallocating the capacity of the resource among the adjusted number of physical devices.

16. The non-transitory, computer-readable storage medium of claim 15, wherein the resource is a logical data volume, wherein the one or more physical devices are one or more storage devices storing respective data volumes that together implement the logical data volume, and wherein, in adjusting the number of the one or more physical devices that provide the capacity, the program instructions cause the one or more computing devices to implement:

provisioning another data volume at another storage device; and

extending the logical data volume to include the other data volume.

17. The non-transitory, computer-readable storage medium of claim 16, wherein, in reallocating the capacity of the resource among the adjusted number of physical devices, the program instructions cause the one or more computing devices to implement moving one or more portions of at least one of the respective data volumes to be included as part of the respective data volume at a different one of the storage devices.

18. The non-transitory, computer-readable storage medium of claim 15, wherein the resource includes at least one of a processing component, a data storage component, a memory component, a communications component, a network I/O (input/output) component, or a data I/O component.

19. The non-transitory, computer-readable storage medium of claim 15, wherein the provisioned database is available for servicing client access requests during the executing of the scaling action to adjust the capacity of the resource.

20. The non-transitory, computer-readable storage medium of claim 15, wherein the provisioned database is hosted as part of a network-based database service, wherein the receiving the performance information, the determining that the first anticipated usage value for the resource will fall outside of the range within the period of time, the determining the scaling action, the determining that authorization is granted, and the executing the scaling action are implemented as part of a control plane for the network-based database service, wherein the provisioned database is stored within a data plane for the network-based database service, and wherein a client of the provisioned database accesses the provisioned database directly in the data plane without accessing the control plane.