Control service for relational data management

• US 8,713,060 B2
• Filed: 03/31/2009
• Issued: 04/29/2014
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sivasubramanian, Swaminathan, McAlister, Grant A. M., Franklin, Paul David, Sheth, Rajesh Sudhakar, Horsley, James
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Waldron, Scott A
• Application Number:
  US12/415,958
• Publication Number:
  US 20100251242A1
• Time in Patent Office: 1,855 Days
• Field of Search: 707/784, 707/790, 719/328
• US Class Current: 707/792
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

Aspects of a data environment, such as the creation, provisioning, and management of data stores and instances, are managed using a separate control environment. A user can call into an externally-facing interface of the control environment, the call being analyzed to determine actions to be performed in the data environment. A monitoring component of the control plane also can periodically communicate with the data environment to determine any necessary actions to be performed, such as to recover from faults or events in the data environment. A workflow can be instantiated that includes tasks necessary to perform the action. For each task, state information can be passed to a component in the data environment operable to perform the task, until all tasks for an action are completed. Data in the data environment can be accessed directly using an externally-facing interface of the data environment, without accessing the control plane.

Claims

1. A computer-implemented method of controlling at least a configuration aspect of a relational database using a control environment, comprising:

under control of one or more computer systems configured with executable instructions,

receiving a Web services call from a customer to the control environment through a control interface;

analyzing the Web services call to determine an action to be performed with respect to the configuration aspect of at least one data instance of the relational database, the configuration aspect relating to at least one of creating, deleting, backing up, recovery, scaling, replicating, provisioning, upgrading, patch management, failover, or capacity management of the at least one data instance, the at least one data instance residing in a data environment that is separate from the control environment;

storing information for the action to a job queue;

extracting the information for the action from the job queue and generating a workflow based at least in part upon the information, the workflow comprising a series of tasks to be performed in the data environment with respect to the action;

for each task of the workflow, sending state information for the task to a host manager for a host machine in the data environment operable to perform the task and return a response regarding performance of the task, a successful completion of each non-final task resulting in sending state information for a subsequent task of the workflow to the host manager; and

when the response received from the host machine indicates that a final task of the workflow has been completed successfully, sending a notification to the customer that the action was successfully performed,

wherein configuration of the at least one data instance by the customer is performed using the control interface of the control environment, and the customer is able to directly access the at least one data instance after the action is performed using a data interface of the data environment without accessing the control environment.

2. The computer-implemented method according to claim 1, further comprising:

monitoring the job queue for information for additional actions to be performed with respect to the data environment.

3. The computer-implemented method according to claim 1, further comprising:

providing the customer with a DNS (domain name system) address and port information enabling the customer to access the relational database using the data interface.

4. A computer-implemented method of controlling at least a configuration aspect of a data environment using a control environment, comprising:

under control of one or more computer systems configured with executable instructions,

analyzing a request received through a control interface of the control environment to determine an action to be performed with respect to the configuration aspect of at least one data instance in the data environment, the data environment being separate from the control environment;

generating a workflow for the action, the workflow comprising at least one task to be performed in the data environment with respect to the action;

for each task of the workflow, sending a task request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when the response received from the component indicates that a final task of the workflow has been completed, sending a notification through the control interface that the action was completed,

wherein configuration of the at least one data instance by a user is performed exclusively via the control environment. and data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

5. The computer-implemented method according to claim 4, wherein:

the data is stored in at least one relational database.

6. The computer-implemented method according to claim 5, wherein:

the request is a Web services call; and

the action includes provisioning one of the at least one relational database.

7. The computer-implemented method according to claim 4, wherein:

the action includes at least one of creating, deleting, backing up, scaling, replicating, provisioning, upgrading, patch management, failover, or capacity management of the at least one data instance in the data environment.

8. The computer-implemented method according to claim 4, wherein:

the request is a Web services call received to an externally-facing application programming interface (API) of the control environment.

9. The computer-implemented method according to claim 4, wherein:

the data interface is accessible using a DNS address for the at least one data instance.

10. The computer-implemented method according to claim 4, wherein:

the at least one task is selected from a group consisting of creating the at least one data instance, setting a storage volume of the at least one data instance, downloading binaries for a selected engine, backup and recovery of the at least one data instance, data replication and failover, creating and managing a security group, managing user credentials, managing key rotation and credential management, and attaching to hardware to support the at least one data instance.

11. The computer-implemented method according to claim 4, wherein:

the request is received to a Web services layer including components operable to perform at least one second task selected from a group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

12. The computer-implemented method according to claim 4, further comprising:

assembling the workflow using tasks selected based at least in part upon a database engine used for the at least one data instance corresponding to the request.

13. The computer-implemented method according to claim 4, further comprising:

logging event information for the data environment to a data store in the control environment.

14. The computer-implemented method according to claim 4, further wherein:

the component in the data environment is a host manager for a host machine operable to perform tasks in the data environment, the host manager being further operable to collect and monitor health information for at least one aspect of the data environment.

15. A computer-implemented method of maintaining a data environment using a control environment, comprising:

under control of one or more computer systems configured with executable instructions,

provision at least one data instance in the data environment, the data environment being separate from the control environment;

periodically sending requests for status from a monitoring component of the control environment to at least one host manager in the data environment responsible for monitoring the at least one data instance;

receiving monitoring information from the at least one host manager which received one of the requests for status;

analyzing the monitoring information to determine an action to be performed with respect to the data environment and generating a workflow for the action, the action involving one or more aspects of configuring the at least one data instance and the workflow comprising at least one task to be performed in the data environment with respect to the action;

for each task of the workflow, sending state information for the task to the at least one host manager to perform the task and return a response with information regarding performance of the task, a successful completion of each non-final task resulting in sending the state information for a subsequent task of the workflow to the at least one host manager; and

when the response received from the at least one host manager indicates that a final task of the workflow has been completed, storing the information for the action to a log for the data environment,

wherein configuration of the at least one data instance by a user is performed via the control environment, and data in the data environment is accessible using a data interface without accessing the control environment.

16. The computer-implemented method according to claim 15, wherein:

the at least one data instance corresponds to a relational database in the data environment.

17. The computer-implemented method according to claim 15, further comprising:

in response to analyzing the request to determine the action to be performed, storing the information for the action to a job queue,

wherein the information for the action is able to be extracted from the job queue to generate the workflow for the action.

18. The computer-implemented method according to claim 15, further comprising:

when the response is not received within a determined period of time, sending another request for status; and

when the response is not received after a determined number of requests, determining a second action to be performed with respect to the data environment and generating a second workflow for the second action.

19. The computer-implemented method according to claim 15, wherein:

the at least one host manager is operable to collect and monitor health information for at least one aspect of the data environment for use in generating the monitoring information.

20. A system for controlling a data environment using a control environment, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the system to;

analyze a request received through a control interface of the control environment to determine an action to be performed with respect to a configuration aspect of at least one data instance in the data environment, the data environment being separate from the control environment;

generate a workflow for the action, the workflow comprising at least one task to be performed in the data environment with respect to the action;

for each task of the workflow, send a task request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when the response received from the component indicates that a final task of the workflow has been completed, send a notification through the control interface that the action was completed,

wherein configuration of the at least one data instance by a user is performed via the control environment, and data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

21. The system according to claim 20, further comprising:

an externally-facing API in the control environment operable to receive the request, wherein the request is a Web services call.

22. The system according to claim 20, wherein:

the data interface is accessible using a DNS address for one or more designated data instances that include the at least one data instance.

23. The system according to claim 20, further comprising:

a Web services layer in the control environment operable to receive the request and perform at least one second task selected from a group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

24. The system according to claim 20, further comprising:

a workflow component operable to assemble the workflow using tasks selected based at least in part upon a database engine used for the at least one data instance corresponding to the request.

25. A computer program product embedded in a non-transitory computer-readable storage medium and including instructions that, when executed by at least one computing device, cause the at least one computing device to:

analyze a request received through a control interface of a control environment to determine an action to be performed with respect to a configuration aspect of at least one data instance in a data environment, the data environment being separate from the control environment;

generate a workflow for the action, the workflow comprising at least one task to be performed in the data environment with respect to the action;

for each task of the workflow, send a task request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when the response received from the component indicates that a final task of the workflow has been completed, send a notification through the control interface that the action was completed,

wherein configuration of the at least one data instance by a user is performed via the control environment, and data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

26. The computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide an externally-facing APT in the control environment operable to receive the request.

27. The computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide a Web services layer in the control environment operable to receive the request and perform at least one second task selected from a group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

28. The computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide a workflow component operable to assemble the workflow using tasks selected based at least in part upon a database engine used for the at least one data instance corresponding to the request.