Control Service for Relational Data Management

• US 20100251242A1
• Filed: 03/31/2009
• Published: 09/30/2010
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Horsley, James, Franklin, Paul David, Sivasubramanian, Swaminathan, McAlister, Grant Alexander MacDonald, Sheth, Rajesh Sudhakar
• Granted Patent: US 8,713,060 B2
• Time in Patent Office: Days
• US Class Current: 718/100
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 12/02 Addressing or allocation; Relocation program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

Aspects of a data environment, such as the creation, provisioning, and management of data stores and instances, are managed using a separate control environment. A user can call into an externally-facing interface of the control environment, the call being analyzed to determine actions to be performed in the data environment. A monitoring component of the control plane also can periodically communicate with the data environment to determine any necessary actions to be performed, such as to recover from faults or events in the data environment. A workflow can be instantiated that includes tasks necessary to perform the action. For each task, state information can be passed to a component in the data environment operable to perform the task, until all tasks for an action are completed. Data in the data environment can be accessed directly using an externally-facing interface of the data environment, without accessing the control plane.

Claims

1. A computer-implemented method of controlling aspects of a relational database using a separate control environment, comprising:

under control of one or more computer systems configured with executable instructions,

receiving a Web services call from a customer to the control environment through a control interface;

analyzing the Web services call to determine an action to be performed with respect to a relational database and storing information for the determined action to a job queue;

extracting the information for the determined action from the job queue and generating a workflow based at least in part upon the information, the workflow comprising a series of tasks to be performed in a data environment with respect to the determined action, the relational database residing in the data environment;

for each task of the workflow, sending state information for the task to a host manager for a host machine in the data environment operable to perform the task and return a response regarding performance of the task, a successful completion of each non-final task resulting in sending state information for a subsequent task of the workflow to the workflow manager; and

when a response is received from the host machine indicating that a final task of the workflow has been completed successfully, sending a notification to the customer that the action was successfully performed,

wherein the customer is able to directly access the relational database after the action is performed using a data interface of the data environment without accessing the control environment.

2. A computer-implemented method according to claim 1, further comprising:

monitoring the job queue for information for any action to be performed with respect to the data environment.

3. A computer-implemented method according to claim 1, further comprising:

providing the customer with a DNS (domain name system) address and port information enabling the customer to access the relational database using the data interface.

4. A computer-implemented method of controlling aspects of a data environment using a separate control environment, comprising:

under control of one or more computer systems configured with executable instructions,

analyzing a request received through a control interface of the control environment to determine an action to be performed with respect to the data environment;

generating a workflow for the determined action, the workflow comprising at least one task to be performed in the data environment with respect to the determined action;

for each task of the workflow, sending a request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when a response is received from the host machine indicating that a final task of the workflow has been completed, sending a notification through the control interface that the requested action was completed,

wherein data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

5. A computer-implemented method according to claim 4, wherein:

the data is stored in at least one relational database.

6. A computer-implemented method according to claim 5, wherein:

the request is a Web services call; and

the action includes provisioning one of the at least one relational database.

7. A computer-implemented method according to claim 4, wherein:

the action includes at least one of creating, deleting, backing up, scaling, replicating, provisioning, or modifying a data instance in the data environment.

8. A computer-implemented method according to claim 4, wherein:

the request is a Web services call received to an externally-facing application programming interface (API) of the control environment.

9. A computer-implemented method according to claim 4, wherein:

the data interface is an externally-facing API of the control environment accessible using a DNS address for a designated data instance.

10. A computer-implemented method according to claim 4, wherein:

at least one of the tasks is selected from the group consisting of creating a data instance, setting a storage volume of a data instance, modifying a data instance, downloading binaries for a selected engine, backup and recovery of a data instance, data replication and failover, creating and managing a security group, managing user credentials, managing key rotation and credential management, and attaching to hardware to support the data instance.

11. A computer-implemented method according to claim 4, wherein:

the request is received to a Web services layer including components operable to perform at least one task selected from the group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

12. A computer-implemented method according to claim 4, further comprising:

assembling the workflow using tasks selected based at least in part upon a database engine used for the data component corresponding to the request.

13. A computer-implemented method according to claim 4, further comprising:

logging event information for the data environment to a data store in the control environment.

14. A computer-implemented method according to claim 4, further wherein:

the component in the data environment is a host manager for a host machine operable to perform the tasks in the data environment, the host manager being further operable to collect and monitor health information for at least one aspect of the data environment.

15. A computer-implemented method of maintaining a data environment using a separate control environment, comprising:

under control of one or more computer systems configured with executable instructions,

periodically sending requests for status from a monitoring component of the control environment to at least one host manager in the data environment responsible for monitoring a data instance;

receiving a response from at least one host manager which received one of the requests for status;

analyzing the response to determine any action to be performed with respect to the data environment and generating a workflow for the determined action, the workflow comprising at least one task to be performed in the data environment with respect to the determined action;

for each task of the workflow, sending state information to the host manager to perform the task and return a response with information regarding performance of the task, a successful completion of each non-final task resulting in sending state information for a subsequent task of the workflow to the workflow manager; and

when a response is received from the host machine indicating that a final task of the workflow has been completed, storing information for the performed action to a log for the data environment,

wherein data in the data environment is accessible using a data interface without accessing the control environment.

16. A computer-implemented method according to claim 15, wherein:

the data instance corresponds to a relational database in the data environment.

17. A computer-implemented method according to claim 15, further comprising:

in response to analyzing the request to determine an action to be performed, storing information for the determined action to a job queue,

wherein the information is able to be extracted from the job queue to generate the workflow.

18. A computer-implemented method according to claim 15, further comprising:

when a response from a host manager is not received within a determined period of time, sending another request for status; and

when a response from a host manager is not received after a determined number of requests, determining an action to be performed with respect to the data environment and generating a workflow for the determined action.

19. A computer-implemented method according to claim 15, wherein:

the host manager is operable to collect and monitor health information for at least one aspect of the data environment for use in generating the response to the request for status.

20. A system for controlling a data environment using a separate control environment, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the system to;

analyze a request received through a control interface of the control environment to determine an action to be performed with respect to the data environment;

generate a workflow for the determined action, the workflow comprising at least one task to be performed in the data environment with respect to the determined action;

for each task of the workflow, send a request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when a response is received from the host machine indicating that a final task of the workflow has been completed, send a notification through the control interface that the requested action was completed,

wherein data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

21. A system according to claim 20, further comprising:

an externally-facing API in the control environment operable to receive the request, wherein the request is a Web services call.

22. A system according to claim 20, wherein:

the data interface is an externally-facing API of the control environment accessible using a DNS address for a designated data instance.

23. A system according to claim 20, further comprising:

a Web services layer in the control environment operable to receive the request and perform at least one task selected from the group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

24. A system according to claim 20, further comprising:

a workflow component operable to assemble the workflow using tasks selected based at least in part upon a database engine used for the data component corresponding to the request.

25. A computer program product embedded in a computer-readable medium and including instructions that, when executed by at least one computing device, cause the at least one computing device to:

analyze a request received through a control interface of the control environment to determine an action to be performed with respect to the data environment;

generate a workflow for the determined action, the workflow comprising at least one task to be performed in the data environment with respect to the determined action;

for each task of the workflow, send a request to a component in the data environment operable to perform the task and return a response regarding performance of the task; and

when a response is received from the host machine indicating that a final task of the workflow has been completed, send a notification through the control interface that the requested action was completed,

wherein data in the data environment is accessible using a data interface of the data environment without accessing the control environment.

26. A computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide an externally-facing API in the control environment operable to receive the request.

27. A computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide a Web services layer in the control environment operable to receive the request and perform at least one task selected from the group consisting of authenticating users based on credentials, authorizing users, throttling user requests, user input validation, and marshalling or unmarshalling requests and responses.

28. A computer program product according to claim 25, further including instructions that, when executed by at least one computing device, cause the at least one computing device to:

provide a workflow component operable to assemble the workflow using tasks selected based at least in part upon a database engine used for the data component corresponding to the request.