Cloning and recovery of data volumes

• US 10,162,715 B1
• Filed: 12/21/2015
• Issued: 12/25/2018
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: MacDonald McAlister, Grant Alexander, Milovanovic, Milovan
• Primary Examiner(s): Lu, Hua
• Application Number:
  US14/977,389
• Time in Patent Office: 1,100 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet

Abstract

Aspects of a data environment, such as the cloning, hibernation, and recovery of databases, are managed using a separate control environment. A monitoring component of the control environment can periodically communicate with the data environment to determine any necessary actions to be performed, such as to recover from faults or events for a data instance in the data environment. A workflow can be instantiated that includes tasks necessary to perform actions such as recovery, hibernation, resumption from hibernation, or backup or cloning. Tasks of the workflow can cause certain jobs to be performed by host managers in the data environment to affect calls made to the control environment.

Claims

1. A system, comprising:

one or more compute nodes, respectively comprising at least one processor and a memory, configured to implement a control plane for a separately hosted data store;

the control plane, configured to;

identify an operation to create a new replica of the data store at a point in time;

determine a snapshot from among a plurality of snapshots captured at different respective points in time of the data store according to the point in time for the identified operation; and

direct the operation to create the new replica of the data store at the point in time, wherein at least the operation;

provisions the new replica of the data store; and

copies the determined snapshot to the new replica of the data store.

2. The system of claim 1, wherein to identify the operation to create the new replica of the data store at the point in time, the control plane is configured to analyze status information for the data store to detect a failure of the data store at the point in time and wherein the identified operation is a recovery operation for the data store.

3. The system of claim 1, wherein to identify the operation to create the new replica of the data store at the point in time, the control plane is configured to receive a request to clone the data store at the point in time and wherein the identified operation is a clone operation.

4. The system of claim 3, wherein the new replica of the data store is created as part of a repartition of the data store and wherein at least the operation further deletes a portion of the snapshot of the new replica so that a remaining portion of the snapshot corresponds to a partition of the data store assigned to the new replica of the data store.

5. The system of claim 3, wherein the data store is available for access during the creation of the new replica of the data store.

6. The system of claim 1, wherein the data store is a hibernated data store that is unavailable for access, wherein to identify the operation to create the new replica of the data store at the point in time, the control plane is configured to receive a request to resume access to the data store, wherein the identified operation is a resume operation, and wherein the point in time is a time at which a hibernate request was received for the data store.

7. The system of claim 1,

wherein the control plane is further configured to identify one or more log records from a plurality of log records maintained as part of a log that describe respective changes to the data store to create the new replica of the data store at the point in time;

and

wherein at least the operation further applies the identified one or more log records to the snapshot copied to the new replica of the data store.

8. A method, comprising:

performing, by one or more computing devices;

identifying, by a control plane for a separately hosted data store, an operation to create a new replica of the data store at a point in time;

determining, by the control plane, a snapshot from among a plurality of snapshots captured at different respective points in time of the data store according to the point in time for the identified operation; and

directing, by the control plane, the operation to create the new replica of the data store at the point in time, wherein the operation comprises;

provisioning the new replica of the data store; and

copying the determined snapshot to the new replica of the data store.

9. The method of claim 8, wherein identifying the operation to create the new replica of the data store at the point in time comprises analyzing status information for the data store to detect a failure of the data store at the point in time and wherein the identified operation is a recovery operation for the data store.

10. The method of claim 8, wherein identifying the operation to create the new replica of the data store at the point in time comprises receiving a request to clone the data store at the point in time and wherein the identified operation is a clone operation.

11. The method of claim 10, wherein the new replica of the data store is created as part of a repartition of the data store and wherein the operation further comprises deleting a portion of the snapshot of the new replica so that a remaining portion of the snapshot corresponds to a partition of the data store assigned to the new replica of the data store.

12. The method of claim 10, wherein the data store is available for access during the creation of the new replica of the data store.

13. The method of claim 8, wherein the data store is a hibernated data store that is unavailable for access, wherein identifying the operation to create the new replica of the data store at the point in time comprises receiving a request to resume access to the data store, wherein the identified operation is a resume operation, and wherein the point in time is a time at which a hibernate request was received for the data store.

14. The method of claim 8,

wherein the method further comprises identifying one or more log records from a plurality of log records maintained as part of a log that describe respective changes to the data store to create the new replica of the data store at the point in time;

and

wherein the operation further comprises applying the identified one or more log records to the snapshot copied to the new replica of the data store.

15. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices causes the one or more computing devices to implement:

identifying, by a control plane for a separately hosted data store, an operation to create a new replica of the data store at a point in time;

determining, by the control plane, a snapshot from among a plurality of snapshots captured at different respective points in time of the data store according to the point in time for the identified operation; and

directing, by the control plane, the operation to create the new replica of the data store at the point in time, wherein the operation comprises;

provisioning the new replica of the data store; and

copying the determined snapshot to the new replica of the data store.

16. The non-transitory medium of claim 15, wherein, in identifying the operation to create the new replica of the data store at the point in time, the program instructions cause the one or more computing devices to implement analyzing status information for the data store to detect a failure of the data store at the point in time and wherein the identified operation is a recovery operation for the data store.

17. The non-transitory medium of claim 15, wherein, in identifying the operation to create the new replica of the data store at the point in time, the program instructions cause the one or more computing devices to implement receiving a request to clone the data store at the point in time and wherein the identified operation is a clone operation.

18. The non-transitory medium of claim 17, wherein the new replica of the data store is created as part of a repartition of the data store and wherein the operation further comprises deleting a portion of the snapshot of the new replica so that a remaining portion of the snapshot corresponds to a partition of the data store assigned to the new replica of the data store.

19. The non-transitory medium of claim 17, wherein the data store is available for access during the creation of the new replica of the data store.

20. The non-transitory medium of claim 15, wherein the data store is a hibernated data store that is unavailable for access, wherein, in identifying the operation to create the new replica of the data store at the point in time, the program instructions cause the one or more computing devices to implement receiving a request to resume access to the data store, wherein the identified operation is a resume operation, and wherein the point in time is a time at which a hibernate request was received for the data store.