Lifecycle support for storage objects having multiple durability levels specifying different numbers of versions

• US 9,658,983 B1
• Filed: 12/14/2012
• Issued: 05/23/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barber, Jeffrey Michael, Gattu, Praveen Kumar, Denny-Brown, II, Derek Ernest, Elving, Christopher Henning, Perry, Carl Yates
• Primary Examiner(s): Barry, Lance L
• Application Number:
  US13/715,863
• Time in Patent Office: 1,621 Days
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2474 Sequence data queries, e.g. querying versioned data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Methods and apparatus for lifecycle support for storage objects are disclosed. A storage medium stores program instructions that when executed on a processor implement a storage lifecycle manager of a multi-tenant storage service. The lifecycle manager determines a lifecycle policy to be implemented for a storage object. The policy indicates lifecycle transitions of the object, including a transition from a first service level to a second service level. The first service level and the second service level may each specify a maximum number of versions of the storage object. The lifecycle manager initiates modification operations corresponding to the storage object'"'"'s data in accordance with the lifecycle policy, and verifies that the modification operations succeeded. The lifecycle manager provides, in response to a query, an indication of a current service level of the storage object.

Claims

1. A system, comprising

a processor;

and

a memory storing instructions that, when executed by the processor, cause the processor to perform operations comprising;

implementing a network-accessible multi-tenant storage service for storage objects owned by a plurality of clients;

receiving an indication of a lifecycle policy to be implemented for a collection of one or more storage objects, wherein the lifecycle policy comprises a representation of one or more lifecycle transitions of a particular storage object of the one or more storage objects and one or more transition criteria corresponding to each lifecycle transition of the one or more lifecycle transitions, wherein a particular lifecycle transition of the one or more lifecycle transitions comprises a transition from a first online service level with a first supported durability level to a different online service level with a second supported durability level, wherein the first supported durability level specifies a first maximum number of versions of the particular storage object, and wherein the second supported durability level specifies a second maximum number of versions of the particular storage object;

determining, based at least in part on the lifecycle policy, (a) a data placement plan for at least a portion of data of the particular storage object in accordance with the particular lifecycle transition and (b) one or more modification operations to be initiated in accordance with the data placement plan;

initiating the one or more modification operations;

verifying that the one or more modification operations have succeeded; and

providing, in response to a query, an indication of a current service level of the particular storage object.

2. The system as recited in claim 1, wherein, in accordance with the first online service level, at least a particular number of replicas of the data of the particular storage object are stored at a first set of storage devices of the storage service, and wherein, in accordance with the different online service level, at least a different number of replicas of the data of the particular storage object are stored at a second set of storage devices of the storage service.

3. The system as recited in claim 1, wherein a particular transition criterion corresponding to the particular lifecycle transition is based at least in part on one or more of:

(a) an elapsed time since the particular storage object was created (b) an elapsed time since the particular storage object was last modified (c) an elapsed time since the particular storage object was last read (d) a frequency of access of the particular storage object, or (e) a cost of retaining the particular storage object in the first service level.

4. The system as recited in claim 1, wherein the lifecycle policy comprises an indication of a second lifecycle transition of a second storage object comprising a transition from a particular online service level to a delayed-access service level, wherein the operations further comprise:

initiating one or more additional operations to copy data of the second storage object from a storage device supporting the particular online service level to a different storage device supporting the delayed-access service level; and

in response to an access request from a client for data of the second storage object, wherein the access request is received after the one or more additional operation have been completed,

scheduling an operation to bring the data of the second storage object online; and

providing a notification to the client, after the operation to bring the data of the second storage object online has been completed, indicating that the data of the second storage object is accessible.

5. The system as recited in claim 1, wherein another lifecycle transition of the one or more lifecycle transitions comprises a deletion of the particular storage object from the storage service.

6. A method, comprising:

implementing a network-accessible multi-tenant storage service for storage objects owned by a plurality of clients;

receiving an indication of a lifecycle policy to be implemented for a collection of one or more storage objects, wherein the lifecycle policy comprises an indication of one or more lifecycle transitions of a particular storage object, wherein a particular lifecycle transition of the one or more lifecycle transitions comprises a transition from a first supported service level to a second supported service level, wherein the first supported service level specifies a first maximum number of versions of the particular storage object, and wherein the second supported service level specifies a second maximum number of versions of the particular storage object;

initiating one or more modification operations associated with the particular storage object in accordance with the lifecycle policy;

verifying that the one or more modification operations have succeeded; and

providing, in response to a query, an indication of a current service level of the particular storage object.

7. The method as recited in claim 6, wherein the first supported service level specifies a first durability level, and the second supported service level specifies a second durability level.

8. The method as recited in claim 7, further comprising:

storing a first number of replicas of at least a portion of data of the particular storage object in accordance with the first durability level; and

storing a different number of replicas of the at least a portion of the data in accordance with the second durability level.

9. The method as recited in claim 7, further comprising:

utilizing a first set of erasure coding parameters to store at least a portion of data of the particular storage object in accordance with the first durability level; and

utilizing a second set of erasure coding parameters to store the at least a portion of data of the particular storage object in accordance with the second durability level.

10. The method as recited in claim 6, wherein the first supported service level specifies a first availability level, and the second supported service level specifies a different availability level.

11. The method as recited in claim 6, wherein the first supported service level specifies a first performance level for data access, and the second supported service level specifies a different performance level for data access.

12. The method as recited in claim 6, wherein the first maximum number of versions is different from the second maximum number of versions.

13. The method as recited in claim 6, wherein the first supported service level specifies a first storage pricing policy, and the second supported service level specifies a different storage pricing policy.

14. The method as recited in claim 6, wherein the lifecycle policy comprises an indication of one or more transition criteria to be used to determine whether to move the particular storage object from the first service level to the second service level.

15. The method as recited in claim 14, wherein the lifecycle policy comprises an indication of one or more transition criteria to be used to determine whether to move the particular storage object from the second service level back to the first service level.

16. The method as recited in claim 14, wherein a particular transition criterion of the one or more transition criteria is based at least in part on one or more of:

(a) an elapsed time since the particular storage object was created (b) an elapsed time since the particular storage object was last modified (c) an elapsed time since the particular storage object was last read (d) a frequency of access of the particular storage object, or (e) a cost of retaining the particular storage object in the first service level.

17. The method as recited in claim 6, further comprising:

generating, in response to a recommendation request from a client, wherein the request indicates at least one cost goal and a set of candidate storage objects, one or more lifecycle transition recommendations based at least in part on (a) the at least one cost goal and (b) usage metrics associated with one or more candidate storage objects of the set.

18. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors implement a storage lifecycle manager configured to

determine a lifecycle policy to be implemented for one or more storage objects of a multi-tenant storage service, wherein the lifecycle policy comprises an indication of one or more lifecycle transitions of a particular storage object, wherein a particular lifecycle transition of the one or more lifecycle transitions comprises a transition from a first supported service level to a second supported service level, wherein the first supported service level specifies a first maximum number of versions of the particular storage object, and wherein the second supported service level specifies a second maximum number of versions of the particular storage object;

initiate one or more modification operations associated with the particular storage object in accordance with the lifecycle policy;

verify that the one or more modification operations have succeeded; and

provide, in response to a query, an indication of a current service level of the particular storage object.

19. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the storage lifecycle manager comprises one or more of:

(a) an interface manager configured to implement a programmatic interface enabling a client to specify the lifecycle policy (b) a transition job generator configured to identify, during a candidate discovery iteration, a collection of storage objects for which lifecycle transitions are to be implemented and (c) a transition task dispatcher configured to assign resources to implement modification operations in accordance with the lifecycle policy.

20. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the first supported service level specifies a first durability level, and the second supported service level specifies a second durability level.

21. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the lifecycle manager is configured to:

utilize a first set of erasure coding parameters to store at least a portion of data of the particular storage object in accordance with the first service level; and

utilize a second set of erasure coding parameters to store the at least a portion of data of the particular storage object in accordance with the second service level.

22. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the first supported service level specifies a first performance level for data access, and the second supported service level specifies a different performance level for data access.

23. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the first supported service level specifies a first storage pricing policy, and the second supported service level specifies a different storage pricing policy.

24. The non-transitory computer-accessible storage medium as recited in claim 18, wherein a particular transition criterion indicated in the lifecycle policy and used to determine whether to move the particular storage object from the first service level to the second service level is based at least in part on one or more of:

(a) an elapsed time since the particular storage object was created (b) an elapsed time since the particular storage object was last modified (c) an elapsed time since the particular storage object was last read (d) a frequency of access of the particular storage object, or (e) a cost of retaining the particular storage object in the first service level.

25. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the lifecycle manager is further configured to:

generate, in response to a recommendation request from a client, wherein the request indicates a client goal and a set of candidate storage objects, one or more lifecycle transition recommendations based at least in part on (a) the client goal and (b) the usage metrics associated with one or more candidate storage objects of the set.