LIFECYCLE SUPPORT FOR STORAGE OBJECTS

• US 20170255589A1
• Filed: 05/22/2017
• Published: 09/07/2017
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barber, Jeffrey Michael, Gattu, Praveen Kumar, Denny-Brown, II, Derek Ernest, Elving, Christopher Henning, Perry, Carl Yates
• Granted Patent: US 10,853,337 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2474 Sequence data queries, e.g. querying versioned data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Methods and apparatus for lifecycle support for storage objects are disclosed. A storage medium stores program instructions that when executed on a processor implement a storage lifecycle manager of a multi-tenant storage service. The lifecycle manager determines a lifecycle policy to be implemented for a storage object. The policy indicates lifecycle transitions of the object, including a transition from a first service level to a second service level. The first service level and the second service level may each specify a maximum number of versions of the storage object. The lifecycle manager initiates modification operations corresponding to the storage object'"'"'s data in accordance with the lifecycle policy, and verifies that the modification operations succeeded. The lifecycle manager provides, in response to a query, an indication of a current service level of the storage object.

Claims

1-25. -25. (canceled)

26. A method, comprising:

determining a lifecycle policy for a storage object of a storage service, wherein the lifecycle policy comprises an indication of a lifecycle transition of the storage object, wherein the lifecycle transition comprises a transition from a first service level to a second service level, wherein the first service level and the second service level each, respectively, comprise at least one of;

a durability level,

an availability level,

a performance level indicating expected latencies for storage operations, or

any combination thereof;

initiating a modification operation associated with the storage object in accordance with the lifecycle policy;

verifying that the modification operation has succeeded; and

providing, in response to a query, an indication of a current service level of the storage object.

27. The method of claim 26, wherein the durability level indicates a probability of permanent data loss, and wherein the availability level indicates an amount of time which the storage object is expected to be inaccessible over a particular time period.

28. The method of claim 26, wherein the first service level and the second service level each further comprise a replication level, the method further comprising:

storing a first number of replicas of at least a portion of data of the storage object based on a first replication count corresponding to the replication level of the first service level; and

storing a second number of replicas of at least a portion of data of the storage object based on a second replication count corresponding to the replication level of the second durability level, wherein the first number and the second number are different.

29. The method of claim 26, further comprising:

storing at least a portion of data of the storage object a first set of erasure in accordance with a first set of coding parameters corresponding to the durability level of the first service level; and

storing the at least a portion of data of the storage object in accordance with a second set of erasure coding parameters corresponding to the durability level of the second service level.

30. The method of claim 26, wherein the performance level of the first service level further indicates an expected transition time for transitioning to the second service level.

31. The method of claim 26, wherein the replicas of the storage object comprise physical replicas of the storage object.

32. The method of claim 26, wherein the replicas of the storage object comprise logical replicas of the storage object.

33. A system, comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to;

determine a lifecycle policy for a storage object of a storage service, wherein the lifecycle policy comprises an indication of a lifecycle transition of the storage object, wherein the lifecycle transition comprises a transition from a first service level to a second service level, wherein the first service level and the second service level each, respectively, comprise at least one of;

a durability level,

an availability level,

a performance level indicating expected latencies for storage operations, or

any combination thereof;

initiate a modification operation associated with the storage object in accordance with the lifecycle policy;

verify that the modification operation has succeeded; and

provide, in response to a query, an indication of a current service level of the storage object.

34. The system of claim 33, wherein the instructions further cause the one or more processors to:

determine a data placement plan for at least a portion of data of the storage object in accordance with the lifecycle transition.

35. The system of claim 34, wherein the instructions further cause the one or more processors to:

determine the modification operation in accordance with the data placement plan.

36. The system of claim 33, wherein the instructions further cause the one or more processors to:

store a first number of replicas of at least a portion of data of the storage object based on a first replication count corresponding to the durability level of the first service level; and

store a second number of replicas of at least a portion of data of the storage object based on a second replication count corresponding to the durability level of the second service level.

37. The system of claim 33, wherein a transition criterion corresponding to the lifecycle transition is based at least in part on one or more of:

(a) an elapsed time since the storage object was created (b) an elapsed time since the storage object was last modified (c) an elapsed time since the storage object was last read (d) a frequency of access of the storage object, or (e) a cost of retaining the storage object in the first service level.

38. The system of claim 33, wherein the instructions further cause the one or more processors to:

generate, in response to a recommendation request indicating a client goal and a set of candidate storage objects, one or more lifecycle transition recommendations based at least in part on (a) the client goal and (b) the usage metrics associated with one or more candidate storage objects of the set.

39. A non-transitory, computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to:

determine a lifecycle policy for a storage object of a storage service, wherein the lifecycle policy comprises an indication of a lifecycle transition of the storage object, wherein the lifecycle transition comprises a transition from a first service level to a second service level, wherein the first service level and the second service level each, respectively, comprise at least one of;

a durability level indicating a probability of permanent data loss of the storage object,

an availability level indicating an amount of time which the storage object is expected to be inaccessible over a particular time period,

a performance level indicating expected latencies for storage operations,

a replication level indicating a number of replicas of the storage object to be stored, or

any combination thereof;

initiate a modification operation associated with the storage object in accordance with the lifecycle policy;

verify that the modification operation has succeeded; and

provide, in response to a query, an indication of a current service level of the storage object.

40. The non-transitory, computer-readable storage medium of claim 39, wherein the instructions further cause the one or more processors to:

store a first number of replicas of at least a portion of data of the storage object based on a first replication count corresponding to the replication level of the first service level; and

store a second number of replicas of at least a portion of data of the storage object based on a second replication count corresponding to the replication level of the second service level.

41. The non-transitory, computer-readable storage medium of claim 39, wherein the performance level of the first service level further indicates response times for the storage operations.

42. The non-transitory, computer-readable storage medium of claim 39, wherein the instructions further cause the one or more processors to:

store at least a portion of data of the storage object a first set of erasure in accordance with a first set of coding parameters corresponding to the first service level; and

store the at least a portion of data of the storage object in accordance with a second set of erasure coding parameters corresponding to the second service level.

43. The non-transitory, computer-readable storage medium of claim 39, wherein the instructions further cause the one or more processors to:

generate, in response to a recommendation request indicating a client goal and a set of candidate storage objects, one or more lifecycle transition recommendations based at least in part on (a) the client goal and (b) the usage metrics associated with one or more candidate storage objects of the set.

44. The non-transitory, computer-readable storage medium of claim 39, wherein the instructions further cause the one or more processors to:

divide data of the storage object into a plurality of fragments in accordance with the lifecycle transition; and

transform the plurality of fragments into a second plurality of fragments having a number of fragments different from the plurality of fragments.

45. The non-transitory, computer-readable storage medium of claim 44, wherein the instructions further cause the one or more processors to:

regenerate the data of the storage object based on a subset of the second plurality of fragments.