Moving data between data stores

• US 8,621,161 B1
• Filed: 09/23/2010
• Issued: 12/31/2013
• Est. Priority Date: 09/23/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Briggs, Gregory J.
• Primary Examiner(s): Gu, Shawn X
• Application Number:
  US12/888,731
• Time in Patent Office: 1,195 Days
• Field of Search: None
• US Class Current: 711/152
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/825 the problem or solution involving locking

Abstract

A data object may be moved from a source data store to a destination data store via replication. The replication is initiated when an original data object in a source data store that is capable of being both read from and being written to is read. Following the read, the original data object is then duplicated to a destination data store. The duplicate data object is provided with a state that indicates the duplicate object is duplicated from the source data store and can be read but cannot accept a data write. Subsequently, the state of the original data object is changed to can be read but cannot be written to using optimistic locking. Further, the state of the duplicate data object is also modified to being capable of both read from and written to with the use of optimistic locking. The replication is completed with the deletion of the original data object from the source data store.

Claims

1. A computer-implemented method, comprising:

under control of one or more computing systems configured with executable instructions,

reading an original data object with a state of “

unlocked”

that is stored in a source data store to obtain substantive data, a version number, and an identifier of the original data object;

storing a duplicate data object of the original data object to a destination data store with a state of “

locked from the source data store”

when no other data object with the identifier preexists on the destination data store, the duplicate data object being provided with an additional version number;

changing the state of the original data object to “

locked to the destination data store”

when the version number of the original data object is unchanged just prior to the changing;

modifying the state of the duplicate data object to “

unlocked”

when the additional version number of the duplicate data object is unchanged just prior to the modifying; and

deleting the original data object from the source data store.

2. The computer-implemented method of claim 1, wherein the source data store and the destination data store are located in different tiers of a tiered data storage system.

3. A computer-implemented method, comprising:

under control of one or more computing systems configured with executable instructions,

reading an original data object that is stored in a source data store having a state indicating that the original data object is able to be read from and written to;

storing, based at least in part on the reading of the original data object, a duplicate data object of the original data object to a destination data store with a state indicating that the duplicate data object is able to be read from but not written to, the state of the duplicate data object further indicating that the duplicate data object is a duplicate from the source data store;

changing, using optimistic locking, the state of the original data object to indicate that the original data object is able to be read from but not written to and to indicate that the original data object is being duplicated to the destination data store;

modifying, using optimistic locking, the state of the duplicate data object to indicate that the duplicate data object is able to be read from and written to; and

deleting the original data object from the source data store.

4. The computer-implemented method of claim 3, wherein the source data store and the destination data store are located in different tiers of a tiered data storage system.

5. The computer-implemented method of claim 3, wherein the original data object includes an identifier, and the storing includes storing the duplicate data object to the destination data store when no other data object with the identifier exists on the destination data store.

6. The computer-implemented method of claim 3, wherein the reading includes reading a version number of the original data object for a first time, and wherein the changing includes reading the version number of the original data object for a second time and changing the state of the original data object when the version number of the original data object at the first time is identical to the version number of the original data object at the second time.

7. The computer-implemented method of claim 6, wherein the version number of the original data object is an incrementally updated value, a globally unique identifier (GUID), a current time, or a Lamport time.

8. The computer-implemented method of claim 3, wherein the storing includes reading a version number of the duplicate data object for a first time, and wherein the modifying includes reading the version number of the duplicate data object for a second time and modifying the state of the duplicate data object when a version time of the duplicate data object at the first time is identical to the version number of the duplicate data object at the second time.

9. The computer-implemented method of claim 8, wherein the version number of the duplicate data object is an incrementally updated value, a globally unique identifier (GUID), a current time, or a Lamport time.

10. The computer-implemented method of claim 3, further comprising repeating performance of the reading and the storing when the changing of the state of the original data object fails due to the optimistic locking.

11. The computer-implemented method of claim 10, further comprising deleting the duplicate data object from the destination data store prior to repeating the performance of the reading and the storing.

12. The computer-implemented method of claim 10, wherein the repeating includes performing the reading and the storing for a second time using exponential back off.

13. The computer-implemented method of claim 3, further comprising altering the state of the original data object from indicating that the original data object is able to be read from but not written to and from indicating that the original data object is being duplicated from another data store and to indicating that the original data object is able to be read from and written to prior to the reading.

14. The computer-implemented method of claim 13, wherein the altering includes altering the state of the original data object when the state does not change in a predetermined time period.

15. The computer-implemented method of claim 3, further comprising providing data from the original data object or the duplicate data object in response to a requesting application when the states of both data objects indicate that the respective data object is able to be read from but not written to.

16. The computer-implemented method of claim 3, further comprising:

receiving a request from an application to read data from the original data object in the source data store; and

providing data from the duplicate data object in the destination data store to the application when the state of the original data object indicates that the original data object is able to be read from but not written to and that the original data object is being duplicated to a the destination data object.

17. The computer-implemented method of claim 3, further comprising:

receiving a request from an application to read data from the duplicate data object in the destination data store; and

providing data from the original data object in the source data store to the application when the state of the duplicate data object indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is a duplicate from the source data store.

18. One or more non-transitory computer readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

determining a state of an original data object that is stored in a source data store at a start of a replication of the original data object to a destination data store; and

conducting the replication when (1) the state of the original data object indicates that the original data object is able to be read from and written to, and (2) a duplicate data object of the original data object does not exist on the destination data store at the start, the replication comprising;

storing the duplicate data object of the original data object to the destination data store with a state that indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is being duplicated from the source data store;

changing the state of the original data object to indicate that the original data object is being duplicated to the destination data store using optimistic locking;

modifying the state of the duplicate data object to indicate that the duplicate data object is able to be read from and written to using optimistic locking; and

deleting the original data object from the source data store.

19. The one or more non-transitory computer readable media of claim 18, further comprising instructions that, when executed, cause the one or more processors to perform an act of terminating the replication when (1) the state of the original data object indicates that the original data object is able to be read from but not written to and indicates that the original data object is being duplicated from the destination data store to the source data store, and (2) the duplicate data object does not exist on the destination data store at the start.

20. The one or more non-transitory computer readable media of claim 18, further comprising instructions that, when executed, cause the one or more processors to perform an act of terminating the replication when (1) the state of the original data object indicates that the original data object is able to be read from but not written to and indicates that the original data object is being duplicated to another data store, and (2) the duplicate data object does not exist on the destination data store at the start.

21. The one or more non-transitory computer readable media of claim 18, further comprising instructions that, when executed, cause the one or more processors to perform an act of changing the state of the original data object to indicate that the original data object is able to be read from and written to prior to the conducting when (1) the state of the original data object indicates that the original data object is able to be read from but not written to and that the original data object is being duplicated from another data store, and (2) the duplicate data object does not exist on the destination data store at the start.

22. The one or more non-transitory computer readable media of claim 18, wherein:

the determining includes determining that the state of the original data object indicates that the original data object is able to be read from but not written to and that the original data object is being duplicated to the destination data store;

a state of the duplicate data object indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is being duplicated to another data store at the start; and

further comprising instructions that, when executed, cause the one or more processors to perform an act of terminating the replication by deleting the original data object from the source data store without storing the duplicate data object to destination data store.

23. The one or more non-transitory computer readable media of claim 18, further comprising instructions that, when executed, cause the one or more processors to perform an act of terminating the replication when the state of the original data object has switched from indicating that the original data object is able to be read from but not written and that the original data object is being replicated to the destination data store to indicating that the original data object is able to be read from and written to prior to the start, and the duplicate data object does not exist on the destination data store at the start.

24. The one or more non-transitory computer readable media of claim 18, wherein the determining includes determining that (1) the state of the original data object indicates that the original data object is able to be read from but not written to and that the original data object is being duplicated to the destination data store and (2) a state of the duplicate data object indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is being duplicated to the source data store at the start;

and

further comprising instructions that, when executed, cause the one or more processors to perform acts of;

converting the state of the duplicate data object to indicate that the original data object is able to be read from but not written to and that the duplicate data object is being duplicated from the source data store; and

completing the replication by further changing the state of the duplicate data object to indicate that the duplicate data is able to read from and written to and deleting the original data object from the source data store using optimistic locking.

25. The one or more non-transitory computer readable media of claim 18, wherein the determining includes determining that (1) the state of the original data object indicates that the original data object is able to be read from but not written to and that the original data object is being duplicated to the destination data store, and (2) a state of the duplicate data object indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is being duplicated to the source data store at the start;

and

further comprising instructions that, when executed, cause the one or more processors to perform acts of;

converting the state of the original data object to indicate that the original data object is able to be read from but not written to and that the original data object is being duplicated from the destination data store; and

completing the replication by further changing the state of the original data object to indicate that the original data object is able to be read from and written to and deleting the duplicate data object from the destination data store using optimistic locking.

26. The one or more non-transitory computer readable media of claim 18, wherein the replication is initiated based at least in part on a request from an application or an archival schedule.

27. A system, comprising:

one or more processors; and

a first data store that stores an original data object that includes substantive data and a state field having a locking indicator that indicates accessibility status, a data store pointer, and a duplication indicator that indicates a direction of duplication;

a second data store; and

memory storing components executable by the one or more processors, the components comprising;

a data relocation component to initiate a move of the original data object to the second data store by storing a duplicate data object that includes the same substantive data as the original data object on the second data store.

28. The system of claim 27, wherein the components further comprises a queue component to store a request of the move of the original data object from an application for completion by the data relocation component.

29. The system of claim 28, wherein the queue component is to store the request of the move until the data relocation component informs the queue component that the move is completed.

30. The system of claim 27, wherein the first data store and the second data store are located in different tiers, and wherein the data relocation component is to initiate the move of the original data object based at least in part on a request from an application or an archival schedule of the data relocation component.

31. The system of claim 27, wherein the state field is stored in a column that is associated with the original data object in a database table or one of a prefix or suffix data stored within the original data object.

32. The system of claim 27, wherein the duplicate data object includes an additional state field having at least one of an additional locking indicator, an additional data store pointer that points to the first data store or another data store, or an additional duplication indicator.

33. The system of claim 32, wherein the data relocation component is to store the duplicate data object to the second data store such that information in the additional state field includes a locking indicator that indicates that the duplicate data object is able to be read from but not written to and that the duplicate data object is being duplicated from the first data store.

34. The system of claim 32, wherein the duplicate data object and the additional state field are protected by optimistic locking, and wherein the data relocation component is to further (1) change the additional locking indicator from indicating that the duplication data object is able to be read from but not written to into indicating that the duplicate object is able to be read from and written to;

and (2) delete the additional data store pointer that points to the first data store and the additional duplication indicator that indicates the data object is duplicated from the first data store.

35. The system of claim 32, wherein the data relocation component is to complete the move of the original data object to the second data store by deleting the original data object from the first data store when the locking indicator indicates that the original data object is able to be read from but not written to, the duplication indicator indicates that the original data object is being duplicated to the second data store, and the additional state field of the duplicate data object includes a locking indicator that indicates that the duplicate data object is able to be read from and written to.

36. The system of claim 27, wherein the original data object further includes an identifier, and wherein the data relocation component is to store the duplicate data object on the second data store when no data object with the identifier preexists on the second data store.

37. The system of claim 27, wherein the original data object and the state field are protected by optimistic locking, and wherein the data relocation component is to further, using optimistic locking (1) change the locking indicator from indicating that the original data object is able to be written to and read from to indicating that the original data object is able to be read from but not written to;

(2) change the data store pointer to point to the second data store; and

(3) change the duplication indicator to indicate that the original data object is being duplicated to the second data store.