Optimistically consistent arbitrary data blob transactions

• US 8,510,344 B1
• Filed: 08/27/2010
• Issued: 08/13/2013
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Briggs, Gregory J., Rohr, Vincent M.
• Primary Examiner(s): Jami, Hares
• Application Number:
  US12/870,531
• Time in Patent Office: 1,082 Days
• Field of Search: 707/790, 707/802, 707/803, 707/804, 707/999.1, 707/999.101, 707/999.102
• US Class Current: 707/802
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/2322 using timestamps

Abstract

A blob transaction component is a tiered middleware framework component that facilitates data transactions across arbitrary data blobs in an optimistically consistent manner. The component receives a request to transact data across multiple stored data blobs. For each stored data blob, the component creates a duplicate data blob. The component then creates a record blob that includes a pointer to the duplicate data blob. The stored data blob is then modified by the component to point to the record blob using optimistic locking. The component then creates a new data blob that includes modified data. The component further updates the record blob to include a pointer to the new data blob using optimistic locking. Following locking of the record blob, the stored data blob is updated with the modified data by the component, and the record blob, the duplicate data blob, and the new data blobs are deleted.

Claims

1. One or more computer readable non-transitory memories storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

receiving a request to transact data across a plurality of stored data blobs that include stored data;

creating duplicate data blobs for stored data blobs;

creating a record blob that includes pointers to the duplicate data blobs;

modifying individual stored data blobs to replace the respective stored data with a pointer that points to the record blob using optimistic locking;

creating new data blobs that include modified data for the stored data blobs based at least in part on the request to transact data;

updating the record blob to include pointers to the new data blobs using optimistic locking;

locking the record blob from additional updates; and

updating the individual stored data blobs to replace a corresponding pointer with the modified data from a corresponding new data blob.

2. The one or more computer readable non-transitory memories of claim 1, further comprising instructions that, when executed, cause the one or more processors to perform acts comprising:

receiving a request from an application to read data from one of the stored data blobs;

when the one stored data blob contains the requested data, retrieving the requested data from the one stored data blob and returning the retrieved data to the application; and

when the one stored data blob contains a pointer to the record blob, retrieving the requested data from a duplicate data blob or a new data blob that corresponds to the one stored data blob and returning the retrieved data to the application.

3. The one or more computer readable non-transitory memories of claim 1, further comprising instructions that, when executed, cause the one or more processors to perform acts comprising:

examining a time stamp of the record blob during a request to read data from a stored data blob, a request to write data to another new data blob, or the request to transact data across the plurality of stored data blobs; and

when the time stamp is older than a predefined timeout threshold, updating the stored data blobs with the modified data from a corresponding new data blob or a corresponding duplicate data blob, and deleting the corresponding duplicate data blobs or the corresponding new data blobs.

4. The one or more computer readable non-transitory memories of claim 3, further comprising instructions that, when executed, cause the one or more processors to perform an act of locking the record blob from further modification when the time stamp is older than a predefined timeout threshold and the record blob is unlocked.

5. The one or more computer readable non-transitory memories of claim 1, further comprising instructions that, when executed, cause the one or more processors to perform an act of deleting the record blob, one or more duplicate data blobs, and one or more new data blobs.

6. A computer-implemented method, comprising:

under control of one or more computing systems comprising one or more processors executing executable instructions,

performing a delegation operation on a plurality of data blobs based on a data transaction, the delegation operation including;

creating a duplicate data blobs for stored data blobs in the plurality of data blobs, the stored data blobs including stored data;

creating a record blob that includes pointers to the duplicate data blobs; and

modifying individual stored data blobs to replace respective stored data with a corresponding additional pointer that points to the record blob storing the pointers to the duplicate data blobs using optimistic locking.

7. The computer-implemented method of claim 6, wherein the modifying using optimistic locking includes:

reading a globally unique identifier (GUID) of a stored data blob for a first time prior to creating the corresponding duplicate data blob for the stored data blob;

reading the GUID of the stored data blob for a second time prior to the modifying of the stored data blob;

modifying the stored data blob to point to the record blob when the GUID of the stored data blob remains unchanged from the first reading to the second reading; and

modifying the GUID of the stored data blob following the modifying of the stored data blob to point to the record blob.

8. The computer-implemented method of claim 6, further comprising performing a transaction operation on the plurality of data blobs, the transaction operation including:

creating a new data blob that includes modified data for the stored data blob based at least in part on the data transaction; and

updating the record blob to include a pointer to the new data blob using optimistic locking.

9. The computer-implemented method of claim 7, wherein the delegation operation and the transaction operation are performed in parallel.

10. The computer-implemented method of claim 7, further comprising performing a delegation removal operation on the plurality of data blobs, the delegation removal operation including:

locking the record blob from additional updates;

updating the stored data blob with the modified data from a corresponding new data blob; and

deleting the record blob, the duplicate data blobs, and the new data blobs.

11. The computer-implemented method of claim 8, wherein the updating the record blob using optimistic locking includes:

reading a globally unique identifier (GUID) of the record blob for a first time prior to creating the new data blob that corresponds to a stored data blob;

reading the GUID of the record blob for a second time prior to the updating of the record blob;

updating the record blob to include a pointer to the new data blob that corresponds to the stored data blob when the GUID of the record blob remains unchanged from the first reading to the second reading; and

modifying the GUID of the record blob following the updating of the record blob.

12. The computer-implemented method of claim 7, further comprising:

receiving a request to read data from one of the stored data blobs from an application;

when the one stored data blob contains the requested data, retrieving the requested data from the one stored data blob and returning the retrieved data to the application; and

when the one stored data blob contains a pointer to the record blob, retrieving the requested data from a duplicate data blob or a new data blob that corresponds to the one stored data blob and returning the retrieved data to the application.

13. The computer-implemented method of claim 12, further comprising determining whether the one stored data blob contains the requested data or a corresponding pointer based at least in part on a custom header of the one stored data blob, a prefix portion of data in the one stored data blob, or a suffix portion of data in the one stored data blob.

14. The computer-implemented method of claim 7, further comprising:

examining a time stamp of the record blob during a request to read data from a stored data blob, a request to write data to another new data blob, or a request to transact data across the plurality of stored data blobs;

when the time stamp is older than a predefined timeout threshold, updating the stored data blob with the modified data from a corresponding new data blob or a corresponding duplicate data blob; and

deleting the corresponding duplicate data blobs or the corresponding new data blobs.

15. The computer-implemented method of claim 14, wherein the time stamp indicates a last modification time of the record blob in real time or Lamport time.

16. The computer-implemented method of claim 14, wherein the creating the new data blob for the stored data blob is completed prior to the creating the record blob.

17. A computer-implemented method, comprising:

under control of one or more computing systems comprising one or more processors executing executable instructions,

storing a first identifier of a stored data blob in a first master blob, the first identifier enabling an application to access data in the stored data blob;

creating a second master blob that stores the first identifier of the stored data blob;

creating a record blob that includes a first pointer to the second master blob;

modifying the first master blob by replacing the first identifier stored in the first master blob with a second pointer to the record blob using optimistic locking;

creating a new data blob that includes modified data for the stored data blob based at least in part on a data transaction and a third master blob stores a second identifier of the new data blob, the second identifier enabling the application to access data in the new data blob;

updating the record blob by replacing the first pointer in the record blob with a third pointer to the third master blob via optimistic locking; and

locking the record blob from additional updates and replacing the second pointer in the first master blob with the second identifier of the new data blob.

18. The computer-implemented method of claim 17, wherein the modifying using optimistic locking includes:

reading a version number of the first master blob for a first time when the identifier of the stored data blob is stored into the first master blob;

reading the version number of the first master blob for a second time prior to the replacing the stored first identifier with the second pointer;

modifying the first master blob by replacing the first identifier with the second pointer when the version number of the first master blob remains unchanged from the first reading to the second reading; and

modifying the version number of the first master blob following the replacing of the stored first identifier with the second pointer.

19. The computer-implemented method of claim 17, wherein the updating the record blob using optimistic locking includes:

reading a globally unique identifier (GUID) of the record blob for a first time prior to creating the new data blob that corresponds to a stored data blob;

reading the GUID of the record blob for a second time prior to the updating of the record blob;

updating the record blob by replacing the first pointer with the third pointer when the GUID of the record blob remains unchanged from the first reading to the second reading; and

modifying the GUID of the record blob following the updating of the record blob.

20. The computer-implemented method of claim 17, further comprising:

receiving a request to read data from the stored data blob from the application;

when the first master blob includes the identifier of the stored data blob, retrieving the requested data from the stored data blob and returning the retrieved data to the application; and

when the first master blob includes the second pointer to the record blob and the record blob includes the first pointer to the second master blob, retrieving the requested data from the stored data blob using second pointer, the first pointer, and the first identifier stored in the second master blob and returning the retrieved data to the application.

21. The computer-implemented method of claim 17, further comprising:

receiving a request to read data from the new data blob from the application;

when the first master blob includes the identifier of the new data blob, retrieving the requested data from the new data blob and returning the retrieved data to the application; and

when the first master blob includes the second pointer to the record blob and the record blob includes the third pointer to the third master blob, retrieving the requested data from the new data blob using second pointer, the third pointer, and the second identifier stored in the second master blob and returning the retrieved data to the application.

22. The computer-implemented method of claim 17, further comprising:

examining a time stamp of the record blob during a request to read data from the stored data blob, a request to write data to another new data blob, or a request to transact data across a plurality of stored data blobs; and

when the time stamp is older than a predefined timeout threshold, updating the first master blob to hold the second identifier of the new data blob, the second identifier enabling the application to access data in the new data blob, and deleting the record blob, the stored data blob, the second master blob and the third master blob.

23. The computer-implemented method of claim 22, further comprising locking the record blob from further modification when the time stamp is older than a predefined timeout threshold and the record blob is unlocked.

24. The computer-implemented method of claim 22, wherein the time stamp indicates a last modification time of the record blob in real time or Lamport time.

25. The computer-implemented method of claim 17, further comprising deleting the record blob, the stored data blob, the second master blob and the third master blob.