Multi-blob consistency for atomic data transactions

• US 8,688,666 B1
• Filed: 08/27/2010
• Issued: 04/01/2014
• Est. Priority Date: 08/27/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Briggs, Gregory J., Rohr, Vincent M.
• Primary Examiner(s): Jami, Hares
• Application Number:
  US12/870,661
• Time in Patent Office: 1,313 Days
• Field of Search: 707/705, 707/722, 707/758, 707/790, 707/999.1, 707/999.3
• US Class Current: 707/705
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof

Abstract

A multi-blob consistency component of a tiered middleware framework ensures data blobs are transacted in an atomic manner. The component determines a data blob of a data store to be modified based on an application request. The component then reads a master blob to locate a stored version number of the data blob to be modified and a version number of the master blob. A new data blob with a new version number that replaces the data blob to be modified is written to the data store. The component then reads the master blob again to re-obtain the version number of the master blob. Thus, when the obtained and re-obtained version numbers match, the component replaces the stored version number of the data blob with the new version number of the new data blob. Further, the component deletes the data blob to be modified using the stored version number.

Claims

1. One or more computer readable non-transitory memory storing computer-executable instructions that, when executed, cause one or more processors to perform acts, comprising:

determining one or more data blobs in at least one data store to be modified based on an application request, wherein individual ones of the data blobs to be modified have associated therewith a unique data blob identifier;

reading a master blob to (1) locate a stored version number of an individual data blob to be modified, and (2) obtain a version number of the master blob;

for the individual data blob to be modified, writing to the data store a new data blob with a new version number and the unique data blob identifier of the respective data blob to be modified;

reading the master blob to re-obtain the version number of the master blob;

determining whether the obtained version number of the master blob matches the re-obtained version number of the master blob;

in response to determining that the obtained version number of the master blob matches the re-obtained version number of the master blob;

replacing the stored version number of the individual data blob to be modified in the master blob with a new version number of a corresponding new data blob; and

deleting the individual data blob to be modified using a corresponding stored version number; and

in response to determining that the obtained version number of the master blob is different than the re-obtained version number of the master blob, deleting the one or more new data blobs.

2. The one or more computer readable non-transitory memory of claim 1, wherein the master blob resides in a first layer of a master blob tree, and the version number of the master blob is tracked by an additional master blob in a second layer of the master blob tree that is closer to an application layer than the first layer.

3. The one or more computer readable non-transitory memory of claim 2, wherein the additional master blob in the second layer includes an additional version number, and further comprising updating the additional version number when the version number of the master blob is updated.

4. One or more computer readable non-transitory memory storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising, comprising:

determining a new data blob to be created in a data store based on an application request;

reading a master blob that corresponds to the new data blob to obtain a version number of the master blob;

writing the new data blob with a new version number and a new data blob identifier to the data store;

reading the master blob to re-obtain the version number of the master blob;

writing the new version number of the new data blob to the master blob when the obtained version number of the master blob matches the re-obtained version number of the master blob; and

deleting the new data blob from the data store when the obtained version number of the master blob is different than the re-obtained version number of the master blob.

5. The one or more computer readable non-transitory memory of claim 4, further comprising instructions that, when executed, cause the one or more processors to perform an act of creating the master blob for the new data blob when no existing master blob storing a version number of an existing data blob that is in the same set as the new data blob is present.

6. The one or more computer readable non-transitory memory of claim 4, further comprising instructions that, when executed, cause the one or more processors to perform an act of updating the version number of the master blob with a new master blob version number following the writing of the new data blob to the data store.

7. The one or more computer readable non-transitory memory of claim 4, further comprising instructions that, when executed, cause the one or more processors to perform acts comprising:

determining a data blob to be deleted from the data store or a different data store based on an additional application request;

reading an additional master blob that corresponds to the data blob to be deleted to (1) locate a stored version number of the data blob to be deleted, and (2) obtain an additional version number of the additional master blob;

determining that no new data blob is to be written to the data store;

reading the additional master blob to re-obtain the additional version number of the additional master blob;

deleting the stored version number and the data blob to be deleted that corresponds to the stored version number when the obtained additional version number of the additional master blob matches the re-obtained additional version number of the additional master blob; and

aborting the deleting of the stored version number and the data blob to be deleted when the obtained additional version number of the additional master blob is different than the re-obtained additional version number of the additional master blob.

8. The one or more computer readable non-transitory memory of claim 7, further comprising instructions that, when executed, cause the one or more processors to perform an act of updating the additional version number of the additional master blob with another new master blob version number following the determining that no new data blob is to be written and prior to the deleting of the data blob to be deleted.

9. The one or more computer readable non-transitory memory of claim 7, further comprising instructions that, when executed, cause the one or more processors to perform an act of deleting the additional master blob when the additional master blob is empty of any stored version numbers.

10. The one or more computer readable non-transitory memory of claim 4, further comprising instructions that, when executed, cause the one or more processors to perform acts that include:

receiving a particular data blob identifier as part of an additional application request to access data from the data store;

reading the master blob to locate a current data blob version number that corresponds to the particular data blob identifier;

locating one or more data blobs in the data store having the particular data blob identifier using a prefix query;

accessing data from one of the one or more data blobs when the one data blob has the particular data blob identifier and the current data blob version number; and

retrying the reading and the accessing when no data blob with the particular data blob identifier and the current data version number is located via the prefix query.

11. A computer-implemented method, comprising:

under control of one or more computing systems comprising one or more hardware processors executing executable instructions to perform acts that include;

determining a data blob in a data store to be modified based on an application request, the data blob to be modified having a particular data blob identifier;

reading a master blob to (1) locate a stored version number of the data blob to be modified and (2) obtain a version number of the master blob;

writing a new data blob with a new version number and the particular data blob identifier to the data store as a replacement for the data blob to be modified;

reading the master blob to re-obtain the version number of the master blob; and

when the obtained version number of the master blob matches the re-obtained version number of the master blob;

replacing the stored version number of the data blob to be modified in the master blob with the new version number; and

deleting the data blob to be modified using the stored version number.

12. The computer-implemented method of claim 11, further comprising deleting the new data blob when the obtained version number of the master blob is different than the re-obtained version number of the master blob.

13. The computer-implemented method of claim 12, further comprising repeating performance of the acts when the obtained version number of the master blob is different than the re-obtained version number of the master blob.

14. The computer-implemented method of claim 11, further comprising updating the version number of the master blob with a new master blob version number following the deletion of the data blob to be modified.

15. The computer-implemented method of claim 11, wherein the reading includes ascertaining the master blob to read based at least in part on the particular data blob identifier of the data blob and a naming convention.

16. The computer-implemented method of claim 11, wherein one or more of the stored version number, the new version number, and the version number of the master blob is a globally unique identifier (GUID).

17. The computer-implemented method of claim 11, wherein the stored version number and the particular data blob identifier of the data blob to be modified forms a versioned identifier.

18. The computer-implemented method of claim 11, wherein the master blob further includes a master blob identifier, and wherein the master blob identifier and the version number of the master blob forms a versioned identifier.

19. The computer-implemented method of claim 11, wherein the master blob resides in a first layer of a master blob tree, and the version number of the master blob is tracked by an additional master blob in a second layer of the master blob tree that is closer to an application layer than the first layer.

20. The computer-implemented method of claim 19, wherein the additional master blob in the second layer includes an additional version number, and further comprising updating the additional version number when the version number of the master blob is updated.

21. The computer-implemented method of claim 11, further comprising:

receiving a specific data blob identifier as part of an additional application request to access data from the data store;

reading the master blob to locate a current data blob version number that corresponds to the specific data blob identifier;

accessing data from an additional data blob in the data store when the additional data blob has the specific data blob identifier and the current data blob version number; and

retrying the reading and the accessing when no data blob with the specific data blob identifier and the current data version number is present in the data store.

22. A computer-implemented method, comprising:

under control of one or more computing systems comprising one or more hardware processors executing executable instructions,

receiving a particular data blob identifier as part of an application request to access data from a data store;

identifying a parent master blob based on the particular data blob identifier, the parent master blob including a data entry that leads to a child master blob, the data entry including a child master blob identifier for the child master blob and a version number for the child master blob,

locating the child master blob based on the child master blob identifier and the version number of the child master blob in the data entry of the parent master blob, the child master blob storing a current data blob version number that corresponds to the particular data blob identifier;

reading the child master blob to locate the current data blob version number that corresponds to the particular data blob identifier;

accessing data from a data blob in the data store when the data blob has the particular data blob identifier and the current data blob version number; and

retrying the reading and the accessing when no data blob with the particular data blob identifier and the current data version number is present in the data store.

23. The computer-implemented method of claim 22, wherein the retrying includes retrying for a predetermined number of times when no data blob with the particular data blob identifier and the current data version number is present in the data store.

24. The computer-implemented method of claim 22, wherein the accessing includes refraining from accessing data from an additional data blob in the data store when the additional data blob has the particular blob identifier and an additional data blob version number that is different from the current data blob version number.

25. The computer-implemented method of claim 23, further comprising returning the data to an application that initiated the application request to access the data from the data store.

26. The computer-implemented method of claim 22, wherein the current data blob version number is a globally unique identifier (GUID).

27. The computer-implemented method of claim 22, wherein the current data blob version number and the particular data blob identifier forms a versioned identifier, and wherein the accessing includes locating the data blob via the versioned identifier.

28. One or more computer readable non-transitory memory storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

in response to receiving a particular data blob identifier as part of an application request to access data from a data store, reading a master blob to locate a current data blob version number that corresponds to the particular data blob identifier;

locating one or more data blobs in the data store having the particular data blob identifier using a prefix query;

accessing data from one of the one or more data blobs when the one data blob has the particular data blob identifier and the current data blob version number;

ignoring the one or more data blobs by refraining from accessing data from the one data blob when the one data blob has the particular data blob identifier and a corresponding data blob version number that is different from the current data blob version number; and

retrying the reading and the accessing when no data blob with the particular data blob identifier and the current data version number is located via the prefix query.

29. The one or more computer readable non-transitory memory of claim 28, wherein the reading and the locating are performed simultaneously or substantially simultaneously.

30. The one or more computer readable non-transitory memory of claim 28, wherein the retrying includes retrying for a predetermined number of times when no data blob with the particular data blob identifier and the current data version number is located via the prefix query.

31. The one or more computer readable non-transitory memory of claim 28, wherein the reading includes ascertaining the master blob to read based at least in part on the identifier of the master blob.

32. The one or more computer readable non-transitory memory of claim 28, wherein the current data blob version number is a globally unique identifier (GUID).

33. The one or more computer readable non-transitory memory of claim 28, wherein the current data blob version number and the particular data blob identifier forms a versioned identifier, and wherein the accessing includes locating the one data blob via the versioned identifier.