Context-aware durability of file variants

• US 8,990,265 B1
• Filed: 03/14/2011
• Issued: 03/24/2015
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Velummylum, Piragash, Olson, Johanna S., Smith, Korwin J., Wood, James H.
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/047,312
• Time in Patent Office: 1,471 Days
• Field of Search: 707/101, 707/736, 707/770, 709/217, 709/219, 711/216, 705/26.1
• US Class Current: 707/803
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

Disclosed are various embodiments for context-aware durability of variants of files. A first data store is associated with a first level of data durability, and a second data store is associated with a second level of data durability that is lower than the first level of data durability. A data object is stored in the second data store when the data object is a derivative of another data object stored in the first data store. A data object is stored in the first data store when the data object is not a derivative of another data object stored in the first data store.

Claims

1. A non-transitory computer-readable medium embodying at least one program executable in at least one computing device, the at least one program having instructions that perform a method comprising:

maintaining, by the at least one computing device, a plurality of file systems for a plurality of users, each of the file systems including a plurality of files and at least one folder, each of the files including a respective unique identifier and a respective reference to a primary data object stored in a first data store;

generating, by the at least one computing device, a variant of one of the files, the variant including the unique identifier of the one of files and a reference to a secondary data object that is derived from the primary data object referenced by the one of the files, the variant being customized for a client associated with the corresponding one of the users that is associated with the one of the files, the secondary data object being stored in a reduced durability data store;

obtaining, by the at least one computing device, a request for the one of the files from the client;

sending, by the at least one computing device, the secondary data object to the client in response to the request instead of the primary data object when the secondary data object is available;

regenerating, by the at least one computing device, the secondary data object from the primary data object when the secondary data object is unavailable and sending the secondary data object to the client in response to the request instead of the primary data object; and

wherein the reduced durability data store is associated with a lower level of data durability than the first data store.

2. The non-transitory computer-readable medium of claim 1, wherein the variant corresponds to one of a plurality of variants of the one of the files, and each one of the variants is customized for a respective one of a plurality of clients associated with the corresponding one of the users that is associated with the one of the files.

3. The non-transitory computer-readable medium of claim 1, wherein the method further comprises:

obtaining, by the at least one computing device, another request for the one of the files from another client; and

sending, by the at least one computing device, the primary data object to the other client in response to the request.

4. A system, comprising:

a first data store that is associated with a first level of data durability;

a reduced-durability data store that is associated with a second level of data durability that is lower than the first level of data durability; and

at least one computing device configured to perform a method comprising;

obtaining a request to store a data object;

determining if the data object is a derivative of another data object stored in the first data store;

storing the data object in the reduced-durability data store when the data object is the derivative of the another data object stored in the first data store; and

storing the data object in the first data store when the data object is not the derivative of the another data object stored in the first data store.

5. The system of claim 4, method further comprises:

obtaining a customer request to store the data object in the reduced-durability data store; and

storing the data object in the reduced-durability data store instead of the first data store in response to the customer request.

6. The system of claim 4, wherein the method further comprises:

determining whether the data object stored in the reduced-durability data store has been lost;

initiating a regeneration of the data object from the another data object stored in the first data store when the data object stored in the reduced-durability data store has been lost; and

storing the data object that has been regenerated in the reduced-durability data store.

7. The system of claim 4, wherein the method further comprises deleting the data object from the reduced-durability data store when the data object has not been accessed within a predetermined length of time.

8. The system of claim 7, wherein the method further comprises initiating a regeneration of the data object from the another data object stored in the first data store after the data object has been deleted from the reduced-durability data store.

9. The system of claim 4, wherein the method further comprises:

obtaining a request for the data object, wherein the data object is the derivative of the another data object;

determining whether the data object is available from the reduced-durability data store;

producing the data object from the reduced-durability data store in response to the request when the data object is available from the reduced-durability data store; and

producing the data object in response to the request by regenerating the data object from the another data object when the data object is not available from the reduced-durability data store.

10. The system of claim 4, wherein the data object stored in the reduced-durability data store is a data-reduced version of the another data object.

11. The system of claim 4, wherein the data object stored in the reduced-durability data store is a version of the another data object that is customized for a type of client that is associated with a file system.

12. The system of claim 4, wherein the method further comprises generating the data object from the another data object.

13. The system of claim 4, wherein the reduced-durability data store is configured to store fewer redundant copies of data than the first data store.

14. The system of claim 4, wherein the reduced-durability data store is associated with a lower level of hardware redundancy than the first data store.

15. The system of claim 4, wherein the first data store includes data that can be processed to regenerate other data stored in the first data store, and the reduced-durability data store excludes data that can be processed to regenerate other data stored in the reduced-durability data store.

16. A method, comprising:

obtaining, by at least one computing device, a request to store a data object from a user;

generating, by the at least one computing device, a variant of the data object that is customized for delivery to a client associated with the user, the variant being generated from the data object;

communicating, by the at least one computing device, responsive to a request for the data object, the variant of the data object instead of the data object to the client;

deleting, by the at least one computing device, the variant of the data object; and

regenerating, by the at least one computing device, the variant of the data object from the data object.

17. The method of claim 16, wherein the deleting is performed in response to determining, by the at least one computing device, that the client is no longer associated with the user.

18. The method of claim 16, wherein the deleting is performed in response to determining, by the at least one computing device, that the variant has not been accessed within a predetermined length of time.

19. The method of claim 16, wherein the client corresponds to a mobile computing device, and the variant is a version of the data object that is customized for streaming to the mobile computing device.

20. The method of claim 16, wherein the variant comprises a data-reduced version of the data object.

21. The method of claim 16, further comprising:

storing, by the at least one computing device, the data object in a first data store;

storing, by the at least one computing device, the variant in a reduced durability data store; and

wherein the reduced durability data store is associated with a lower level of data durability than the first data store.