Data storage application programming interface

• US 9,225,675 B2
• Filed: 08/08/2012
• Issued: 12/29/2015
• Est. Priority Date: 08/08/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Patiejunas, Kestutis, Claiborn, Christian L., Lazier, Colin L., Suver, Claire E., Seigle, Mark C.
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Khurshid, Zia
• Application Number:
  US13/570,074
• Publication Number:
  US 20140047040A1
• Time in Patent Office: 1,238 Days
• Field of Search: 707/687, 709/206
• US Class Current: 1/1
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1443 Transmit or communication errors
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages

Abstract

An application programming interface for a data storage service provides a convenient mechanism for clients of the data storage service to access its various capabilities. An API call may be made to initiate a job and in response a job identifier may be provided. A separate API call specifying the job identifier may be made and a response providing information related to the job may result. Various API calls may be used to store data, retrieve data, obtain an inventory of stored data, and to obtain other information relating to stored data.

Claims

1. A computer-implemented method for providing cost-effective and durable archival data storage service, comprising:

under the control of one or more computer systems of an archival data storage system that are configured with executable instructions,

receiving, over a communications network from a requesting computer system, an electronic retrieval request to retrieve a data object from persistent storage, the electronic retrieval request encoding an identifier of the data object;

in response to receipt of the electronic retrieval request;

initiating a retrieval job to obtain the data object from persistent storage;

providing, to the requesting computer system, a retrieval job identifier for the retrieval job that is usable to obtain a status of the retrieval job and that, after the retrieval job is completed, is usable to retrieve the data object;

receiving, over the communications network, an electronic download request to download at least a portion of the data object, the electronic download request encoding the retrieval job identifier; and

in response to receipt of the electronic download request and as a result of the retrieval job having been completed, providing the at least a portion of the data object for download.

2. The computer-implemented method of claim 1, further comprising:

receiving an electronic request for a status of the retrieval job, the electronic request for the status encoding the retrieval job identifier; and

in response to receipt of the electronic request for the status of the retrieval job, providing a current status of the retrieval job.

3. The computer-implemented method of claim 1, wherein:

the download request includes information specifying a portion of the data object; and

providing the at least a portion of the data object includes providing the at least the portion.

4. The computer-implemented method of claim 3, wherein providing the at least the portion includes providing a data verification value for the at least the portion, the data verification value usable to determine whether the provided at least the portion is corrupted.

5. The computer-implemented method of claim 1, further comprising, as a result of the electronic retrieval request including a notification parameter, providing a notification upon completion of the retrieval job prior to receipt of the electronic download request.

6. A computer-implemented method comprising:

under the control of one or more computer systems configured with executable instructions,

receiving a first electronic message comprising a first identifier corresponding to data stored by a storage system;

in response to receipt of the first electronic message, initiating a job in connection with the data;

at a time after receiving the first electronic message, receiving a second electronic message comprising a second identifier corresponding to the job, where the second identifier is associated with the job and useable to retrieve the data corresponding to the first identifier; and

in response to receipt of the second electronic message, providing information associated with the job;

receiving a third electronic message comprising the second identifier; and

in response to receipt of the third electronic message, providing at least a portion of the data for download.

7. The computer-implemented method of claim 6, wherein the first identifier and the second identifier are different.

8. The computer-implemented method of claim 6, wherein:

the data comprises one or more data objects contained in a logical data container; and

the first identifier is an identifier for the logical data container.

9. The computer-implemented method of claim 8, wherein the information associated with the job includes a collection of one or more identifiers that each identifies one of the one or more data objects.

10. The computer-implemented method of claim 8, wherein the information associated with the job includes a log of one or more actions performed in connection with the one or more data objects contained in the logical data container.

11. The computer-implemented method of claim 6, wherein:

the first electronic message originated from a customer computer system of a customer of the storage system;

the first electronic message includes metadata provided by the customer computer system; and

the information associated with the job includes at least a portion of the metadata.

12. A system, comprising:

one or more processors; and

memory storing executable instructions that, when executed by the one or more processors, cause the system to;

receive a first electronic message comprising a first identifier corresponding to data stored by a storage system;

in response to receipt of the first electronic message, initiate a job in connection with the data;

at a time after receiving the first electronic message, receive a second electronic message comprising a second identifier corresponding to the job, where the second identifier is associated with the job and useable to retrieve the data corresponding to the first identifier; and

in response to receipt of the second electronic message, provide information associated with the job;

receive a third electronic message comprising the second identifier; and

in response to receipt of the third electronic message, provide at least a portion of the data for download.

13. The system of claim 12, wherein the first identifier and the second identifier are different.

14. The system of claim 12, wherein:

the data comprises one or more data objects contained in a logical data container; and

the first identifier identifies the logical data container.

15. The system of claim 14, wherein the information associated with the job includes a collection of one or more identifiers that each identifies one of the one or more data objects.

16. The system of claim 14, wherein the information associated with the job includes a log of one or more actions performed in connection with the one or more data objects contained in the logical data container.

17. The system of claim 12, wherein:

the first electronic message originated from a customer computer system of a customer of the storage system;

the first electronic message includes metadata provided by the customer computer system; and

the information associated with the job includes at least a portion of the metadata.

18. The system of claim 12, wherein the system provides the at least the portion of the data for download by moving the at least the portion of the data to another storage device of the storage system.

19. The system of claim 12, wherein the instructions, when executed by the one or more processors, cause the system to provide a notification of completion of the job.

20. The system of claim 19, wherein the notification is provided as a result of the first electronic message specifying a notification parameter.

21. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by one or more processors of a computer system, cause the computer system to:

in response to receipt of a first electronic message comprising a first identifier corresponding to data stored by a storage system, initiate a job in connection with the data;

at a time after receiving the first electronic message, in response to receipt of a second electronic message comprising a second identifier corresponding to the job, where the second identifier is associated with the job and useable to retrieve the data corresponding to the first identifier, provide information associated with the job; and

in response to receipt of a third electronic message comprising the second identifier, provide the at least a portion of the data for download.

22. The non-transitory computer-readable storage medium of claim 21, wherein the first identifier and the second identifier are different.

23. The non-transitory computer-readable storage medium of claim 21, wherein:

the data comprises one or more data objects contained in a logical data container; and

the first identifier identifies the logical data container.

24. The non-transitory computer-readable storage medium of claim 23, wherein the information associated with the job includes a collection of one or more identifiers that each identifies one of the one or more data objects.

25. The non-transitory computer-readable storage medium of claim 23, wherein:

the first electronic message originated from a customer computer system of a customer of the storage system;

the first electronic message includes metadata provided by the customer computer system; and

the information associated with the job includes at least a portion of the metadata.

26. The non-transitory computer-readable storage medium of claim 21, wherein the instructions that cause the computer system to provide the at least the portion of the data for download, when executed, cause the computer system to provide the at least the portion of the data for download by moving the at least the portion of the data to another storage device of the storage system.

27. The non-transitory computer-readable storage medium of claim 21, wherein the system provides the at least the portion at least in part by providing a data verification value for the at least the portion, the data verification value usable to determine whether the provided specified portion is corrupted.

28. The non-transitory computer-readable storage medium of claim 21, wherein the information associated with the job indicates whether data has been stored in a component of the storage system from which the data is downloadable.

29. The non-transitory computer-readable storage medium of claim 21, wherein the instructions, when executed by the one or more processors, cause the system to provide a notification of completion of the job.

30. The non-transitory computer-readable storage medium of claim 29, wherein the notification is provided as a result of the first electronic message specifying a notification parameter.