Command parallelization for data storage systems

• US 10,496,327 B1
• Filed: 09/28/2016
• Issued: 12/03/2019
• Est. Priority Date: 09/28/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Paterra, Frank Charles, Neilsen, Eric, Portnoy, Andrew, Basarir, Firat
• Primary Examiner(s): Leung, Robert B
• Application Number:
  US15/279,348
• Time in Patent Office: 1,161 Days
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A data storage system implements aggregation, bifurcation, and/or reduction techniques to improve the efficiency of processing data storage requests. Data storage requests and/or their associated payloads may be aggregated based on one or more parameters. Data to be the stored and the associated commands may be separated so as to optimize a system'"'"'s throughput and latency for each. Furthermore, extraneous commands and requests may be reduced or eliminated based on heuristics associated with the requests and the data.

Claims

1. A computer-implemented method, comprising:

receiving, at a control plane, an encrypted first programmatic request to store data on data storage of a server entity, the data storage being configured with a file system;

receiving, at a data plane, a second programmatic request in connection with the first programmatic request, the second programmatic request including an encrypted form of the data to be stored;

processing the first programmatic request so as to initiate a write command for the data received in connection with the second programmatic request;

processing, by the control plane, one or more additional file system commands related to the write command; and

providing an outcome of the first programmatic request to a requestor.

2. The computer-implemented method of claim 1, further comprising decrypting the encrypted form of the data prior to processing the first programmatic request.

3. The computer-implemented method of claim 1, wherein the first programmatic request and the second programmatic request are received from an entity that receives a third programmatic request that includes the first programmatic request and the data to be stored.

4. The computer-implemented method of claim 1, wherein the first programmatic request is an API call.

5. A system, comprising:

one or more processors; and

memory including instructions that, as a result of being executed by the one or more processors, cause the system to;

receive, at a control plane, a first programmatic request including a write command to store data on data storage;

receive, at a data plane, a second programmatic request in connection with the first programmatic request, the second programmatic request including the data to be stored;

process the first programmatic request so as to initiate the write command using the data received in connection with the second programmatic request; and

process, by the control plane, one or more additional file system commands related to the initiated write command.

6. The system of claim 5, wherein the first request is an API call.

7. The system of claim 5, wherein the first request is submitted through a user interface.

8. The system of claim 7, wherein the second request identifies the first request.

9. The system of claim 7, wherein the second request is unencrypted.

10. The system of claim 7, wherein the first request is received using SSL.

11. The system of claim 7, wherein the one or more additional commands are generated as a result of processing the first request.

12. A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to at least:

receive, at a control plane, a first set of programmatic requests to store data on data storage;

receive, at a data plane, a second set of programmatic requests in connection with the first set of programmatic requests, the second set of programmatic requests including the data to be stored;

process the first set of programmatic requests so as to initiate one or more write commands for the data received in connection with the second set of programmatic requests; and

process, by the control plane, one or more additional file system commands related to the initiated one or more write commands.

13. The non-transitory computer-readable storage medium of claim 12, wherein at least some of the first set of requests are generated via a user interface.

14. The non-transitory computer-readable storage medium of claim 12, wherein the data plane is hosted by a different computer system to which the computer system connects via a network.

15. The non-transitory computer-readable storage medium of claim 12, wherein the data includes files.

16. The non-transitory computer-readable storage medium of claim 12, wherein the second set of requests is received via an unencrypted network connection.

17. The non-transitory computer-readable storage medium of claim 12, wherein outcomes of the first set of requests are provided to a requestor via the computer system.

18. The non-transitory computer-readable storage medium of claim 12, wherein the executable instructions that, as a result of being executed, cause the computer system to process the first set of requests, further include executable instructions to cause the computer system to first decrypt the data.

19. The non-transitory computer-readable storage medium of claim 12, wherein at least some of the first set of requests are web service calls.