Point of presence based data uploading

• US 10,225,326 B1
• Filed: 03/23/2015
• Issued: 03/05/2019
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Puchala, Katarzyna Anna, Radlein, Anton Stephen, Dunlap, David Alexander
• Primary Examiner(s): Bates, Kevin T
• Assistant Examiner(s): Huang, Chen-Liang
• Application Number:
  US14/666,205
• Time in Patent Office: 1,443 Days
• Field of Search: 709224
• CPC Class Codes: H04L 12/5691 Access to open networks; Ingress point selection, e.g. ISP selection
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/18 Multiprotocol handlers, e.g. single devices capable of handling multiple protocols

Abstract

A system, method and computer-readable medium for data uploading based on points of presence (POPs) are provided. In response to a client'"'"'s request for data uploading, the system provides routing information for POPs that may facilitate data communications between the client and a data storage service provider. The client may fragment the upload data and transmit the data fragments via data connections to POPs, which in turn may relay the received fragments to the data storage service provider. Upon receipt of necessary data fragments, the data storage service provider may merge the data fragments to reconstruct a copy of the upload data for storage.

Claims

1. A computer-implemented method for data communication, the computer-implemented method comprising:

under control of one or more computing devices configured with specific computer executable instructions;

obtaining a request to upload target data from a client computing device to the one or more computing devices;

determining a set of points of presence (POPs) to facilitate uploading of the target data from the client computing device to the one or more computing devices based on performance information associated with the set of POPs, wherein the performance information is maintained by the one or more computing devices based on communications between the set of POPs and the one or more computing devices;

providing routing information regarding the set of POPs to the client computing device, wherein the client computing device transmits fragments of the target data to at least a subset of the set of POPs based, at least in part, on the routing information, and wherein, during fragment transmission, the client computing device adjusts a number of the fragments transmitted to individual POPs in the subset based at least in part on a performance of the individual POPs in the subset;

obtaining the fragments of the target data from the subset of POPs;

merging the fragments of the target data into a copy of the target data; and

causing storage of the copy of the target data.

2. The computer-implemented method of claim 1, wherein the client computing device generates the target data fragments in accordance with a data fragmentation encoding.

3. The computer-implemented method of claim 2, wherein the data fragmentation encoding comprises forward error correction code.

4. The computer-implemented method of claim 1, wherein the client computing device transmits at least a subset of the target data fragments to a POP of the subset of POPs via a first network path, wherein the POP corresponds to a destination of the first network path.

5. The computer-implemented method of claim 4, wherein the POP transmits the at least a subset of the target data fragments via a second network path, wherein the POP corresponds to a source of the second network path.

6. The computer-implemented method of claim 1, wherein the client computing device communicates with a first POP of the subset of POPs in accordance with a first network protocol and communicates with a second POP of the subset of POPs in accordance with a second network protocol.

7. The computer-implemented method of claim 1, wherein the request to upload target data includes at least one of a size, type, or priority associated with the target data.

8. The computer-implemented method of claim 1, wherein determining the set of POPs comprises analyzing the performance information associated with at least one of the POPs in the set of POPs and wherein the analysis of the performance information is based, at least in part, on an attribute associated with the target data or associated with the client computing device.

9. The computer-implemented method of claim 8, wherein the performance information corresponds to at least one of latency, geographic proximity, bandwidth, throughput, capacity, cost, load or availability.

10. A non-transitory computer readable storage medium storing computer executable instructions that when executed by one or more processors of one or more computing devices perform operations comprising:

obtaining a request to upload target data to the one or more computing devices;

causing transmission of routing information regarding a set of points of presence (POPs) to a client computing device, wherein the set of POPs are determined based on performance information associated with the set of POPs, wherein the performance information is maintained by the one or more computing devices based on communications between the set of POPs and the one or more computing devices, wherein the client computing device transmits fragments of the target data to at least a first subset of POPs of the set of POPs based, at least in part, on the routing information, and wherein, during fragment transmission, the client computing device adjusts a number of the fragments transmitted to individual POPs in the first subset based at least in part on a performance of the individual POPs in the first subset;

obtaining the fragments of the target data from the set of POPs; and

causing reconstruction of the target data based, at least in part, on the fragments of the target data.

11. The non-transitory computer readable storage medium of claim 10, wherein the client computing device transmits a portion of the target data fragments to a target POP of the first subset of POPs in accordance with a first network protocol and wherein the target POP transmits the portion of the target data fragments to a data storage system in accordance with a second network protocol.

12. The non-transitory computer readable storage medium of claim 10, wherein transmission of the routing information includes transmission of the performance information corresponding to individual POPs of the set of POPs.

13. The non-transitory computer readable storage medium of claim 12, wherein the client computing device generates the target data fragments based, at least in part, on a number of POPs included in the first subset.

14. The non-transitory computer readable storage medium of claim 12, wherein the client computing device selects the first subset of POPs based, at least in part, on the performance information.

15. The non-transitory computer readable storage medium of claim 10, wherein the client computing device initiates transmission of the target data fragments to individual POPs of the first subset of POPs.

16. The non-transitory computer readable storage medium of claim 10, wherein the client computing device transmits at least a subset of the target data fragments to a second subset of POPs of the set of POPs.

17. The non-transitory computer readable storage medium of claim 16, wherein obtaining the target data fragments from the set of POPs comprises obtaining the target data fragments exclusively from the second subset of POPs.

18. A system comprising:

a data store configured to at least store computer-executable instructions; and

a processor in communication with the data store, the processor configured to execute the computer-executable instructions to at least;

send a request to upload target data to a data storage service provider;

in response to the request, receive routing information regarding a set of points of presence (POPs) determined by the data storage service provider, wherein the set of POPs are determined based on performance information associated with the set of POPs, wherein the performance information is maintained by the data storage service provider based on communications between the set of POPs and the data storage service provider;

cause fragmentation of the target data;

transmit fragments of the target data resulting from the fragmentation to at least a subset of the set of POPs based, at least in part, on the routing information; and

as the fragments are transmitted, adjust a number of the fragments transmitted to individual POPs in the subset based at least in part on a performance of the individual POPs in the subset.

19. The system of claim 18, wherein the processor is further configured to execute the computer-executable instructions to send processing information for reconstructing the target data based on at least a portion of the transmitted target data fragments.

20. The system of claim 18, wherein at least one POP of the subset of POPs transmits at least a portion of the target data fragments to a POP of a second set of POPs.