Locality based content distribution

• US 8,060,561 B2
• Filed: 06/27/2011
• Issued: 11/15/2011
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Richardson, David R., Marshall, Bradley E., Sivasubramanian, Swaminathan
• Primary Examiner(s): Jacobs; Lashonda T
• Application Number:
  US13/170,102
• Publication Number:
  US 20110258253A1
• Time in Patent Office: 141 Days
• Field of Search: 709/231, 709/223, 709/224, 709/229, 709/203, 709/217, 709/219
• US Class Current: 709/203
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1082 involving incentive schemes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/562 Brokering proxy services

Abstract

A system and method for content distribution are provided. A content provider generates a network topology having one or more subnetworks made up of content sources. The content provider segments content, such as applications, into segments for distribution. The content provider then distributes the content such that for each identified subnetwork, each content segment is distributed to a content source. Subsequent, client computing device requests for content can be serviced by selecting a content source in a particular subnetwork selected for delivering the requested content to the client computing device to minimize traffic flow of the requested content through the nodes of the distribution network.

Claims

1. A system for managing content at a content provider, the system comprising:

at least one computing device configured to;

associate content into a set of content segments;

associate a distribution network into a set of subnetworks, wherein each subnetwork includes one or more content sources;

distribute the content so that the set of content segments is distributed among the one or more content sources in each subnetwork in the distribution network;

obtain a request for the content from a client computing device;

identify a subnetwork in the distribution network for providing the requested content based on a network locality of the client computing device relative to subnetworks in the distribution network;

for each content segment in the set of content segments, identify at least one or more content sources in the identified subnetwork corresponding to a registered content provider for providing the content segment; and

provide identification information responsive to the request for content from the client computing device, wherein the identification information includes, for each content segment, network addresses of the identified registered content provider.

2. The system as recited in claim 1, wherein the identification information further includes, for each content segment, reconciliation information to be provided to the identified registered content provider.

3. The system as recited in claim 2, wherein the at least one computing device is further configured to obtain from at least one registered content provider a reconciliation request, wherein the reconciliation request includes reconciliation information received by the registered content provider from client computing devices.

4. The system as recited in claim 2, wherein the reconciliation information includes an electronic token including information unique to the client computing device.

5. The system as recited in claim 2, wherein the reconciliation information includes an electronic token including information unique to the registered computing device.

6. The system as recited in claim 3, wherein the at least one computing device is further configured to reconcile the reconciliation request from the at least one registered content provider.

7. The system as recited in claim 5, wherein at least one level in the distribution network corresponds to a datacenter.

8. The system as recited in claim 5, wherein the nodes correspond to a network router and wherein identifying a subnetwork in the distribution network for providing the requested content includes identifying a subnetwork such that content responsive to content requests between the client computing device and a content sources do not pass through the network router more than one time.

9. The system as recited in claim 1, wherein the at least one computing device is further configured to:

obtain registration requests from third party computing devices;

associate the third party computing devices as registered content providers; and

identify a subnetwork in the distribution network corresponding to each registered content provider.

10. The system as recited in claim 1, wherein the distribution network includes two or more levels defined by nodes in the network and wherein associating a distribution network into a set of subnetworks includes:

identifying a node in the distribution network as a parent node;

identifying all children registered content providers to the parent node; and

defining the subnetwork to include all the identified children registered content providers.

11. The system as recited in claim 1, wherein the set of content segments corresponds to equal sized content segments.

12. The system as recited in claim 1, wherein the set of content segments corresponds to varied sized content segments.

13. The system as recited in claim 1, wherein the set of network addresses is prioritized in an order of a preferred provider of the requested content segment.

14. A system for managing content at a content provider, the system comprising:

at least one computing device configured to;

distribute content made up of a set of content segments to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network corresponds to a two or more subnetworks and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtain a content request for content from a client computing device;

determine at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that the selected content source will minimize distribution node traffic between one or more parent nodes to the client computing device and the content source; and

provide identification information identifying the at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the identified content source.

15. The system as recited in claim 14, wherein determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device comprises determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that the selected content source will minimize distribution node traffic between one or more parent nodes to the client computing device and the content source.

16. The system as recited in claim 14, wherein determining at least one content source for providing at least a portion of the requested content includes determining a different content source for providing at least each segment in the set of content segments.

17. The system as recited in claim 14, wherein determining at least one content source for providing at least a portion of the requested content includes determining a same content source for providing every segment in the set of content segments.

18. The system as recited in claim 14, wherein determining at least one content source for providing at least a portion of the content includes identifying at least one peer computing device for providing at least a portion of the requested content.

19. The system as recited in claim 18, wherein the at least one computing device is further operable to:

obtain reconciliation requests from the peer computing device, wherein the reconciliation requests include reconciliation tokens received from client computing devices; and

reconcile the reconciliation requests from the peer computing devices.

20. A system for managing content at a content delivery network service provider, the system comprising:

at least one computing device configured to;

distribute content made up of a set of content segments to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network includes a first level corresponding to two or more content sources and a second level corresponding to datacenter nodes in the distribution network, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

determine at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that information flow is minimized through the datacenter nodes of the distribution network; and

provide identification information identifying the at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the identified content source.

21. The system as recited in claim 20, wherein determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device comprises determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that information flow is minimized through the datacenter nodes of the distribution network.

22. The system as recited in claim 20, wherein the at least one computing device is further operable to obtain a content request for content from a client computing device and wherein providing identification information identifying the at least one content source for providing at least a portion of the requested content includes transmitting the identification information in response to the client request.

23. The system as recited in claim 20, wherein determining at least one content source for providing at least a portion of the content includes determining at least one content source having a network proximity to the client computing device.

24. The system as recited in claim 23, wherein determining at least one content source having a network proximity to the client computing device includes determining at least one content source residing in a same datacenter node in the distribution network.

25. The system as recited in claim 20, wherein determining at least one content source for providing at least a portion of the requested content includes determining a different content source for providing at least each segment in the set of content segments.

26. The system as recited in claim 20, wherein determining at least one content source for providing at least a portion of the requested content includes determining a same content source providing every segment in the set of content segments.

27. The system as recited in claim 20, wherein the identification information includes a network address of a resource cache component of the content delivery network service provider.

28. The system as recited in claim 20, wherein the identification information includes a network address of a peer computing device registered with a content provider.