Locality based content distribution

• US 7,970,820 B1
• Filed: 03/31/2008
• Issued: 06/28/2011
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Richardson, David R., Marshall, Bradley E., Sivasubramanian, Swaminathan
• Primary Examiner(s): Jacobs; Lashonda T
• Application Number:
  US12/060,124
• Time in Patent Office: 1,184 Days
• Field of Search: 709/231, 709/223, 709/224, 709/229, 709/203, 709/217, 709/219
• US Class Current: 709/203
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1082 involving incentive schemes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/562 Brokering proxy services

Abstract

A system and method for content distribution are provided. A content provider generates a network topology having one or more subnetworks made up of content sources. The content provider segments content, such as applications, into segments for distribution. The content provider then distributes the content such that for each identified subnetwork, each content segment is distributed to a content source. Subsequent, client computing device requests for content can be serviced by selecting a content source in a particular subnetwork selected for delivering the requested content to the client computing device to minimize traffic flow of the requested content through the nodes of the distribution network.

Claims

1. A method for managing content comprising:

associating content, by a content provider, into a set of content segments;

associating a distribution network into a set of subnetworks, wherein each subnetwork includes one or more content sources;

distributing the content so that the set of content segments is distributed among the one or more content sources in each subnetwork in the distribution network;

obtaining a request for the content from a client computing device;

identifying a subnetwork in the distribution network for providing the requested content based on a network locality of the client computing device relative to subnetworks in the distribution network;

for each content segment in the set of content segments, identifying at least one or more content sources in the identified subnetwork corresponding to a registered content provider for providing the content segment;

providing identification information responsive to the request for content from the client computing device, wherein the identification information includes, for each content segment, network addresses of the identified registered content provider and reconciliation information to be provided to the identified registered content provider;

obtaining from at least one registered content provider a reconciliation request, wherein the reconciliation request includes reconciliation information received by the registered content provider from client computing devices; and

reconciling the reconciliation request from the at least one registered content provider.

2. The method as recited in claim 1 further comprising:

obtaining registration requests from third party computing devices;

associating the third party computing devices as registered content providers; and

identifying a subnetwork in the distribution network corresponding to each registered content provider.

3. The method as recited in claim 1, wherein the distribution network includes two or more levels defined by nodes in the network and wherein associating a distribution network into a set of subnetworks includes:

identifying a node in the distribution network as a parent node;

identifying all children registered content providers to the parent node; and

defining the subnetwork to include all the identified children registered content providers.

4. The method as recited in claim 3, wherein at least one level in the distribution network corresponds to a datacenter.

5. The method as recited in claim 3, wherein the nodes correspond to a network router and wherein identifying a subnetwork in the distribution network for providing the requested content includes identifying a subnetwork such that content responsive to content requests between the client computing device and a content sources do not pass through the network router more than one time.

6. The method as recited in claim 1, wherein the set of content segments corresponds to equal sized content segments.

7. The method as recited in claim 1, wherein the set of content segments corresponds to varied sized content segments.

8. The method as recited in claim 1, wherein the reconciliation information includes an electronic token including information unique to the client computing device.

9. The method as recited in claim 1, wherein the reconciliation information includes an electronic token including information unique to the registered computing device.

10. The method as recited in claim 1, wherein the set of network addresses is prioritized in an order of a preferred provider of the requested content segment.

11. A computer readable medium having computer executable instructions for performing the method recited in claim 1.

12. A method for managing content comprising:

distributing content made up of a set of content segments, by a content provider, to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network corresponds to a two or more subnetworks, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtaining a content request for content from a client computing device;

determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that the selected content source will minimize distribution node traffic between one or more parent nodes to the client computing device and the content source; and

providing identification information identifying the at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the identified content source.

13. The method as recited in claim 12, wherein determining at least one content source for providing at least a portion of the requested content includes determining a different content source for providing at least each segment in the set of content segments.

14. The method as recited in claim 12, wherein determining at least one content source for providing at least a portion of the requested content includes determining a same content source for providing every segment in the set of content segments.

15. The method as recited in claim 12, wherein determining at least one content source for providing at least a portion of the content includes identifying at least one peer computing device for providing at least a portion of the requested content.

16. The method as recited in claim 15 further comprising:

obtaining reconciliation requests from the peer computing device, wherein the reconciliation requests includes reconciliation tokens received from client computing devices; and

reconciling the reconciliation requests from the peer computing devices.

17. A method for managing content comprising:

distributing content made up of a set of content segments, by a content provider, to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network includes a first level corresponding to two or more content sources and a second level corresponding to datacenter nodes in the distribution network, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

determining at least one content source from a distribution subnetwork for providing at least a portion of the requested content to the client computing device such that information flow is minimized through the datacenter nodes of the distribution network; and

providing identification information identifying the at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the identified content source.

18. The method as recited in claim 17 further comprising obtaining a content request, by a content delivery network service provider, for content from a client computing device, and wherein providing identification information identifying the at least one content source for providing at least a portion of the requested content includes transmitting the identification information in response to the client request.

19. The method as recited in claim 17, wherein determining at least one content source for providing at least a portion of the content includes determining at least one content source having a network proximity to the client computing device.

20. The method as recited in claim 17, wherein determining at least one content source having a network proximity to the client computing device includes determining at least one content source residing in a same datacenter node in the distribution network.

21. The method as recited in claim 17, wherein determining at least one content source for providing at least a portion of the requested content includes determining a different content source for providing at least each segment in the set of content segments.

22. The method as recited in claim 17, wherein determining at least one content source for providing at least a portion of the requested content includes determining a same content source providing every segment in the set of content segments.

23. The method as recited in claim 17, wherein the identification information includes a network address of a resource cache component of a content delivery network service provider.

24. The method as recited in claim 17, wherein the identification information includes a network address of a peer computing device registered with a content provider.