Locality based content distribution

• US 8,275,874 B2
• Filed: 11/14/2011
• Issued: 09/25/2012
• Est. Priority Date: 03/31/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sivasubramanian, Swaminathan, Richardson, David R., Marshall, Bradley E.
• Primary Examiner(s): JACOBS, LASHONDA T
• Application Number:
  US13/296,189
• Publication Number:
  US 20120102099A1
• Time in Patent Office: 316 Days
• Field of Search: 709/231, 709/223, 709/224, 709/229, 709/203, 709/217, 709/219
• US Class Current: 709/223
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1082 involving incentive schemes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/562 Brokering proxy services

Abstract

A system and method for content distribution are provided. A content provider generates a network topology having one or more subnetworks made up of content sources. The content provider segments content, such as applications, into segments for distribution. The content provider then distributes the content such that for each identified subnetwork, each content segment is distributed to a content source. Subsequent, client computing device requests for content can be serviced by selecting a content source in a particular subnetwork selected for delivering the requested content to the client computing device to minimize traffic flow of the requested content through the nodes of the distribution network.

Claims

1. A system for managing content at a content provider, the system comprising:

at least one computing device configured to;

associate a distribution network into a set of subnetworks, wherein each subnetwork includes one or more content sources;

distribute content so that the content is distributed among the one or more content sources in each subnetwork in the distribution network;

obtain a request for the content from a client computing device;

identify a subnetwork in the distribution network for providing the requested content based on a network locality of the client computing device relative to subnetworks in the distribution network;

identify at least one or more content sources in the identified subnetwork corresponding to a registered content provider for providing the content; and

provide identification information responsive to the request for content from the client computing device, wherein the identification information includes a network address of the identified content source.

2. The system as recited in claim 1, wherein the identification information further includes reconciliation information to be provided to the identified content source.

3. The system as recited in claim 2, wherein the at least one computing device is further configured to obtain from the content source a reconciliation request, wherein the reconciliation request includes reconciliation information received by the identified content source from client computing devices.

4. The system as recited in claim 3, wherein the at least one computing device is further configured to reconcile the reconciliation request from the identified content source.

5. The system as recited in claim 2, wherein the reconciliation information includes an electronic token including information unique to the client computing device.

6. The system as recited in claim 2, wherein the reconciliation information includes an electronic token including information unique to the identified content source.

7. The system as recited in claim 1, wherein the at least one computing device is further configured to:

obtain registration requests from third party computing devices;

associate the third party computing devices as registered content providers; and

identify a subnetwork in the distribution network corresponding to each registered content provider.

8. The system as recited in claim 1, wherein the distribution network includes two or more levels defined by nodes in the network and wherein associating a distribution network into a set of subnetworks includes:

identifying a node in the distribution network as a parent node;

identifying all children registered content providers to the parent node; and

defining the subnetwork to include all the identified children registered content providers.

9. The system as recited in claim 8, wherein at least one level in the distribution network corresponds to a datacenter.

10. The system as recited in claim 8, wherein the nodes correspond to a network router and wherein identifying a subnetwork in the distribution network for providing the requested content includes identifying a subnetwork such that content responsive to content requests between the client computing device and a content source do not pass through the network router more than one time.

11. A system for managing content at a content provider, the system comprising:

at least one computing device configured to;

distribute content to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network corresponds to two or more subnetworks and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtain a content request for content from a client computing device;

determine a content source from a subnetwork for providing at least a portion of the requested content to the client computing device such that the determined content source will minimize distribution node traffic between one or more parent nodes to the client computing device and the content source; and

provide identification information identifying the determined content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the determined content source.

12. The system as recited in claim 11, wherein determining a content source for providing at least a portion of the content includes identifying at least one peer computing device for providing at least a portion of the requested content.

13. The system as recited in claim 12, wherein the at least one computing device is further operable to:

obtain reconciliation requests from the peer computing device, wherein the reconciliation requests include reconciliation tokens received from client computing devices; and

reconcile the reconciliation requests from the peer computing devices.

14. A system for managing content at a content provider, the system comprising:

at least one computing device configured to;

distribute content to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network includes a first level corresponding to two or more content sources and a second level corresponding to datacenter nodes in the distribution network, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtain a content request for content from a client computing device;

determine at least one content source from a subnetwork for providing at least a portion of the requested content to the client computing device such that information flow is minimized through the datacenter nodes of the distribution network; and

provide identification information identifying the determined at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the determined at least one content source.

15. The system as recited in claim 14, wherein providing identification information identifying the at least one content source for providing at least a portion of the requested content includes transmitting the identification information in response to the content request.

16. The system as recited in claim 14, wherein determining at least one content source for providing at least a portion of the content includes determining at least one content source having a network proximity to the client computing device.

17. The system as recited in claim 16, wherein determining at least one content source having a network proximity to the client computing device includes determining at least one content source residing in a same datacenter node in the distribution network.

18. The system as recited in claim 14, wherein the identification information includes a network address of a resource cache component of the content provider.

19. The system as recited in claim 14, wherein the identification information includes a network address of a peer computing device registered with the content provider.

20. A method for managing content comprising:

associating a distribution network, by a content provider, into a set of subnetworks, wherein each subnetwork includes one or more content sources;

distributing the content so that the content is distributed among the one or more content sources in each subnetwork in the distribution network;

obtaining a request for the content from a client computing device;

identifying a subnetwork in the distribution network for providing the requested content based on a network locality of the client computing device relative to subnetworks in the distribution network;

identifying at least one or more content sources in the identified subnetwork corresponding to a registered content provider for providing the requested content; and

providing identification information responsive to the request for content from the client computing device, wherein the identification information includes a network address of the identified at least one or more content sources.

21. The method as recited in claim 20, wherein the identification information further includes reconciliation information to be provided to the identified at least one or more content sources.

22. The method as recited in claim 20 further comprising:

obtaining from the content source a reconciliation request, wherein the reconciliation request includes reconciliation information received by the identified at least one or more content sources from client computing devices.

23. The method as recited in claim 22 further comprising:

reconciling the reconciliation request from the identified at least one or more content sources.

24. The method as recited in claim 20 further comprising:

obtaining registration requests from third party computing devices;

associating the third party computing devices as registered content providers; and

identifying a subnetwork in the distribution network corresponding to each registered content provider.

25. The method as recited in claim 20, wherein the distribution network includes two or more levels defined by nodes in the network and wherein associating a distribution network into a set of subnetworks includes:

identifying a node in the distribution network as a parent node;

identifying all children registered content providers to the parent node; and

defining the subnetwork to include all the identified children registered content providers.

26. The method as recited in claim 25, wherein at least one level in the distribution network corresponds to a datacenter.

27. The method as recited in claim 25, wherein the nodes correspond to a network router and wherein identifying a subnetwork in the distribution network for providing the requested content includes identifying a subnetwork such that content responsive to content requests between the client computing device and a content source do not pass through the network router more than one time.

28. A method for managing content comprising:

distributing content, by a content provider, to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network corresponds to a two or more subnetworks, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtaining a content request for content from a client computing device;

determining at least one content source from a subnetwork for providing at least a portion of the requested content to the client computing device such that the determined at least one content source will minimize distribution node traffic between one or more parent nodes to the client computing device and the content source; and

providing identification information identifying the determined at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the determined at least one content source.

29. The method as recited in claim 28, wherein determining at least one content source for providing at least a portion of the content includes identifying at least one peer computing device for providing at least a portion of the requested content.

30. The method as recited in claim 29 further comprising:

obtaining reconciliation requests from the peer computing device, wherein the reconciliation requests includes reconciliation tokens received from client computing devices; and

reconciling the reconciliation requests from the peer computing devices.

31. A method for managing content comprising:

distributing content, by a content provider, to one or more content sources in each defined subnetwork of a distribution network, wherein the distribution network includes a first level corresponding to two or more content sources and a second level corresponding to datacenter nodes in the distribution network, and wherein each subnetwork corresponds to two or more logically grouped nodes within the distribution network;

obtaining a content request, by the content provider, for content from a client computing device;

determining at least one content source from a subnetwork for providing at least a portion of the requested content to the client computing device such that information flow is minimized through the datacenter nodes of the distribution network; and

providing identification information identifying the at least one content source for providing at least a portion of the requested content, wherein the identification information includes a network address of the identified content source.

32. The method as recited in claim 31, wherein providing identification information identifying the at least one content source for providing at least a portion of the requested content includes transmitting the identification information in response to the content request.

33. The method as recited in claim 31, wherein determining at least one content source for providing at least a portion of the content includes determining at least one content source having a network proximity to the client computing device.

34. The method as recited in claim 33, wherein determining at least one content source having a network proximity to the client computing device includes determining at least one content source residing in a same datacenter node in the distribution network.

35. The method as recited in claim 31, wherein the identification information includes a network address of a resource cache component of the content provider.

36. The method as recited in claim 31, wherein the identification information includes a network address of a peer computing device registered with the content provider.