Shared content delivery infrastructure and method of generating a web page

• US 8,572,210 B2
• Filed: 10/31/2007
• Issued: 10/29/2013
• Est. Priority Date: 02/10/1998
• Status: Expired due to Fees

• Current Assignee: Level 3 Communications LLC (Lumen Technologies, Inc.)
• Original Assignee: Level 3 Communications LLC (Lumen Technologies, Inc.)
• Inventors: Farber, David A., Greer, Richard E., Swart, Andrew D., Balter, James A.
• Primary Examiner(s): BLAIR, DOUGLAS B
• Application Number:
  US11/980,356
• Publication Number:
  US 20080140800A1
• Time in Patent Office: 2,190 Days
• Field of Search: 709/217, 709/218, 709/219, 709/245
• US Class Current: 709/219
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: H04L 43/065 related to network devices
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 61/30 Managing network names, e.g. use of aliases or nicknames name-to-address mapping H04L61/45
• CPC Class Codes: H04L 61/301 Name conversion
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1006 with static server selection, e.g. the same server being selected for a specific client
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1025 : Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1036 : Load balancing of requests to servers for services different from user content provisioning, e.g. load balancing across domain name servers
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/2895 : Intermediate processing functionally located close to the data provider application, e.g. reverse proxies
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/561 : Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/563 : Data redirection of data network streams
• CPC Class Codes: H04L 67/564 : Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/61 : taking into account QoS or priority requirements
• CPC Class Codes: H04L 9/40 : Network security protocols

Abstract

A first URL in a first document is modified to produce a first modified URL, that includes a first modified hostname resolvable to a shared content delivery network (CDN) formed by a plurality of repeater servers. A second URL in a second document is modified to produce a second modified URL that includes a hostname that is also resolvable to the shared CDN.

Claims

1. A method, in a system in which a plurality of content providers provide resources to clients, wherein each content provider provides at least some resources via one or more content sources associated with that content provider, and

wherein a first document is associated with a first content provider of said plurality of content providers, said first document comprising at least one reference to a first resource, said at least one reference comprising a first original hostname that is associated with at least a first origin server associated with said first content provider, and

wherein a second document is associated with a second content provider of said plurality of content providers, said second document including at least one reference to a second resource, said at least one reference to said second resource comprising a second original hostname that is associated with at least a second origin server associated with said second content provider,

the method comprising:

(A) providing a plurality of repeater servers forming a shared content delivery network (CDN), wherein each of the plurality of repeater servers is distinct from the first origin server and the second origin server; and

(B) configuring each of the plurality of repeater servers to serve resources on behalf of subscribers to the shared CDN, wherein the subscribers to the shared CDN comprise the first content provider and the second content provider, the configuring act comprising;

(B)(i) associating at least a first repeater server of said plurality of repeater servers with at least a first modified hostname comprising a first string of characters and at least a portion of said first original hostname; and

(B)(ii) associating at least a second repeater server of said plurality of repeater servers with a second modified hostname comprising a second string of characters and at least a portion of said second original hostname;

(C) after the first document has been served to a first client from an origin server associated with said first content provider, serving said first resource to said first client from the first repeater server in response to receipt by the first repeater server of a first request comprising an HTTP header having the first modified hostname, and

(D) after the second document has been served to a second client from an origin server associated with said second content provider, serving the second resource to the second client from the second repeater server in response to receipt by the second repeater server of a second request comprising an HTTP header having the second modified hostname.

2. A method as recited in claim 1, wherein said first document is a hypertext document associated with said first content provider, and wherein said first resource reference is an embedded object.

3. A method as recited in claim 2, wherein said second document is a hypertext document associated with said second content provider, and wherein said second resource reference is an embedded object.

4. A method as recited in claim 1, wherein said first modified hostname is at least partially distinct from said second modified hostname.

5. A method as recited in claim 1, wherein the serving act (C) comprises:

if said first resource is not available at said first repeater server, said first repeater server obtaining said first resource.

6. A method as recited in claim 5, wherein the serving act (D) comprises:

if said second resource is not available at said second repeater server, said second repeater server obtaining said second resource.

7. A method as recited in claim 5, wherein said first repeater server obtains said first resource from another repeater server or from said first origin server associated with said first content provider.

8. A method as recited in claim 7, wherein said first repeater server uses information in said first modified hostname to identify said first origin server.

9. A method as recited in claim 8, wherein said first original hostname is maintained in part of said first modified hostname, and wherein said first repeater server uses information is said first modified hostname to identify said first origin server.

10. A method as recited in claim 5, wherein said first repeater server was selected based, at least in part, on:

(i) load conditions on at least some of said repeater servers, and

(ii) a location corresponding to said first client.

11. A method as recited in claim 1, wherein said first original hostname in said first document is modified in response to a request for the first document.

12. A method as recited in claim 1, wherein, further responsive to the first document being served to the first client, an HTTP REDIRECT command comprising the first modified hostname is transmitted to the first client.

13. A method as recited in claim 1, wherein said first resource and said second resource are each selected from the group comprising static and dynamic:

video content, audio content, text, image content, web pages, HTML files, XML files, files in a markup language, documents, hypertext documents, data files, and embedded resources.

14. A method as recited in claim 1, wherein the shared CDN comprises at least a first repeater selector mechanism and a second repeater selector mechanism, the method further comprising:

by the first repeater selector mechanism, prior to the serving act (C), selecting the first repeater server to serve the first resource to the first client in response to a request by the first client for the first document; and

by the second repeater selector mechanism, prior to the serving act (D), selecting the second repeater server to serve the second resource to the second client in response to a request by the second client for the second document.

15. A method as recited in claim 14, wherein:

the step of selecting by the first repeater selector mechanism comprises a step for determining a Group Reduction Table and a Link Cost Table;

wherein said Group Reduction Table and said Link Cost Table are operable for use in selecting the first repeater server to serve the first resource to the first client and a step for redirecting the first client to the first repeater server.

16. A method as recited in claim 1, wherein the associating act (B)(i) further comprises:

associating at least the first repeater server with a third modified hostname comprising an identifier and at least the portion of the second original hostname.

17. A method as recited in claim 16, wherein the identifier comprises a third string of characters distinct from the second string of characters.

18. A method as recited in claim 1, wherein a third document is associated with a third content provider of said plurality of content providers, said third document comprising at least one reference to a third resource, said at least one reference comprising a third original hostname that is associated with at least a third origin server associated with said third content provider, wherein the third content provider is a subscriber to the shared CDN, wherein the associating act (B)(i) further comprises:

associating at least the first repeater server with a third modified hostname comprising an identifier and at least a portion of the third original hostname.

19. A method as recited in claim 1, wherein each of the plurality of repeater servers is configured to record activity related to serving resources therefrom on behalf of content providers that subscribe to the shared CDN, the method further comprising:

responsive to the act of serving the first resource, the first repeater server recording at least information related to the serving of at least the first resource on behalf of the first content provider; and

responsive to the act of serving the second resource, the second repeater server recording at least information related to the serving of at least the second resource on behalf of the second content provider.

20. A method as recited in claim 19, wherein the information recorded by the first repeater server related to the serving of at least the first resource comprises a first timestamp associated with serving the first resource to the first client, and wherein the information recorded by the second repeater server related to the serving of at least the second resource comprises a second timestamp associated with serving the second resource to the second client.

21. A method as recited in claim 20, further comprising:

collecting first information related to the serving of at least the first resource by at least some repeater servers in the shared CDN that have served the first resource during a first given period in time, the first given period in time including a first time specified in the first timestamp, said first information being derived at least in part from the recorded information related to the serving of at least the first resource to the first client; and

collecting second information related to the serving of at least the second resource by at least some repeater servers in the shared CDN that have served the second resource during a second given period in time, the second given period in time including a second time specified in the second time stamp, said second information being derived at least in part from the recorded information related to the serving of at least the second resource to the second client.

22. A method as recited in claim 1, wherein:

if said first resource is not available at said first repeater server at the time of said first request, said first repeater server obtaining said first resource using at least the first modified hostname to locate the first resource prior to serving the first resource to the first client; and

if said second resource is not available at said second repeater server at the time of said second request, said second repeater server obtaining said second resource using at least the second modified hostname to locate the second resource prior to serving the second resource to the second client.

23. A method as recited in claim 22, wherein the configuring act (B) comprises configuring each of the plurality of repeater servers with a table listing at least one origin server associated with each of the plurality of content providers that subscribe to the content delivery network, and wherein:

the act of obtaining by the first repeater server further comprises using at least the first modified hostname and the table of the first repeater server to identify an origin server from which the first resource can be obtained; and

the act of obtaining by the second repeater server further comprises using at least the second modified hostname and the table of the second repeater server to identify an origin server from which the second resource can be obtained.

24. A method as recited in claim 23, wherein:

the act of using at least the first modified hostname and the table of the first repeater server identifies the first origin server; and

the act of using at least the second modified hostname and the table of the second repeater server identifies the second origin server.

25. An Internet method for delivering resources on behalf of a plurality of content providers to clients, wherein each content provider provides at least some resources via one or more origin servers associated with that content provider, and

wherein a first web page is associated with a first content provider of said plurality of content providers, said first web page comprising at least one reference to a first resource, said first resource being associated with said first content provider, said at least one reference comprising a first original hostname that is associated with at least a first origin server associated with said first content provider, and

wherein a second web page is associated with a second content provider of said plurality of content providers, said second web page including at least one reference to a second resource, said second resource being associated with said second content provider, said at least one reference to said second resource comprising a second original hostname that is associated with at least a second origin server associated with said second content provider,

the method comprising:

(A) providing a plurality of repeater servers forming a shared content delivery network (CDN), wherein each of the plurality of repeater servers is distinct from the first origin server and the second origin server;

(B) configuring each of the plurality of repeater servers to serve resources on behalf of subscribers to the shared CDN, wherein the subscribers to the shared CDN comprise the first content provider and the second content provider, the configuring act comprising;

(B)(i) associating at least a first repeater server of said plurality of repeater servers with at least a first hostname comprising an identifier and at least a portion of the first original hostname, and

(B)(ii) associating at least a second repeater server of said plurality of repeater servers with at least a second hostname comprising an identifier and at least a portion of the second original hostname;

(C) after said first web page has been served to a first client from an origin server of said at least a first origin server associated with the first content provider, and responsive to resolution of the first hostname to identify at least the first repeater server, serving the first resource to the first client from the first repeater server in response to receipt by the first repeater server of a first request comprising an HTTP header having the first hostname; and

(D) after said second web page has been served to a second client from an origin server of said at least a second origin server associated with the second content provider, and responsive to resolution of the second hostname to identify at least the second repeater server, serving the second resource to the second client from the second repeater server in response to receipt by the second repeater server of a second request comprising an HTTP header having the second hostname.

26. A method as recited in claim 25, further comprising:

if said first resource is not available at said first repeater server, said first repeater server obtaining said first resource from said first origin server prior to serving said first resource to said first client.

27. A method as recited in claim 25, wherein resolution of the first hostname to identify at least the first repeater server is responsive to the first client requesting delivery of the first web page.

28. A method as recited in claim 27, wherein the first web page is served to the first client from an origin server associated with said first content provider.

29. A method as recited in claim 25, wherein the shared CDN comprises at least a first repeater selector mechanism and a second repeater selector mechanism, the method further comprising:

by the first repeater selector mechanism, prior to the serving act (C), selecting the first repeater server to serve the first resource to the first client in response to a request by the first client for the first web page; and

by the second repeater selector mechanism, prior to the serving act (D), selecting the second repeater server to serve the second resource to the second client in response to a request by the second client for the second web page.

30. A method as recited in claim 29, wherein:

the step of selecting by the first repeater selector mechanism comprises a step for determining a Group Reduction Table and a Link Cost Table;

wherein said Group Reduction Table and said Link Cost Table are operable for use in selecting the first repeater server to serve the first resource to the first client and a step for redirecting the first client to the first repeater server.

31. A method as recited in claim 25, wherein the associating act (B)(i) further comprises:

associating at least the first repeater server with a third modified hostname comprising an identifier and at least the portion of the second original hostname.

32. A method as recited in claim 31, wherein the identifier comprises a third string of characters distinct from the second string of characters.

33. A method as recited in claim 25, wherein a third document is associated with a third content provider of said plurality of content providers, said third document comprising at least one reference to a third resource, said at least one reference comprising a third uniform resource locator (URL) identifying said third resource, said third URL including a third original hostname that is resolvable to at least a third origin server associated with said third content provider, wherein the third content provider is a subscriber to the shared CDN, wherein the associating act (B)(i) further comprises:

associating at least the first repeater server with a third modified hostname comprising an identifier and at least a portion of the third original hostname.

34. A method as recited in claim 25, wherein each of the plurality of repeater servers is configured to record activity related to serving resources therefrom on behalf of content providers that subscribe to the shared CDN, the method further comprising:

responsive to the act of serving the first resource, the first repeater server recording at least information related to the serving of at least the first resource on behalf of the first content provider; and

responsive to the act of serving the second resource, the second repeater server recording at least information related to the serving of at least the second resource on behalf of the second content provider.

35. A method as recited in claim 34, wherein the information recorded by the first repeater server related to the serving of at least the first resource comprises a first timestamp associated with serving the first resource to the first client, and wherein the information recorded by the second repeater server related to the serving of at least the second resource comprises a second timestamp associated with serving the second resource to the second client.

36. A method as recited in claim 35, further comprising:

collecting first information related to the serving of at least the first resource by at least some repeater servers in the shared CDN that have served the first resource during a first given period in time, the first given period in time including a first time specified in the first timestamp, said first information being derived at least in part from the recorded information related to the serving of at least the first resource to the first client; and

collecting second information related to the serving of at least the second resource by at least some repeater servers in the shared CDN that have served the second resource during a second given period in time, the second given period in time including a second time specified in the second timestamp, said second information being derived at least in part from the recorded information related to the serving of at least the second resource to the second client.

37. An method for delivering resources on behalf of a plurality of content providers to multiple clients via a shared content delivery network (CDN) comprising a plurality of repeater servers, said resources being selected from the group comprising static and dynamic:

video content, audio content, text, image content, web pages, HTML files, XML files, files in a markup language, documents, hypertext documents, data files, and embedded resources,

wherein each content provider provides at least some resources via one or more origin servers associated with that content provider, and

wherein a first hypertext document is associated with a first content provider of said plurality of content providers, said first hypertext document comprising including at least one reference to a first resource, said first resource being associated with said first content provider, said at least one reference comprising a first uniform resource locator (URL) identifying said first resource, said first URL including a first original hostname that is associated with at least a first origin server associated with said first content provider, and

wherein a second hypertext document is associated with a second content provider of said plurality of content providers, said second hypertext document including at least one reference to a second resource, said second resource being associated with said second content provider, said at least one reference to said second resource comprising a second URL identifying said second resource, said second URL including a second original hostname that is associated with at least a second origin server associated with said second content provider,

the method comprising;

(A) providing said shared CDN, wherein each of the plurality of repeater servers in said shared CDN is distinct from the first origin server and the second origin server; and

(B) configuring each of the plurality of repeater servers to serve resources on behalf of subscribers to the shared CDN, wherein the subscribers to the shared CDN comprise the first content provider and the second content provider, the configuring act comprising;

(B)(i) associating at least a first repeater server of said plurality of repeater servers with at least a first hostname comprising an identifier and at least a portion of the first original hostname; and

(B)(ii) associating at least a second repeater server of said plurality of repeater servers with at least-a second hostname comprising an identifier and at least a portion of the second original hostname;

(C) after said first hypertext document has been served to a first client from said first origin server associated with the first content provider, and responsive to resolution of the first hostname to identify at least the first repeater server, serving the first resource to the first client from the first repeater server in response to receipt by the first repeater server of a first request comprising an HTTP header having the first hostname; and

(D) after said second hypertext document has been served to a second client from said second origin server associated with the second content provider, and responsive to resolution of the second hostname to identify at least the second repeater server, serving the second resource to a second client from the second repeater server in response to receipt by the second repeater server of a second request comprising an HTTP header having the second hostname.

38. A method as recited in claim 37, further comprising:

if said first resource is not available at said first repeater server, said first repeater server obtaining said first resource from said first origin server prior to serving said first resource to said first client.

39. A repeater server for use in a content delivery network to which a plurality of content providers subscribe to have resources delivered to clients on their behalf,

wherein each content provider provides at least some resources via one or more origin servers associated with that content provider, and

wherein a first document is associated with a first content provider of said plurality of content providers, said first document comprising at least one reference to a first resource stored on at least a first origin server associated with the first content provider, and

wherein a second document is associated with a second content provider of said plurality of content providers, said second document including at least one reference to a second resource stored at least on a second origin server associated with the second content provider,

wherein the repeater server comprises:

a cache device; and

a table listing at least a plurality of names of subscribers to the content delivery network, wherein the listing of names comprises at least a first name associated with the first content provider and a second name associated with the second content provider, wherein said table associates the first name with at least the first origin server and the second name with at least the second origin server;

wherein in response to receipt by said repeater server of a first request for the first resource, the first request comprising at least the first name associated with the first content provider, the repeater server is operable to;

(i) reference the table using at least the first name from the first request to determine that the first origin server is associated therewith and replicate the first resource on the cache device from the first origin server if the first resource is not already contained on the cache device;

(ii) serve the first resource in response to the first request; and

(iii) record information related to the serving of at least the first resource therefrom; and

wherein in response to receipt by said repeater server of a second request for the second resource, the second request comprising at least the second name associated with the second content provider, the repeater server is further operable to;

(i) reference the table using at least the second name from the second request to determine that the second origin server is associated therewith and replicate the second resource on the cache device from the second origin server if the second resource is not already contained on the cache device;

(ii) serve the second resource in response to the second request; and

(iii) record information related to the serving of at least the second resource.

40. A repeater server as recited in claim 39, wherein the first request comprises an HTTP header containing the first name, and wherein the second request comprises an HTTP header containing the second name.

41. A method as recited in claim 39, wherein the repeater server is operable to selectively redirect requests away from the repeater server in order to reduce load on the repeater server.

42. A method as recited in claim 41, wherein the repeater server is further operable to select requests for redirecting away from the repeater server based at least in part on an aggregate information rate associated with at least some of the plurality of content providers.

43. A repeater server for use in a content delivery network to which a plurality of content providers subscribe to have resources delivered to clients on their behalf, wherein each content provider provides at least some resources via one or more origin servers associated with that content provider, and

wherein a first document is associated with a first content provider of said plurality of content providers, said first document comprising at least one reference to a first resource stored on at least a first origin server of said first content provider, and

wherein a second document is associated with a second content provider of said plurality of content providers, said second document including at least one reference to a second resource stored at least on a second origin server associated with said second content provider,

wherein the repeater server comprises:

a cache device;

a table listing at least a plurality of names of subscribers to the content delivery network, wherein the listing of names comprises at least a first name associated with the first content provider and a second name associated with the second content provider, wherein the table associates the first name with at least the first origin server and the second name with at least the second origin server;

wherein in response to receipt of a first request for the first resource, the first request comprising at least the first name, the repeater server is operable to;

(i) reference the table using at least the first name from the first request to determine that the first origin server is associated therewith and obtain the first resource from the first origin server if the first resource is not already contained on the cache device; and

(ii) serve the first resource in response to the first request; and

wherein in response to receipt of a second request for the second resource, the second request comprising at least the second name, the repeater server is further operable to;

(i) reference the table using at least the second name from the second request to determine that the second origin server is associated therewith and obtain the second resource from the second origin server if the first resource is not already contained on the cache device; and

(ii) serve the second resource in response to the second request.

44. A repeater server as recited in claim 43, wherein the first request comprises an HTTP header containing the first name, and wherein the second request comprises an HTTP header containing the second name.

45. A method as recited in claim 43, wherein the repeater server is operable to selectively redirect requests away from the repeater server in order to reduce load on the repeater server.

46. A method as recited in claim 45, wherein the repeater server is further operable to select requests for redirecting away from the repeater server based at least in part on an aggregate information rate associated with at least some of the plurality of content providers.