Delivering resources to clients in a distributed computing environment

• US 8,468,245 B2
• Filed: 05/30/2007
• Issued: 06/18/2013
• Est. Priority Date: 02/10/1998
• Status: Expired due to Fees

• Current Assignee: Sandpiper CDN, LLC (Theseus LF Asset Holdings, LLC)
• Original Assignee: Level 3 Communications LLC (Lumen Technologies, Inc.)
• Inventors: Farber, David A., Greer, Richard E., Swart, Andrew D., Balter, James A.
• Primary Examiner(s): BLAIR, DOUGLAS B
• Application Number:
  US11/806,147
• Publication Number:
  US 20070233846A1
• Time in Patent Office: 2,211 Days
• Field of Search: 709/219, 709/226
• US Class Current: 709/226
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: H04L 43/065 related to network devices
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 61/30 Managing network names, e.g. use of aliases or nicknames name-to-address mapping H04L61/45
• CPC Class Codes: H04L 61/301 Name conversion
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1006 with static server selection, e.g. the same server being selected for a specific client
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1025 : Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1036 : Load balancing of requests to servers for services different from user content provisioning, e.g. load balancing across domain name servers
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/2895 : Intermediate processing functionally located close to the data provider application, e.g. reverse proxies
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/561 : Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/563 : Data redirection of data network streams
• CPC Class Codes: H04L 67/564 : Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/61 : taking into account QoS or priority requirements
• CPC Class Codes: H04L 9/40 : Network security protocols

Abstract

A method for delivering resources to clients in a distributed computing environment. At least a first resource associated with a first content provider and maintained on an origin server references a second resource. The second resource is associated with a network formed by a plurality of repeater servers operable to serve the second resource to clients on behalf of the first content provider, the origin server being distinct from the plurality of repeater servers. Responsive to a request that causes the first resource to be served to a client from the origin server, at least one of the plurality of repeater servers is selected to serve the second resource to the client. If a copy of the second resource is available on the selected repeater server, the copy of the second resource is served to the client from the selected repeater server; otherwise, if a copy of the second resource is not available on the selected repeater server, the second resource is replicated on the selected repeater server.

Claims

1. A method for delivering resources to clients in a distributed computing environment, wherein at least a first resource associated with a first content provider and maintained on a storage device associated with an origin server references a second resource, the method comprising:

providing a network formed by a plurality of repeater servers configured to serve at least the second resource to clients on behalf of the first content provider, wherein each of the plurality of repeater servers comprises a cache storage distinct from the storage device associated with the origin server; and

responsive to a request that causes the origin server to serve the first resource to a client from the storage device associated with the origin server, selecting at least one of the plurality of repeater servers to serve the second resource to the client; and

responsive to the selected repeater server being requested to serve the second resource;

checking to determine whether a copy of the second resource is available in the cache storage of the selected repeater server;

if a copy of the second resource is available in the cache storage of the selected repeater server, serving the copy of the second resource to the client from the cache storage of the selected repeater server;

otherwise,

if a copy of the second resource is not available in the cache storage of the selected repeater server, using at least a table on the selected repeater server and information included in a request for the second resource to determine an origin server associated with the second resource, and replicating the second resource in the cache storage of the selected repeater server, wherein the replicating act comprises;

requesting a copy of the second resource from the determined origin server.

2. A method as recited in claim 1, wherein the replicating step further comprises:

requesting a copy of the second resource from a peer repeater server in the network.

3. A method as recited in claim 1, wherein the second resource is originally identified using a first uniform resource locator (URL) that comprises a first domain name, the method further comprising:

associating the second resource with at least one of the plurality of repeater servers via a second URL, wherein the second URL is at least partially distinct from the first URL and wherein at least one of the plurality of repeater servers associated with the second resource via the second URL comprises the selected repeater server.

4. A method as recited in claim 3, wherein the associating step further comprises:

forming the second URL by prepending data to the first domain name.

5. A method as recited in claim 4, wherein the first resource references the second resource at least by way of a textual reference embedded in the first resource, the method further comprising:

modifying the textual reference to comprise the second URL.

6. A method as recited in claim 5, wherein the modifying step is performed in response to initiation of the request that causes the first resource to be served to the client by the origin server on which the first resource is maintained.

7. A method as recited in claim 6, wherein further responsive to the request that causes the first resource to be served to the client by the origin server on which the first resource is maintained, the method further comprising:

transmitting to the client an HTTP REDIRECT command comprising the modified textual reference.

8. A method as recited in claim 5, wherein the modifying step is performed prior to initiation of the request that causes the first resource to be served to the client by the origin server on which the first resource is maintained.

9. A method as recited in claim 1, wherein the selecting step is performed at least in part by a software component co-located with the origin server on which the first resource is maintained.

10. A method as recited in claim 9, wherein the act of selecting comprises:

a step for determining a Group Reduction Table and a Link Cost Table, wherein said Group Reduction Table and said Link Cost Table are operable for use in identifying the selected repeater server.

11. A method as recited in claim 9, wherein the software module comprises a reflector configured to perform the selecting act.

12. A method as recited in claim 11, further comprising:

by the reflector, intercepting the request and determining that the first resource is to be served to the client by the origin server on which the first resource is maintained; and

by the reflector, sending the request to the origin server on which the first resource is maintained.

13. A method as recited in claim 1, wherein the first resource is a web page.

14. A method as recited in claim 1, wherein further responsive to the request that causes the first resource to be served to a client by the origin server on which the first resource is maintained, causing the client to be provided an address of the selected repeater server.

15. A method as recited in claim 1, wherein the network comprises additional repeater servers comprising additional cache storages distinct from the plurality of repeater servers, wherein the storage device associated with the origin server on which the first resource is maintained is further distinct from the additional cache storages.

16. A method as recited in claim 1, wherein the second resource is originally identified using a first uniform resource locator (URL) that comprises a first domain name, the method further comprising:

assigning the second resource a second URL identifying at least the selected repeater server, wherein the second URL is at least partially distinct from the first URL.

17. A method as recited in claim 16, wherein the first resource references the second resource at least by way of a textual reference embedded in the first resource, the assigning step further comprising:

forming the second URL by prepending data identifying the selected repeater server to the first domain name, and wherein the method further comprises;

modifying the textual reference to comprise the second URL.

18. A method as recited in claim 1, wherein the plurality of repeater servers are each configured to serve resources on behalf of a plurality of content providers, the plurality of content providers comprising the first content provider, wherein the act of providing comprises:

configuring each of the plurality of repeater servers with an activity logging module for use in recording an identity of each content provider associated with resources served therefrom, wherein responsive to the act of serving,

recording, by the activity logging module, information concerning delivery of at least the second resource by the selected repeater server in conjunction with at least identification of the first content provider.

19. A method as recited in claim 18, wherein the information concerning delivery of at least the first resource comprises a timestamp associated with serving the second resource by the selected repeater server.

20. A method as recited in claim 1, wherein the plurality of repeater servers are each configured to serve resources on behalf of a plurality of content providers, the plurality of content providers comprising the first content provider, wherein the act of providing comprises:

configuring the table on each of the repeater servers to comprise a listing of subscribers authorized to have resources served to clients therefrom, wherein the subscribers comprise the plurality of content providers.

21. A method as recited in claim 1, wherein the method further comprises:

by a reflector, intercepting the request and determining that the first resource is to be served to the client by the origin server on which the first resource is maintained; and

by the reflector, sending the request to the origin server on which the first resource is maintained.

22. A method as recited in claim 21, wherein the reflector is co-located with the origin server on which the first resource is maintained.

23. A method for delivering resources to clients from a network of repeater servers on behalf of content providers, wherein at least a first resource maintained on a storage device associated with an origin server references a second resource, the method comprising

configuring at least a plurality of the repeater servers in the network of repeater servers to serve at least the second resource to clients, wherein each of the repeater servers comprises a cache storage distinct from the storage device associated with the origin server;

responsive to a request that causes the origin server to serve the first resource to a client from the storage device associated with the origin server, selecting at least one of the plurality of repeater servers in the network from which to serve the second resource to the client;

responsive to the selected repeater server being requested to serve the second resource;

checking to determine whether a copy of the second resource is available in the cache storage of the selected repeater server;

if a copy of the second resource is available in the cache storage of the selected repeater server, serving the copy of the second resource to the client from the cache storage of the selected repeater server;

otherwise,

if a copy of the second resource is not available in the cache storage of the selected repeater server, using at least a table on the selected repeater server and information included in a request for the second resource to determine an origin server associated with the second resource, and replicating the second resource in the cache storage of the selected repeater server, wherein the replicating act comprises;

requesting a copy of the second resource from the determined origin server.

24. A method as recited in claim 23, wherein the second resource is originally identified using a first uniform resource locator (URL) that comprises a first domain name, the method further comprising:

associating the second resource with at least one of the plurality of repeater servers via a second URL, wherein the second URL is at least partially distinct from the first URL and wherein the plurality of repeater servers associated with the second resource via the second URL comprises the selected repeater server.

25. A method as recited in claim 24, wherein the associating step further comprises:

forming the second URL by prepending data to the first domain name.

26. A method as recited in claim 23, wherein the second resource is associated with a first content provider, wherein the plurality of repeater servers are each configured to serve resources on behalf of a plurality of content providers, the plurality of content providers comprising the first content provider, wherein the act of configuring comprises:

configuring each of the plurality of repeater servers with an activity logging module for use in recording an identity of each content provider associated with resources served therefrom, wherein responsive to the act of serving,

recording, by the activity logging module, information concerning delivery of at least the second resource by the selected repeater server in conjunction with at least identification of the first content provider.

27. A method as recited in claim 26, wherein the information concerning delivery of at least the second resource comprises a timestamp associated with serving the second resource by the selected repeater server.

28. A method as recited in claim 23, wherein the plurality of repeater servers are each configured to serve resources on behalf of the plurality of content providers, wherein the first resource and the second resource are associated with a first content providers of the plurality of content providers, wherein the act of providing comprises:

configuring the table on each of the repeater servers to comprise a listing of subscribers authorized to have resources served to clients therefrom, wherein the subscribers comprise each of the plurality of content providers.

29. A method as recited in claim 23, wherein the method further comprises:

by a reflector, intercepting the request and determining that the first resource is to be served to the client by the origin server on which the first resource is maintained; and

by the reflector, sending the request to the origin server on which the first resource is maintained.

30. A method as recited in claim 29, wherein the selecting act is performed at least in part by the reflector.

31. A method as recited in claim 29, wherein the reflector is co-located with the origin server on which the first resource is maintained.

32. A method for delivering resources to clients in a distributed computing environment, wherein at least a first resource associated with a first content provider and maintained on a storage device associated with an origin server references a second resource, the method comprising:

providing a network formed by a plurality of repeater servers configured to serve at least the second resource to clients on behalf of the first content provider, wherein each of the plurality of repeater servers comprises a cache storage distinct from the storage device associated with the origin server; and

responsive to a request that causes the origin server to serve the first resource to a client from the storage device;

selecting a repeater server in the network from which to serve the second resource to the client; and

causing the client to be provided an address of the selected repeater server;

responsive to the selected repeater server being requested to serve the second resource;

checking to determine whether a copy of the second resource is available in the cache storage of the selected repeater server;

if a copy of the second resource is available on the cache storage of the selected repeater server, serving the copy of the second resource to the client from the cache storage of the selected repeater server;

otherwise,

if a copy of the second resource is not available in the cache storage of the selected repeater server, using at least a table on the selected repeater server and information included in a request for the second resource to determine an origin server associated with the second resource, and replicating the second resource in the cache storage of the selected repeater server, wherein the replicating act comprises;

requesting a copy of the second resource from the determined origin server.

33. A method as defined in claim 32, wherein the selecting step is performed at least in part by a software component co-located with the origin server on which the first resource is maintained.

34. A method as recited in claim 33, wherein the software module comprises a reflector configured to perform the selecting act.

35. A method as recited in claim 34, further comprising:

by the reflector, intercepting the request and determining that the first resource is to be served to the client by the origin server on which the first resource is maintained; and

by the reflector, sending the request to the origin server on which the first resource is maintained.

36. A method as recited in claim 32, wherein the second resource is originally identified using a first uniform resource locator (URL) that comprises a first domain name, the method further comprising:

assigning the second resource a second URL identifying at least the selected repeater server, wherein the second URL is at least partially distinct from the first URL.

37. A method as recited in claim 36, wherein the assigning step further comprises:

forming the second URL by prepending data identifying the selected repeater server to the first domain name.

38. A method as recited in claim 37, wherein the first resource references the second resource at least by way of a textual reference embedded in the first resource, the method further comprising:

modifying the textual reference to comprise the second URL.

39. A method as recited in claim 38, wherein the modifying step is performed in response to initiation of the request that causes the first resource to be served to the client by the origin server on which the first resource is maintained.

40. A method as recited in claim 32, wherein the plurality of repeater servers are each configured to serve resources on behalf of a plurality of content providers, the plurality of content providers comprising the first content provider, wherein the act of providing comprises:

configuring each of the plurality of repeater servers with an activity logging module for use in recording an identity of each content provider associated with resources served therefrom, wherein responsive to the act of serving,

recording, by the activity logging module, information concerning delivery of at least the first resource by the particular repeater server in conjunction with at least identification of the first content provider.

41. A method as recite in claim 40, wherein the information concerning delivery of at least the first resource comprises a timestamp associated with serving the first resource by the particular repeater server.

42. A method as recited in claim 32, wherein the plurality of repeater servers are each configured to serve resources on behalf of a plurality of content providers, the plurality of content providers comprising the first content provider, wherein the act of providing comprises:

configuring the table on each of the repeater servers to comprise a listing of subscribers authorized to have resources served to clients therefrom, wherein the subscribers comprise the plurality of content providers.

43. An Internet content delivery method for delivering resources from multiple content providers to multiple clients via a network of repeater servers, said resources including web pages associated with said content providers, at least some of said web pages including references to other resources, and wherein a web page associated with a first content provider includes a reference to a first resource also associated with said first content provider, wherein the web page is maintained on a storage device associated with an origin server, the method comprising:

after said web page has been served to a requesting client from the storage device, serving the first resource to the requesting client from a particular repeater server in the network of repeater servers by;

(A) if a copy of the first resource is not cached in a cache storage associated with the particular repeater server, wherein the cached storage is distinct from the storage device associated with the origin server, said particular repeater server uses at least a table on the particular repeater server and information included in a request for the first resource to determine an origin server associated with the first resource and attempts to obtain a copy of said first resource from a peer repeater server or from the origin server determined to be associated with the first resource; and

then serving the copy of the first resource by the particular repeater server to the requesting client;

otherwise,

(B) if a copy of the first resource is cached in the cache storage, serving the copy of the first resource by the particular server to the requesting client.

44. An Internet method as in claim 43, wherein said web page is identified by a first Uniform Resource Locator (URL) having a first domain name that resolves to said origin server and wherein said first resource is identified in said web page by a second URL having a second domain name that resolves to said network of repeater servers.

45. An Internet method as in claim 44, wherein said first domain name and said second domain name are fully qualified domain names.

46. An Internet method as in claim 45, wherein said second domain name identifies said particular repeater server.

47. An Internet method as in claim 44 wherein said second URL includes said second domain name prepended to said first domain name.

48. A method as recited in claim 43, wherein each of the repeater servers are configured to serve resources on behalf of the multiple content providers, the multiple content providers comprising the first content provider, the method further comprising:

configuring the table on each of the repeater servers to comprise a listing of subscribers authorized to have resources served to clients therefrom, wherein the subscribers comprise the plurality of content providers.

49. A method as recited in claim 43, wherein each of the repeater servers are configured to serve resources on behalf of the multiple content providers, the multiple content providers comprising the first content provider, the method further comprising:

configuring each of the repeater servers with an activity logging module for use in recording an identity of each content provider associated with resources served therefrom, wherein responsive to the act of serving,

recording, by the activity logging module of the particular repeater server, information related to the serving of the copy of the first resource by the particular repeater server in conjunction with at least identification of the first content provider.

50. A method as recite in claim 49, wherein the information related to the serving of at least the first resource comprises a timestamp associated with serving the first resource by the particular repeater server.