Internet content delivery network

• US 6,654,807 B2
• Filed: 12/06/2001
• Issued: 11/25/2003
• Est. Priority Date: 02/10/1998
• Status: Expired due to Term

• Current Assignee: Level 3 Communications LLC (Lumen Technologies, Inc.)
• Original Assignee: Cable & Wireless Internet Services, Inc. (Cyxtera Technologies, Inc.)
• Inventors: Greer, Richard E., Farber, David A., Swart, Andrew D., Balter, James A.
• Primary Examiner(s): Geckil, Mehmet B.
• Application Number:
  US10/004,430
• Publication Number:
  US 20020049857A1
• Time in Patent Office: 719 Days
• Field of Search: 709/225, 709/226, 709/229
• US Class Current: 709/225
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: H04L 43/065 related to network devices
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 61/30 Managing network names, e.g. use of aliases or nicknames name-to-address mapping H04L61/45
• CPC Class Codes: H04L 61/301 Name conversion
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1006 with static server selection, e.g. the same server being selected for a specific client
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1019 Random or heuristic server selection
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1025 : Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1036 : Load balancing of requests to servers for services different from user content provisioning, e.g. load balancing across domain name servers
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/2895 : Intermediate processing functionally located close to the data provider application, e.g. reverse proxies
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/561 : Adding application-functional data or data for application control, e.g. adding metadata
• CPC Class Codes: H04L 67/563 : Data redirection of data network streams
• CPC Class Codes: H04L 67/564 : Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/568 : Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/61 : taking into account QoS or priority requirements
• CPC Class Codes: H04L 9/40 : Network security protocols

Abstract

Resource requests made by clients of origin servers in a network are intercepted by reflector mechanisms and selectively reflected to other servers called repeaters. The reflectors select a best repeater from a set of possible repeaters and redirect the client to the selected best repeater. The client then makes the request of the selected best repeater. The resource is possibly rewritten to replace at least some of the resource identifiers contained therein with modified resource identifiers designating the repeater instead of the origin server.

Claims

1. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, wherein the plurality of subscribers are entities that publish information via one or more origin servers, the origin servers being distinct from the plurality of repeater servers, and in which at least some of the repeater servers replicate some or all of the information available on the origin servers, (c) a repeater selector mechanism constructed and adapted to identify, for a particular client request, an appropriate repeater server from the plurality of repeater servers, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, the method comprising:

obtaining a client request for information, wherein the client request is for a resource which is embedded in another document;

identifying, by the repeater selector mechanism, a repeater server of the repeater server network to handle the client request;

determining, using at least the subscriber verifying mechanism, whether the requested information is from any one of the plurality of subscribers to the repeater server network; and

providing, from the repeater server identified by the repeater selector mechanism, the requested information when the client request is determined to be for information from one of the plurality of subscribers to the repeater server network.

2. A method as in claim 1 further comprising:

rejecting the client request when it is not for information from any one of the plurality of subscribers.

3. A method as in claim 2, further comprising:

when the client request is for information from a subscriber being the one of the plurality of subscribers,

determining whether the requested information is cached locally at the repeater server, and, based, at least in part, on said determining, if the requested information is cached locally, retrieving the requested information; and

when the requested information is determined not to be cached locally, obtaining the requested information either from an origin server of the subscriber from a peer cache.

4. A method as in claim 3, further comprising:

upon obtaining the requested information, caching the requested information locally.

5. A method as in claim 4 further comprising:

constructing a reply including the requested information; and

sending the reply to a client initiating the client request.

6. A method as in claim 5 further comprising:

recording details of a transaction with the repeater server network.

7. A method as in claim 6 wherein the details include one or more of:

(a) a current time, (b) an address of a requester, (c) a Uniform Resource Locator (URL) requested, (d) a type of response generated by the repeater server, (e) whether the client request was served by the repeater server from the cache;

(f) whether the client request was served by the repeater server alter filling a cache;

(g) whether a request pending invalidation was served by the repeater server;

(h) a duration indicative of the time required to serve the resource, and (i) bandwidth used to serve the resource.

35. A method as in claim 1 wherein, if the repeater server fails, another repeater server of the plurality of repeater servers will take over the role of the failed repeater server.

36. A method as in any one of claims 1-9, 11-21 wherein the repeater selector mechanism identifies the repeater server based, at least in part, on the load on at least some of the repeater servers.

37. A method as in any one of claims 1-9, 1-21 wherein the repeater selector mechanism identifies the repeater server based, at least in part, on a location on the network of a client sending the client request.

38. A method as in claim 1, wherein the repeater selector mechanism includes a network map for use in directing the client request for information.

39. A method as in claim 1 wherein the repeater selector mechanism is co-located wit one of the one or more origin servers.

40. A method as in any one of claims 1-9, 11-21 wherein the repeater selector mechanism identifies the repeater server based, at least in part, on at least (a) a load on at least some of the plurality of repeater servers forming the repeater server network, and (b) a location of a client sending The client request.

41. A method as in any one of claims 1-9, 11-21 wherein the repeater selector mechanism identifies the repeater server based, at least in part, on a relative cost of transmitting requested information between the repeater server and a client sending the client request.

42. A method as in any of claims 1-9, 11-21, wherein the other document is an HTML (Hyper Text Markup Language) document or an XML document which was served from the origin server.

43. A method as in any of claims 1-9, 11-21, wherein the document in which the client request was embedded was served from the origin server.

44. A method as in claim 42 wherein the requested resource is a video stream.

45. A method as in any of claims 1-9, 11-21, 22-25, wherein the subscriber verifying mechanism is located at a repeater server.

46. A method as in any claims 1-9, 11-21, 22-25, wherein the subscriber verifying mechanism comprises a table used to verify that an origin server belongs to a known subscriber.

8. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, the plurality of subscribers being entities that publish information via one or more origin servers, and in which the origin servers are distinct from the plurality of repeater servers, and in which at least some of the plurality of repeater servers replicate some or all of the information available on at least some of the origin servers, (c) a repeater selector mechanism constructed and adapted to identify, for a particular client request, an appropriate repeater server from the plurality of repeater servers, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, method comprising:

obtaining a client request for information by a repeater server of the plurality of repeater servers forming the repeater server network, the repeater server being identified by the repeater selector mechanism, wherein the client request is for a resource which is embedded in another document;

determining, using at least the subscriber verifying mechanism and based, at least in part, on a name by which the repeater server was addressed, whether the requested information is from any one of the plurality of entities that publish information to the repeater server network; and

when the client request is determined to be for information from one of the plurality of entities that publish information to the repeater server network, serving the requested information from the repeater sewer as identified by the repeater selector mechanism.

10. A method as in any one of claims 8 and 9, the determining further comprises:

using at least information in an Hypertext Transfer Protocol (HTTP) header of the client request to determine a name by which the repeater server was addressed.

9. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, the plurality of subscribers being entities that publish information via one or more origin servers, and in which the origin servers are distinct from the plurality of repeater servers, and in which at least some of the plurality of repeater servers replicate some or all of the information available on at least some of the origin servers, (c) a repeater selector mechanism constructed and adapted to identify, for a particular client request, an appropriate repeater server from the plurality of repeater servers, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, the method comprising:

obtaining a client request for information, the client request being for a resource which is embedded in another document which is sewed by an origin server of the one or more origin servers;

identifying, by the repeater selector mechanism, a repeater server of the repeater server network to handle the client request;

determining, using at least the subscriber verifying mechanism. and based, at least in part, on an origin sewer name in a Uniform Resource Locator (URL) associated with the client request, whether the requested information is from any one of the plurality of entities that publish information to the repeater server network; and

when the client request is determined to be for information from one of the plurality of entities that publish information to the repeater server network, serving the requested information from the repeater server as identified by the repeater selector mechanism.

11. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, wherein the plurality of subscribers are entities that publish information via at least one origin server, wherein the at least one origin server is distinct from the plurality of repeater servers, and in which at least some of the plurality of repeater servers replicate some or all of the information available on the at least one origin server, (c) a repeater selector mechanism constructed and adapted to identify, for a client request, a repeater server from the plurality of repeater servers to handle the client request, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, the method comprising, at one of the plurality of repeater servers:

(A) obtaining a client request for information from a client by the repeater server of the plurality of repeater servers forming the repeater server network, the repeater server having been identified by the repeater selector mechanism, the client request being for a resource which is embedded in another document;

(B) determining, using at least the subscriber verifying mechanism, whether the requested information is from a subscriber of the plurality of subscribers, said determining being based, at least in part, on at least one of (a) a name by which the repeater server was addressed, and (b) an origin server name in a Uniform Resource Locator (URL) used to make the client request;

(C) when the client request is determined to be for information from a subscriber to the repeater server network, then at the repeater server selected by the repeater selector mechanism;

(C1) if is determined that the requested information is cached locally at the repeater server, retrieving the requested information from a local cache at the repeater server; and

(C2) when the requested information is determined not to be cached locally at the repeater server,

(C21) obtaining the requested information from the origin server or from a peer cache;

(C22) caching the information locally at the repeater server; and

(C3) constructing a reply including the requested information; and

(C4) sending the reply to the client; and

(D) when the client request is determined not to be for information from a subscriber to the repeater server network, rejecting the client request.

12. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, wherein the plurality of subscribers are entities that publish resources via at least one origin server, and wherein the at least one origin server is distinct from the plurality of repeater servers, (c) a repeater selector mechanism constructed and adapted to identify, for a particular request, a repeater server to handle the request, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is an one of the plurality of subscribers to the repeater server network, the method comprising:

on at least some of the repeater sewers in the repeater server network, replicating some or all of the information available on the at least one origin server;

upon receipt of a client request for information, the client request being for a resource which is embedded in another document, determining, using at least the subscriber verifying mechanism, whether the client request is for information from a known entity that publishes resources to the repeater server network; and

serving the information, from the repeater server identified by the repeater selector mechanism, when the client request is determined to be for information from one of the plurality of subscribers being an entity that publishes resources to the repeater server network.

13. A method as in claim 12 further comprising:

rejecting the client request when it is not for information from a subscriber.

14. A method as in claim 13, further comprising:

if the client request is for information from a subscriber,

if the requested information is cached locally, retrieving the requested information from a cache;

otherwise

obtaining the requested information from the at least one origin server or from a per cache.

15. A method as in claim 14 further comprising:

upon obtaining the requested information, caching the information locally.

16. A method as in claim 15 further comprising:

constructing a reply including the requested information, and

sending the reply to a client issuing the client request.

17. A method as in claim 16 further comprising:

recording details about a transaction between the client and the repeater server network.

18. A method as in claim 17 wherein the details include one or more of:

(a) a current time, (b) an address of a requester, (c) a Uniform Resource Locator GIRL) requested, (d) a type of response generated, (e) whether the client request was served by a repeater server from the cache;

(1) whether the client request was served by a repeater server after filling a cache;

(g) whether a request pending invalidation was served by a repeater server;

(h) a duration indicative of the time required to serve the resource, and (i) bandwidth used to serve the resource.

19. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of sub scribers to the repeater server network, wherein the plurality of subscribers are entities that publish resources via one or more origin servers, wherein the origin servers are distinct from the plurality of repeater servers, (c) a repeater selector mechanism constructed and adapted to identify, for a client request, a repeater server from the plurality of repeater servers to handle the client request, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is an one of the plurality of subscribers to the repeater server network, the method comprising:

on at least some of the plurality of repeater servers in the repeater sewer network, replicating some or all of the information available on the one or more origin servers;

identifying, by the repeater selector mechanism, a repeater server of the repeater server network to handle the client request;

upon receipt of the client request for information, the client request being for a resource which is embedded in another document, determining, by the subscriber verifying mechanism, and based, at least in part, on a name by which the repeater server in receipt of the client request was addressed, whether the client request is for information from a known entity that publishes resources to the repeater server network; and

serving the information from the repeater server selected by the repeater selector mechanism when the client request is determined to be for information from a known entity that publishes resources to the repeater server network.

20. A method as in claim 19, wherein the repeater server in receipt of the request uses at least information in an Hypertext Transfer Protocol (HTTP) header of the client request to determine the name by which the repeater server was addressed.

21. A method, in a system which includes (a) a repeater server network including a plurality of repeater servers, (b) a plurality of subscribers to the repeater server network, wherein the plurality of subscribers are entities that publish resources via one or more origin servers, wherein the origin servers are distinct from the repeater servers, (c) a repeater selector mechanism constructed and adapted to identify, for a client request, a repeater server from the plurality of repeater servers to handle the request, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, the method comprising:

on at least some of the repeater servers in the repeater server network, replicating some or all of the information available on the one or more origin servers;

upon receipt of a client request for information, the request being for a resource which is embedded in another document, determining, using at least the subscriber verifying mechanism, and based, at least in part, on an origin server name in a Uniform Resource Locator (URL) used to make the client request, whether the client request is for information from a known entity that publishes resources to the repeater server network; and

serving the information from the repeater server identified by the repeater selector mechanism when the client request is determined to be for information from a known entity that publishes resources to the repeater server network.

22. In a computer network which includes (a) a plurality of origin servers, (b) a plurality of repeater servers distinct from the plurality of origin servers and forming at least one repeater server network, (c) a repeater selector mechanism for identifying, for a request from a client, and based, at least in part, on a location of the client in the computer network, a repeater server of the plurality of repeater servers to handle the request, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of the plurality of subscribers to the repeater server network, and in which at least one of the plurality of repeater servers replicates some or all of the information available on the plurality of origin servers, and in which requests for resources may be handled by the repeater server network in order for the resources to be served, a method comprising:

obtaining a client request from the client for a resource, the resource being embedded in another document

if the client request for the resource is not for a resource from a subscriber to the repeater server network, as determined at least in part by the subscriber verifying mechanism rejecting the client request, otherwise

constructing a reply including the resource and serving that reply from the repeater server identified by the repeater selector mechanism.

23. A method as in claim 22, wherein the resource is obtained by the repeater server of the plurality of repeater servers from the origin server of the plurality of origin servers, and

wherein the repeater server determines the origin server from which to obtain the resources based, at least in part, upon the name by which the repeater server was addressed.

24. A method as in claim 22 wherein the resource is obtained by the repeater server of the plurality of repeater servers from the origin server of the plurality of origin servers, and

wherein the repeater server determines the origin server from which to obtain the resources based, at least in part, upon information in an Hypertext Transfer Protocol (HTTP) header.

25. In a computer network which includes (a) a plurality of origin servers, and (b) at least one repeater server network including a plurality of repeater servers distinct from the plurality of origin servers, (c) a repeater selector mechanism for identifying an appropriate repeater server to handle a request, and in which at least one of the plurality of repeater servers replicates a portion of the information available on at least one of the plurality of origin servers, and (d) a subscriber verifying mechanism constructed and adapted to verify whether an entity is any one of a plurality of subscribers to the repeater server network, and in which requests may be handled by the repeater server network in order for resources to be served, a method comprising:

obtaining a request from a client for a resource, the resource being embedded in another document;

identifying, by the repeater selector mechanism, a repeater server of the repeater server network to handle the request;

if it is determined, at least in part by the subscriber verifying mechanism that the request is for information from a known subscriber,

determining whether the resource is cached locally at the repeater server selected by the repeater selector mechanism, and, based, at least in part, on said determining, if the requested resource is in a cache of the repeater server, retrieving the resource; and

when the resource is determined not to be cached locally at the repeater server, attempting to obtain the resource from the origin server of the plurality of origin servers or a peer cache,

wherein the repeater server determines which origin server of the plurality of origin servers to use based, at least in part, upon the name by which the repeater server was addressed.

26. A server including a processor adapted and programmed to:

(i) replicate at least a portion of information available on a plurality of origin servers distinct from the server;

(ii) obtain a client request for information, the client request being for a resource which is embedded in another document;

(iii) determine, based, at least in part, on a name by which the server was addressed, and using a subscriber verifying mechanism constructed and adapted to verify that the name belongs to a subscriber to a repeater server network which includes the server, whether the client request is for information from a known subscriber to the repeater server network which includes the server; and

(iv) serve the requested information to the client when the request is determined to be for information from the known subscriber to the repeater server network.

28. A server as in any one of claims 26 and 27, wherein the server uses at least information in an Hypertext Transfer Protocol (HTTP) header to determine the name by which the server was addressed.

29. A server as in claim 26 further programmed to:

reject the client request if the client request is not for information from a known subscriber.

30. A server as in claim 26, further programmed to:

if it is determined tat the client request is for information from the known subscriber, determine whether the information is cached locally at the server, and, based, at least in part, on said determining, if the information is cached locally, retrieve the information; and

when the information is determined not to be cached locally, attempt to obtain the information from the origin server or from a peer cache.

31. A server as in claim 30 further programmed to:

upon obtaining the information, cache the information locally.

32. A server as in claim 31 further programmed to:

construct and send a reply including the information.

33. A server as in claim 32 further programmed to:

record details about a transaction involving the client request.

34. A server as in claim 33 wherein the details include one or more of:

the current time, the address of the client issuing the client request, the Uniform Resource Locator (URL) requested, and a type of response generated by the server.

47. A server as in any of claims 26 and 27, wherein the subscriber verifying mechanism is located at a repeater server.

48. A method as in any of claims 26 and 27 wherein the subscriber verifying mechanism comprises a table used to verify that an origin server belongs to a known subscriber.

27. A server including a processor adapted and programmed to:

(i) replicate at least a portion of information available on a plurality of origin servers distinct from the server;

(ii) obtain a client request for information, the client request being for a resource which is embedded in another document served from an origin server of the plurality of origin servers;

(iii) determine, based, at least in part, on at least an origin server name in a Uniform Resource Locator (URL) used to make the client request, and using subscriber verifying mechanism constructed and adapted to verify that the origin server name belongs to a subscriber to a repeater server network which includes the server, whether the client request is for information from a known subscriber to a repeater server network which includes the server; and

(iv) serve the information to the client when the client request is determined to be for information from the known subscriber to the repeater server network.