Proxy-based cache content distribution and affinity

• US 9,667,739 B2
• Filed: 09/08/2015
• Issued: 05/30/2017
• Est. Priority Date: 02/07/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Yoo, Won Suk, Holt, Nick, Lopez, Daniel Vasquez, Di Marco, Aniello Scotto
• Primary Examiner(s): Avellino, Joseph E
• Assistant Examiner(s): MCLEOD, MARSHALL M
• Application Number:
  US14/847,204
• Publication Number:
  US 20150381757A1
• Time in Patent Office: 630 Days
• Field of Search: None
• CPC Class Codes: H04L 45/44 Distributed routing
• CPC Class Codes: H04L 45/58 Association of routers
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/2885 Hierarchically arranged intermediate devices, e.g. for hierarchical caching
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

A distributed caching hierarchy that includes multiple edge routing servers, at least some of which receiving content requests from client computing systems via a load balancer. When receiving a content request, an edge routing server identifies which of the edge caching servers the requested content would be in if the requested content were to be cached within the edge caching servers, and distributes the content request to the identified edge caching server in a deterministic and predictable manner to increase the likelihood of increasing a cache-hit ratio.

Claims

1. A distribution network for retrieving cached content comprising:

a load balancer that receives requests from one or more client computing systems and, without being aware of where any content corresponding to the content requests may reside in the distribution network, the load balancer routing said content requests to one or more edge caching servers;

a plurality of edge caching servers each comprising;

an edge caching unit having at least one hardware processor and memory for caching content obtained from one or more origin servers; and

an edge routing unit which receives content requests from the load balancer, the edge routing unit comprising at least one hardware processor and memory containing computer-executable instructions which, when implemented, cause the edge routing unit to process one or more content requests received from the load balancer according to a computer-implemented predictive routing protocol which predicts where cached content corresponding to requested content resides in the distribution network, and wherein the computer-implemented predictive routing protocol comprises acts of;

identifying which of the plurality of edge caching servers is correlated to the requested content based on the predictive routing protocol; and

routing the content request to the identified edge caching server without having to first determine whether the edge routing server contains the requested content.

2. The distribution network in accordance with claim 1, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines whether the edge caching unit of the particular edge caching server has the requested content cached.

3. The distribution network in accordance with claim 2, wherein the edge caching unit initiates of the particular edge caching server return of the requested content to a client computing system from which the given content request was sent.

4. The distribution network in accordance with claim 1, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines that the edge caching unit of the particular edge caching server does not have the requested content, and wherein an edge routing unit of the particular caching server then identifies, based on the predictive routing protocol, which other of the plurality of edge caching servers the given content request would be in, and the given content request is then sent from the particular edge caching server to said identified other of the plurality of edge caching servers.

5. The distribution network in accordance with claim 1, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines that the edge caching unit of the particular edge caching server does not have the requested content, and wherein the edge routing unit probabilistically identifies a further tier of edge caching servers at which the given content request would likely be cached, and

the given content request is then routed to one of the edge servers in the identified next tier of edge caching servers.

6. The distribution network of claim 1, wherein the load balancer routes the content requests based at least in part on a health status associated with each of the edge routing units.

7. The distribution network of claim 1, wherein each edge routing unit is configured to refrain from caching content.

8. The distribution network of claim 1, wherein each of the edge caching units caches mutually exclusive sets of cached content from one another.

9. The distribution network of claim 1, wherein the edge routing unit and the one edge caching unit of at least one edge caching server reside on a common computing device.

10. A computer-implemented method for retrieving cached content from a hierarchy of edge caching servers, the computer-implemented method being performed by one or more processors executing computer executable instructions for the computer-implemented method, and the computer-implemented method comprising:

receiving at a load balancer one or more requests from one or more client computing systems;

without being aware of where any content corresponding to content requests may reside in the distribution network, the load balancer routing said content requests to one or more edge caching servers;

a plurality of the edge caching servers each comprising an edge caching unit having at least one hardware processor and memory for caching content obtained from one or more origin servers, and an edge routing unit, and wherein the computer implemented method further comprises;

receiving at one or more edge routing units, content requests from the load balancer;

each edge routing unit comprising at least one hardware processor and memory containing computer-executable instructions which, when implemented, cause the edge routing unit to process one or more content requests received from the load balancer according to a computer-implemented predictive routing protocol which predicts where cached content corresponding to requested content resides in the distribution network, and wherein the computer-implemented predictive routing protocol comprises acts of;

identifying which of the plurality of edge caching servers is correlated to the requested content based on the predictive routing protocol; and

routing the content request to the identified edge caching server without having to first determine whether the edge routing server contains the requested content.

11. The computer-implemented method in accordance with claim 10, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines whether the edge caching unit of the particular edge caching server has the requested content cached.

12. The computer-implemented method in accordance with claim 11, wherein the edge caching unit initiates of the particular edge caching server return of the requested content to a client computing system from which the given content request was sent.

13. The computer-implemented method in accordance with claim 11, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines that the edge caching unit of the particular edge caching server does not have the requested content, and wherein an edge routing unit of the particular caching server then identifies, based on the predictive routing protocol, which other of the plurality of edge caching servers the given content request would be in, and the given content request is then sent from the particular edge caching server to said identified other of the plurality of edge caching servers.

14. The computer-implemented method in accordance with claim 10, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines that the edge caching unit of the particular edge caching server does not have the requested content, and wherein the edge routing unit probabilistically identifies a further tier of edge caching servers at which the given content request would likely be cached, and the given content request is then routed to one of the edge servers in the identified next tier of edge caching servers.

15. The computer-implemented method of claim 10, wherein the load balancer routes the content requests based at least in part on a health status associated with each of the edge routing units.

16. The computer-implemented method of claim 10, wherein each edge routing unit is configured to refrain from caching content.

17. The computer-implemented method of claim 10, wherein each of the edge caching units caches mutually exclusive sets of cached content from one another.

18. The computer-implemented method of claim 10, wherein the edge routing unit and the one edge caching unit of at least one edge caching server reside on a common computing device.

19. One or more hardware storage devices containing computer-executable instructions which, when executed by one or more processors, cause the one or more processors to perform a computer-implemented method for retrieving cached content from a hierarchy of edge caching servers, and wherein the computer-implemented method comprises:

receiving at a load balancer one or more requests from one or more client computing systems;

without being aware of where any content corresponding to content requests may reside in the distribution network, the load balancer routing said content requests to one or more edge caching servers;

a plurality of the edge caching servers each comprising an edge caching unit having at least one hardware processor and memory for caching content obtained from one or more origin servers, and an edge routing unit, and wherein the computer implemented method further comprises;

receiving at one or more edge routing units, content requests from the load balancer;

each edge routing unit comprising at least one hardware processor and memory containing computer-executable instructions which, when implemented, cause the edge routing unit to process one or more content requests received from the load balancer according to a computer-implemented predictive routing protocol which predicts where cached content corresponding to requested content resides in the distribution network, and wherein the computer-implemented predictive routing protocol comprises acts of;

identifying which of the plurality of edge caching servers is correlated to the requested content based on the predictive routing protocol; and

routing the content request to the identified edge caching server without having to first determine whether the edge routing server contains the requested content.

20. The one or more hardware storage devices of claim 19, wherein the edge routing unit sends the given content request to a particular edge caching server and the edge caching unit of the particular edge caching server determines that the edge caching unit of the particular edge caching server does not have the requested content, and wherein an edge routing unit of the particular caching server then identifies, based on the predictive routing protocol, which other of the plurality of edge caching servers the given content request would be in, and the given content request is then sent from the particular edge caching server to said identified other of the plurality of edge caching servers.