NETWORK CACHING FOR MULTIPLE CONTEMPORANEOUS REQUESTS

• US 20100268789A1
• Filed: 04/17/2009
• Published: 10/21/2010
• Est. Priority Date: 04/17/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Patel, Himanshu, Ruia, Anil K., Bocharov, John A., Lin, Ning, Yoo, Won Suk
• Granted Patent: US 8,046,432 B2
• Time in Patent Office: Days
• US Class Current: 709/214
• CPC Class Codes: H04L 67/2885 Hierarchically arranged intermediate devices, e.g. for hierarchical caching
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A live caching system is described herein that reduces the burden on origin servers for serving live content. In response to receiving a first request that results in a cache miss, the system forwards the first request to the next tier while “holding” other requests for the same content. If the system receives a second request while the first request is pending, the system will recognize that a similar request is outstanding and hold the second request by not forwarding the request to the origin server. After the response to the first request arrives from the next tier, the system shares the response with other held requests. Thus, the live caching system allows a content provider to prepare for very large events by adding more cache hardware and building out a cache server network rather than by increasing the capacity of the origin server.

Claims

1. A computer-implemented method for handling requests for live media events, the method comprising:

receiving a request for media content;

identifying a cache key based on distinguishing information in the received request;

looking up the identified key in the cache to determine whether the requested media content is stored in the cache.

determining whether there is a related pending request; and

in response to determining that the requested media content is not stored in the cache and that there is a pending request for the same content,

waiting for a response to the pending request without forwarding the received request to an origin server; and

receiving a response to the pending request and responding to both the received request and the pending request based on the single received response,

wherein the preceding steps are performed by at least one processor.

2. The method of claim 1 wherein the received request includes a URL and identifying a cache key comprises deriving the key from the URL.

3. The method of claim 1 wherein the cache includes a hash table, and the key acts as a hash value for looking up content in the cache.

4. The method of claim 1 wherein determining whether there is a related pending request comprises using the identified cache key to identify pending requests.

5. The method of claim 1 further comprising, in response to determining that there is not a related pending request, forwarding the received request to another server that can provide the requested media content.

6. The method of claim 1 further comprising, in response to determining that the requested media content is stored in the cache, providing a response to the received request based on cached media content.

7. The method of claim 1 further comprising detecting that the pending request did not reach a destination and sending the pending request to a new destination to retrieve the media content.

8. The method of claim 1 wherein the received request identifies a media fragment of a media presentation, and wherein each media fragment is individually cacheable.

9. The method of claim 1 wherein the cache includes media content stored for partially received responses, and wherein the received request identifies media content for which the cache includes at least part of the media content responsive to the request.

10. A computer system for caching requests on a network, the system comprising:

a processor and memory configured to execute software instructions;

a request component configured to receive requests for content from clients over the network;

a matching component configured to receive a key based on a received request with which to look up items in a cache stored by the system;

a data store component configured to store cache items and configuration information;

a forwarding component configured to forward requests for content that is not available in the cache to a next tier for responding to a received request, wherein the forwarding component holds requests for content that arrive during a short period of time and reduces a burden on the next tier by holding at least some requests; and

a response component configured to receive forwarded responses from the next tier and provide the responses to pending requests.

11. The system of claim 10 wherein the system operates as an HTTP cache and wherein the request component is further configured to receive HTTP requests that specify a URL that a client is requesting to retrieve.

12. The system of claim 10 wherein the request component is further configured to parse a protocol of a received request, retrieve distinguishing information from the request, and pass the distinguishing information to the matching component.

13. The system of claim 10 wherein the matching component is further configure to receive a key derived from a URL, identify matching content in the cache, and provide an HTTP response based on the identified matching content.

14. The system of claim 10 wherein the data store component is further configured to store responses to previous requests and provide fast access to respond to subsequent requests from the cache rather than forwarding a request to the origin server or other next tier server.

15. The system of claim 10 wherein the forwarding component is further configured to route requests to the next tier based on a content delivery network that specifies a defined hierarchy of cache servers and origin servers for providing content.

16. The system of claim 10 wherein the forwarding component places a placeholder in the cache that subsequently received requests use to determine that a pending request for content identified by the placeholder is in progress.

17. The system of claim 10 wherein the response component is further configured to identify one or more requests associated with a response, and reply to one or more senders of the requests based on the response.

18. The system of claim 10 wherein the response component is further configured to add received responses to the cache so that the system can serve subsequent requests for content of the response from the cache.

19. The system of claim 10 wherein the system is part of a multi-server system that uses HTTP-based caching for delivering a live media event.

20. A computer-readable storage medium comprising instructions for controlling a computer system to provide a response to multiple pending requests, wherein the instructions, when executed, cause a processor to perform actions comprising:

receiving a response to a request forwarded from a cache server to an origin server, wherein the response includes a content item requested in the forwarded request;

identifying a key associated with the received response, wherein the key distinguishes the content item from other content items;

adding the received response to the cache based on the identified key to allow subsequent requests to respond directly from the cache without forwarding a new request or waiting for an existing forwarded request to the origin server;

identifying an original request associated with the received response;

responding to the identified original request based on the received response;

identifying at least one held request for the content item that arrived while the original request was pending; and

providing the received response to each identified held request, so that for events that comprise multiple requests that are close in time, the system receives fewer responses from the origin server than a number of received requests.