Method and apparatus for dispatching document requests in a proxy

• US 6,647,421 B1
• Filed: 12/10/2001
• Issued: 11/11/2003
• Est. Priority Date: 06/03/1996
• Status: Expired due to Term

• Current Assignee: Rovi Technologies Corporation (Adeia Inc.)
• Original Assignee: WebTV Networks Inc. (Microsoft Corporation)
• Inventors: Mighdoll, Lee S., Logue, Jay D.
• Primary Examiner(s): WINDER, PATRICE L
• Application Number:
  US10/016,412
• Time in Patent Office: 701 Days
• Field of Search: 709/203, 709/213, 709/214, 709/215, 709/217, 709/218, 709/219, 709/220, 709/221, 709/226, 709/238
• US Class Current: 709/226
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/222 using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages
• CPC Class Codes: H04L 51/234 for tracking messages
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 2242/22 Automatic class or number identification arrangements
• CPC Class Codes: H04M 3/42059 Making use of the calling party identifier
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2356 by altering the spatial resolution
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/25875 involving end-user authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; arrangements for secret or secure communication including means for verifying the identity or authority of a user of the system H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25891 : being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4331 : Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/435 : Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 : Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 21/478 : Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4782 : Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4786 : e-mailing message switching systems, e.g. electronic mail systems H04L51/00
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8166 : involving executable data, e.g. software arrangements for executing specific programs G06F9/44; broadcasting computer programmes in broadcast systems H04H20/91; involving the movement of software or configuration parameters H04L67/34
• CPC Class Codes: H04N 5/602 : for digital sound signals
• CPC Class Codes: H04N 7/088 : the inserted signal being digital

Abstract

A method and apparatus for dispatching document requests in a proxy to more efficiently allocate the document cache space within the proxy is provided. A proxy includes a document cache storing recently requested documents. The proxy is coupled to a client and to a remote server. A Uniform Resource Locator (“URL”) is included in the document request. The proxy forwards the request to one of a plurality of proxy servers based upon the URL. According to another aspect of the present invention, the proxy performs a hash function on the URL that maps the URL to exactly one of the plurality of proxy servers. Advantageously, in this manner, mutually exclusive portions of the Web'"'"'s content can be allocated to particular proxy servers.

Claims

1. In a networked computer system that includes a plurality of remote servers, a plurality of proxy servers, and a plurality of client systems, all of which are logically interconnected so that the client systems can access informational content stored at the one or more remote servers, a method of efficiently allocating cache space within the plurality of proxy servers so that requested content is distributively cached at mutually exclusive proxy servers, comprising steps for:

dividing responsibility for obtaining and caching content among a plurality of proxy servers, wherein at least two proxy servers are responsible for obtaining and caching mutually exclusive content;

receiving a request for content that is available at one or more particular remote servers;

rather than allowing any arbitrary proxy server to obtain and cache the requested content from the one or more particular remote servers, mapping the request to a specific proxy server; and

at the specific proxy server to which the request is mapped, searching for the requested content in a local cache, and if the requested content is found in the local cache, returning it to a client system from which the request was received, and if the requested content is not found, then obtaining the requested content from the one or more particular remote servers and storing it in the local cache, and returning the requested content to the client system from which the request was received.

2. A method as recited in claim 1, wherein the requested content comprises one or more Web pages, and wherein the request comprises one or more uniform resource locators for the one or more Web pages.

3. A method as recited in claim 1, wherein the step for mapping identifies a cluster of two or more proxy servers, the method further comprising a step for storing requested content that is obtained from at least one remote server in the local cache of each proxy server in the cluster.

4. A method as recited in claim 3, further comprising a step for distributing multiple requests for the requested content among the two or more proxy servers of the cluster.

5. A method as recited in claim 1, wherein the step for mapping comprises an act of applying a hash algorithm.

6. A method as recited in claim 5, wherein the hash algorithm comprises a Message Digest 5 algorithm.

7. A method as recited claim 1, wherein a central dispatcher receives the request for content and maps the request, the method further comprising a step for forwarding the request to the specific proxy server.

8. A method as recited in claim 1, wherein an initial proxy server receives the request for content, the method further comprising a step for forwarding the request to the specific proxy server after the initial proxy server searches for the requested content in a local cache and the requested content is not found.

9. In a networked computer system that includes a plurality of remote servers, a plurality of proxy servers and a plurality of client systems, all of which are logically interconnected so that the client systems can access informational content stored at the one or more remote servers, a computer program product for implementing a method of efficiently allocating cache space within the plurality of proxy servers so that requested content is distributively cached at mutually exclusive proxy servers, comprising a computer readable medium for storing executable instructions for implementing the method, wherein the method comprises steps for:

dividing responsibility for obtaining and caching content among a plurality of proxy servers, wherein at least two proxy servers are responsible for obtaining and caching mutually exclusive content;

receiving a request for content that is available at one or more particular remote servers;

rather than allowing any arbitrary proxy server to obtain and cache the requested content from the one or more particular remote servers, mapping the request to a specific proxy server; and

at the specific proxy server to which the request is mapped, searching for the requested content in a local cache, and if the requested content is found in the local cache, returning it to a client system from which the request was received, and if the requested content is not found, then obtaining the requested content from the one or more particular remote servers and storing it in the local cache, and returning the requested content to the client system from which the request was received.

10. A computer program product as recited in claim 9, wherein the requested content comprises one or more Web pages, and wherein the request comprises one or more uniform resource locators for the one or more Web pages.

11. A computer program product as recited in claim 9, wherein the step for mapping identifies a cluster of two or more proxy servers, the method further comprising a step for storing requested content that is obtained from at least one remote server in the local cache of each proxy server in the cluster.

12. A computer program product as recited in claim 11, the method further comprising a step for distributing multiple requests for the requested content among the two or more proxy servers of the cluster.

13. A computer program product as recited in claim 9, wherein the step for mapping comprises an act of applying a hash algorithm.

14. A computer program product as recited in claim 13, wherein the hash algorithm comprises a Message Digest 5 algorithm.

15. A computer program product as recited claim 9, wherein a central dispatcher receives the request for content and maps the request, the method further comprising a step for forwarding the request to the specific proxy server.

16. A computer program product as recited in claim 9, wherein an initial proxy server receives the request for content, the method further comprising a step for forwarding the request to the specific proxy server after the initial proxy server searches for the requested content in a local cache and the requested content is not found.

17. In a networked computer system that includes a plurality of remote servers, a plurality of proxy servers and a plurality of client systems, all of which are logically interconnected so that the client systems can access informational content stored at the one or more remote servers, a method of efficiently allocating cache space within the plurality of proxy servers so that requested content is distributively cached at mutually exclusive proxy servers, comprising acts of:

assigning to each of a plurality of proxy servers a mutually exclusive range of content to retrieve from one or more remote servers on behalf of one or more client systems and cache;

hashing at least a portion of a request received from a client system for content that is available at one or more particular remote servers;

rather than allowing any arbitrary proxy server to retrieve and cache the requested content, identifying, with the at least a portion of the hashed request, at least one proxy server assigned to a particular mutually exclusive range of content that corresponds to the requested content; and

at the at least one identified proxy server, examining a local cache for the requested content, and if found in the local cache, sending the requested content to the client system, but if not found in the local cache, retrieving the requested content from the one or more particular remote servers, storing the requested content in the local cache, and sending the requested content to the client system.

18. A method as recited in claim 17, wherein the requested content comprises one or more Web pages, and wherein the request comprises one or more uniform resource locators for the one or more Web pages.

19. A method as recited in claim 17, wherein the act of hashing identifies a cluster of two or more proxy servers collectively assigned to the particular mutually exclusive range of content that corresponds to the requested content, the method further comprising an act of adding any requested content that is received from the one or more particular remote servers to the local cache of each proxy server in the cluster.

20. A method as recited in claim 19, further comprising an act of load balancing multiple requests for the requested content among the two or more proxy servers of the cluster.

21. A method as recited in claim 17, wherein the hashing comprises a Message Digest 5 algorithm.

22. A method as recited in claim 17, wherein a central dispatcher receives the request for content and hashes the at least a portion of the request, the method further comprising an act of sending the request to the at least one identified proxy server.

23. A method as recited in claim 17, wherein an initial proxy server receives the request for content, the method further comprising an act of sending the request to the at least one identified proxy server after the initial proxy server searches for the requested content in a local cache and the requested content is not found.

24. In a networked computer system that includes a plurality of remote servers, a plurality of proxy servers and a plurality of client systems, all of which are logically interconnected so that the client systems can access informational content stored at the one or more remote servers, a computer program product for implementing a method of efficiently allocating cache space within the plurality of proxy servers so that requested content is distributively cached at mutually exclusive proxy servers, comprising a computer readable medium for storing executable instructions for implementing the method, and wherein the method comprises acts of:

assigning to each of a plurality of the proxy servers a mutually exclusive range of content to retrieve from one or more remote servers on behalf of one or more client systems and cache;

hashing at least a portion of a request received from a client system for content that is available at one or more particular remote servers;

rather than allowing any arbitrary proxy server to retrieve and cache the requested content, identifying, with at least a portion of the hashed request, at least one proxy server assigned to a particular mutually exclusive range of content that corresponds to the requested content; and

at the at least one identified proxy server, examining a local cache for the requested content, and if found in the local cache, sending the requested content to the client system, but if not found in the local cache, retrieving the requested content from the one or more particular remote servers, storing the requested content in the local cache, and sending the requested content to the client system.

25. A computer program product as recited in claim 24, wherein the requested content comprises one or more Web pages, and wherein the request comprises one or more uniform resource locators for the one or more Web pages.

26. A computer program product as recited in claim 24, wherein the act of hashing identifies a cluster of two or more proxy servers collectively assigned to the particular mutually exclusive range of content that corresponds to the requested content, the method further comprising an act of adding any requested content that is received from the one or more particular remote servers to the local cache of each proxy server in the cluster.

27. A computer program product as recited in claim 26, the method further comprising an act of load balancing multiple requests for the requested content among the two or more proxy servers of the cluster.

28. A computer program product as recited in claim 24, wherein the hashing comprises a Message Digest 5 algorithm.

29. A computer program product as recited in claim 24, wherein a central dispatcher receives the request for content and hashes the at least a portion of the request, the method further comprising an act of sending the request to the at least one identified proxy server.

30. A computer program product as recited in claim 24, wherein an initial proxy server receives the request for content, the method further comprising an act of sending the request to the at least one identified proxy server after the initial proxy server searches for the requested content in a local cache and the requested content is not found.