Caching web objects transformed by a pipeline of adaptation services

• US 7,565,399 B1
• Filed: 08/26/2002
• Issued: 07/21/2009
• Est. Priority Date: 08/26/2002
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Goel, Atul
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US10/228,639
• Time in Patent Office: 2,521 Days
• Field of Search: 709/203, 709217-219, 709223-225, 711118-119
• US Class Current: 709/203
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

In one embodiment, a method for processing a client request for content comprises receiving, at an intermediary network node, a client request for content on a network. The client request is associated with a set of attributes indicating a requirement to process the requested content by multiple designated applications prior to transferring the requested content to a client. The method further comprises searching, at the intermediary network node, a content cache for matching content conforming to the processing requirement, and determining, at the intermediary network node, whether the matching content is currently valid. If this determination is positive, the matching content is transferred to the client.

Claims

1. A method comprising:

storing a plurality of versions of an item of content in a content cache within an intermediary network node, wherein each of said versions is a result of processing the item of content using one or more of a plurality of applications;

receiving, at the intermediary network node, a client request for the item of content, the client request being associated with an indication of a processing requirement for processing the item of content with a combination of the plurality of applications prior to transferring the item of content to the client;

determining an order in which the plurality of applications is to be invoked to fulfill the processing requirement;

computing a pipeline key based on an identifier of each of the plurality of applications and the order for invoking the plurality of applications;

computing a cache key for the item of content based on the identifier of the item of content and the pipeline key;

searching the content cache within the intermediary network node using the computed cache key for matching content including one of said versions conforming to the processing requirement;

determining, at the intermediary network node, whether the matching content is currently valid;

and if the matching content is currently valid, transferring the matching content to the client.

2. The method of claim 1 wherein the plurality of applications comprises at least one of:

a language translation application, a format translation application, a content filtering application, an advertisement insertion application, a content adaptation application, or a virus check application.

3. The method of claim 1 wherein the identifier of the item of content is a uniform resource locator (URL) associated with the item of content.

4. The method of claim 1 wherein determining whether the matching content is currently valid includes:

determining whether the matching content is stale due to a change associated with an origin server of the item of content; and

if the matching content is not stale, determining whether the matching content is consistent with a current state of the plurality of applications.

5. The method of claim 4 wherein determining whether the matching content is stale due to the change associated with the origin server comprises analyzing current aging data associated with the matching content, the aging data being provided by the origin server.

6. The method of claim 4 wherein determining whether the matching content is consistent with the current state of the plurality of applications comprises:

computing a new service bitmask and a new consistency key based on the current state of the plurality of applications;

comparing the new service bitmask with an existing service bitmask associated with the matching content; and

comparing the new consistency key with an existing consistency key associated with the matching content.

7. The method of claim 6 wherein the current state of the plurality of applications depends on current communication abilities of the plurality of applications and current versions of the plurality of applications.

8. The method of claim 7 wherein:

the intermediary network node is a network caching device communicating with the plurality of applications using an Internet Content Adaptation Protocol (ICAP); and

a current version of each of the plurality of applications is identified by an ICAP services tag (ISTAG) provided by said each of the plurality of applications.

9. The method of claim 1 further comprising:

determining that the matching content is not currently valid; and

discarding the matching content.

10. The method of claim 1 further comprising:

determining, based on the search, that the content cache does not store matching content conforming to the processing requirement;

finding, in the content cache, pre-processed content having an identifier of the item of content;

determining that the found content is currently valid;

initiating the required processing of the found content by the plurality of applications; and

transferring the processed content to the client.

11. The method of claim 1 further comprising:

determining, based on the search, that the content cache does not store matching content conforming to the processing requirement;

determining that the content cache does not store pre-processed content having an identifier of the item of content;

retrieving the item of content from an origin server;

initiating the required processing of the retrieved content by the plurality of applications;

computing a cache key for the processed content;

computing a service bitmask for the processed content;

computing a consistency key for the processed content;

storing a copy of the processed content, together with the cache key, the service bitmask, the consistency key and a content identifier, in the content cache; and

transferring the processed content to the client.

12. A device to operate as an intermediary node on a network, the device comprising:

a content cache to store a plurality of versions of an item of content, wherein each of said versions is a result of processing the item of content using one or more of a plurality of applications;

a request processing unit to receive a client request for the item of content, to identify a processing requirement for processing the item of content with a combination of the plurality of applications prior to transferring the item of content to a client, and to determine an order in which the plurality of applications are to be invoked to fulfill the processing requirement;

a key calculator to compute a pipeline key based on an identifier of each of the plurality of applications and the order for invoking the plurality of applications, and to compute a cache key for the item of content based on the identifier of the item of content and the pipeline key;

a searcher to search the content cache using the cache key for matching content including one of said versions conforming to the processing requirement;

a content consistency tester to determine whether the matching content is currently valid; and

a transmitter to transfer the matching content to the client if the matching content is currently valid.

13. The device of claim 12 wherein the plurality of applications comprises at least one:

of a language translation application, a format translation application, a content filtering application, an advertisement insertion application, a content adaptation application, or a virus check application.

14. The device of claim 12 wherein the identifier of the item of content is a uniform resource locator (URL) associated with the item of content.

15. The device of claim 12 wherein the content consistency tester is to determine whether the matching content is currently valid by determining whether the matching content is stale due to a change associated with an origin server of the item of content, and, if the matching content is not stale, determining whether the matching content is consistent with a current state of the plurality of applications.

16. A device to operate as an intermediary node on a network, the device comprising:

a processor;

a network interface to allow the device to communicate on the network;

a content cache to store a plurality of versions of an item of content, wherein each of said versions is a result of processing the item of content using one or more of a plurality of applications; and

a storage facility to store program code for execution by the processor to cause the device to receive a client request for the item of content over the network, the client request being associated with one or more attributes indicating a processing requirement for processing the requested content with a combination of the plurality of applications prior to transferring the requested content to a client, the program code for execution by the processor further to cause the device to;

determine an order in which the plurality of applications is to be invoked to fulfill the processing requirement,

compute a pipeline key based on an identifier of each of the plurality of applications and the order for invoking the plurality of applications,

compute a cache key for the item of content based on the identifier of the item of content and the pipeline key;

search the content cache using the cache key to identify matching content in the content cache including one of said versions conforming to the processing requirement;

determine whether the matching content is currently valid, and

transfer the matching content to the client if the matching content is currently valid.

17. The device of claim 16 wherein the plurality of applications comprises at least one of:

a language translation application, a format translation application, a content filtering application, an advertisement insertion application, a content adaptation application, or a virus check application.

18. The device of claim 16 wherein the intermediary network node is a network caching device communicating with the plurality of designated applications using an Internet Content Adaptation Protocol (ICAP).

19. A method comprising:

storing a plurality of versions of an item of content in a content cache within an intermediary network node, wherein each of said versions is a result of processing the item of content using one or more of a plurality of applications; and

computing a pipeline key based on an identifier of each of the plurality of applications and the order for invoking the plurality of applications;

computing a cache key for the item of content based on the identifier of the item of content and the pipeline key;

searching the content cache within the intermediary node using the cache key for matching content including one of said versions conforming to the processing requirement;

using the plurality of versions of the item of content in the content cache to service a plurality of client requests for the item of content at the intermediary network node, each request having a processing requirement that the item of content be processed by a combination of the plurality of applications and an order in which the plurality of applications are to be invoked to fulfill the processing requirement, without having to process the item of content with any of the plurality of applications to service said requests.

20. The method of claim 19, wherein said servicing a plurality of client requests comprises:

searching the content cache for a version of the item of content which matches a client request; and

using said cache key to identify said version as conforming to the processing requirement of the client request.