System for accessing distributed data cache channel at each network node to pass requests and data

• US 20040172458A1
• Filed: 03/04/2004
• Published: 09/02/2004
• Est. Priority Date: 11/28/1994
• Status: Abandoned Application

• Current Assignee: RPX Corporation
• Original Assignee: RPX Corporation
• Inventors: Pitts, William Michael
• Application Number:
  US10/794,723
• Publication Number:
  US 20040172458A1
• Time in Patent Office: Days
• US Class Current: 709/213
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Network Distributed Caches (“NDCs”) (50) permit accessing a named dataset stored at an NDC server terminator site (22) in response to a request submitted to an NDC client terminator site (24) by a client workstation (42). In accessing the dataset, the NDCs (50) form an NDC data conduit (62) that provides an active virtual circuit (“AVC”) from the NDC client site (24) through intermediate NDC sites (26B, 26A) to the NDC server site (22). Through the AVC provided by the conduit (62), the NDC sites (22, 26A and 26B) project an image of the requested portion of the named dataset into the NDC client site (24). The NDCs (50) maintain absolute consistency between the source dataset and its projections at all NDC client terminator sites (24, 204B and 206) at which client workstations access the dataset. Channels (116) in each NDC (50) accumulate profiling data from the requests to access the dataset for which they have been claimed. The NDCs (50) use the profile data stored in channels (116) to anticipate future requests.

Claims

1. In a network of digital computers that includes a plurality of Network Distributed Cache (“

NDC”

) sites, each NDC site including an NDC that has an NDC buffer, a method for projecting an image of a stored dataset from an NDC server terminator site into an NDC client terminator site in response to a request to access such dataset transmitted from a client site to the NDC client terminator site, the method comprising the steps of;

(a) the NDC receiving the request to access data in the stored dataset;

(b) the NDC checking the NDC buffer at this NDC site to determine if a projected image of data requested from the dataset is already present there;

(c) if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if the NDC site receiving the request is not the NDC server terminator site for the dataset, the NDC of this NDC site transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

(d) if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if the NDC site receiving the request is the NDC server terminator site for the dataset, the NDC of this NDC site accessing the stored dataset to project an image of the requested data into its NDC buffer;

(e) repeating the steps (a) through (d) until the NDC buffer of the downstream NDC site receiving the request contains a projected image of all requested data;

(f) each successive NDC site, having obtained a projected image of all the requested data, returning data requested from it upstream to the NDC site from which it received the request until the requested data arrives at the NDC client terminator site; and

(g) the NDC client terminator site, upon receiving the requested data, returning the requested data to the client site.

2. The method of claim 1 wherein, prior to returning the requested data to the client site, the NDC client terminator site reformats the requested data from the protocol employed for communications between pairs of NDC sites into the protocol in which the client site requested access to the dataset from the NDC client terminator site.

3. The method of claim 2 wherein the NDC client terminator site selectively reformats the requested data from the protocol employed for communications between pairs of NDC sites into a particular one of a plurality of different protocols that matches the protocol in which the client site requested access to the dataset from the NDC client terminator site.

4. The method of claim 1 further comprising the steps of:

(h) each NDC upon receiving the initial request to access the dataset claiming a channel and designating the channel for storing various data relevant to processing requests to access the dataset;

(i) storing in the channel of this NDC site data collected by the NDC while processing requests to access the dataset; and

(j) if the NDC site is not performing any steps of the method that are required for responding to a request to access a dataset, the method at NDC sites further including;

i. periodically analyzing data stored in the channel of this NDC site to determine whether it is possible to anticipate future requests to access the dataset;

ii. if the analysis of data stored in the channel of this NDC site establishes that it is possible to anticipate future requests to access the dataset, further analyzing data stored in the channel to determine whether anticipated future requests will soon present this NDC site with another request to access the dataset and whether the projected image of data present in the NDC buffer includes sufficient data to respond immediately to the anticipated request; and

iii. if anticipated future requests to access the dataset will soon present this NDC site with another request to access the dataset and if the projected image of data present in the NDC buffer lacks sufficient data to respond immediately to the anticipated request, this NDC site, before receiving a request therefor, requesting data from the dataset from the next downstream NDC site.

5. The method of claim 4 wherein the NDC site, in requesting data from the next downstream NDC site, requests a quantity of data that is larger than the amount of data returned by the NDC client terminator site to the requesting client site in response to a request to access the dataset received by the NDC client terminator site from the client site.

6. The method of claim 1 wherein each NDC upon receiving the initial request to access the dataset claims a channel and designates the channel for storing various data relevant to processing requests to access the dataset, the method at NDC sites further comprising the step of:

(h) if the NDC is not performing any steps of the method that are required for responding to a request to access a dataset, the method at NDC sites further including;

i. periodically analyzing channels to determine if such channel is presently no longer needed for responding to requests to access the dataset for which the channel was claimed; and

ii. if the channel being analyzed is no longer needed for responding to requests to access the dataset, then processing the no longer needed channel to prepare it for immediate claiming in response to a subsequent request from a client site to access another dataset.

7. The method of claim 6 wherein the periodic analysis of channels to determine if such channel is presently no longer needed for responding to requests is halted if the number of channels available to respond to a future request to access a dataset exceeds a pre-established upper threshold.

8. The method of claim 7 wherein the NDC having stopped periodically analyzing channels, resumes periodically analyzing channels if the number of channels available to respond to a future request to access a dataset drops below a pre-established lower threshold that is less than the pre-established upper threshold.

9. A network of digital computers that includes a client site which requests access to a dataset that is stored at a location that can be accessed through the network, the network comprising:

a plurality of NDC sites, the dataset whose access is requested by the client site being stored at an NDC server terminator site, a request from the client site for access to the dataset being received by an NDC client terminator site, each NDC site including;

(a) an NDC that has an NDC buffer;

(b) means for the NDC to receive the request to access the dataset;

(c) means for the NDC to check the NDC buffer at this NDC site to determine if a projected image of data requested from the dataset is already present there wherein;

i. if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if this NDC site is not the NDC server terminator site for the dataset, the NDC including means for transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

ii. if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if this NDC site is the NDC server terminator site for the dataset, the NDC including means for accessing the dataset to project an image of the requested data into its NDC buffer; and

iii. if the NDC buffer of an NDC site contains a projected image of all requested data, the NDC including means for returning data requested from it upstream to the NDC site from which it received the request, whereby through a succession of such returns of data from one NDC site to the next upstream NDC site the requested data ultimately arrives at the NDC client terminator site; and

(d) data return means for returning the requested data from the NDC client terminator site to the client site.

10. The network of claim 9 wherein, prior to returning the requested data to the client site, said data return means reformats the requested data from the protocol employed for communications between pairs of NDC sites into the protocol in which the client site requested access to the dataset from the NDC client terminator site.

11. The network of claim 10 wherein the data return means selectively reformats the requested data from the protocol employed for communications between pairs of NDC sites into a particular one of a plurality of different protocols that matches the protocol in which the client site requested access to the dataset from the NDC client terminator site.

12. The network of claim 9 wherein each NDC site further comprises:

(e) means for the NDC, upon receiving the initial request to access the dataset, to claim a channel for storing various data relevant to processing requests to access the dataset;

(f) means for the NDC to store in the channel data collected by the NDC while processing requests to access the dataset;

(g) if the NDC site is not responding to a request to access dataset, means for the NDC to periodically analyze data stored in the channel to determine whether it is possible to anticipate future requests to access the dataset;

(h) if the analysis of data stored in the channel by the NDC establishes that it is possible to anticipate future requests to access the dataset, means for the NDC to further analyze data stored in the channel to determine whether anticipated future requests will soon present this NDC with another request to access the dataset and whether the projected image of data present in the NDC buffer includes sufficient data to respond immediately to the anticipated request; and

(i) if anticipated future requests will soon present this NDC site with another request to access the dataset and if the projected image of data present in the NDC buffer lacks sufficient data to respond immediately to the anticipated request, means for the NDC to request from the next downstream NDC site data from the dataset before receiving a request therefor.

13. The network of claim 12 wherein the NDC site requests from the next downstream NDC site a quantity of data from the dataset before receiving a request therefor, the quantity of data requested by the NDC site being larger than the amount of data returned by the NDC client terminator site to the requesting client site in response to a request to access the dataset received by the NDC client terminator site from the client site.

14. The network of claim 9 wherein each NDC site further comprises:

(e) means for the NDC, upon receiving the initial request to access the dataset, to claim a channel for storing various data relevant to processing requests to access the dataset;

(f) if the NDC is not responding to a request to access a dataset, means for the NDC to periodically analyze channels to determine if such channel is presently no longer needed for responding to requests to access the dataset for which the channel was claimed; and

(g) if analysis of the channel establishes that the channel is no longer needed for responding to requests to access the dataset, means for the NDC to process the no longer needed channel to prepare it for immediate claiming in response to a subsequent request from a client site to access another dataset.

15. The network of claim 14 wherein each NDC site further comprises means for halting the analysis of channels to determine if such channels are presently no longer needed for responding to requests if the number of channels available to respond to future requests to access datasets exceeds a pre-established upper threshold.

16. The network of claim 15 wherein each NDC site further comprises means for resuming the previously halted analysis of channels to determine if such channels are presently no longer needed for responding to requests if the number of channels available to respond to future requests to access datasets drops below a pre-established lower threshold that is less than the pre-established upper threshold.

17. In a network of digital computers that includes a plurality of NDC sites, each NDC site including an NDC that has an NDC buffer, a method for projecting images of a stored dataset from an NDC server terminator site into a plurality of NDC client terminator sites in response to requests to access such dataset transmitted from client sites to the NDC client terminator sites, the method comprising the steps of:

(a) the NDC receiving the request to access data in the stored dataset;

(b) the NDC checking the NDC buffer at this NDC site to determine if a projected image of data requested from the dataset is already present there;

(c) if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if the NDC site receiving the request is not the NDC server terminator site for the dataset, the NDC of this NDC site transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

(d) if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if the NDC site receiving the request is the NDC server terminator site for the dataset, the NDC of this NDC site accessing the stored dataset to project an image of the requested data into its NDC buffer;

(e) repeating the steps (a) through (d) until the NDC buffer of the downstream NDC site receiving the request contains a projected image of all requested data;

(f) each successive NDC site, having obtained a projected image of all the requested data, returning the data requested from it upstream to the NDC site from which it received the request until the requested data arrives at the NDC client terminator site; and

(g) the NDC client terminator site, upon receiving the requested data, returning the requested data to the client site.

18. The method of claim 17 wherein, prior to returning the requested data to the client site, the NDC client terminator site reformats the requested data from the protocol employed for communications between pairs of NDC sites into the protocol in which the client site requested access to the dataset from the NDC client terminator site.

19. The method of claim 18 wherein the NDC client terminator site selectively reformats the requested data from the protocol employed for communications between pairs of NDC sites into a particular one of a plurality of different protocols that matches the protocol in which the client site requested access to the dataset from the NDC client terminator site.

20. The method of claim 17 further comprising the steps of:

(h) each NDC upon receiving the initial request to access the dataset claiming a channel and designating the channel for storing various data relevant to processing requests to access the dataset;

(i) storing in the channel of this NDC site data collected by the NDC while processing requests to access the dataset; and

(j) if the NDC site is not performing any steps of the method that are required for responding to a request to access a dataset, the method at NDC sites further including;

i. periodically analyzing data stored in the channel of this NDC site to determine whether it is possible to anticipate future requests to access the dataset;

ii. if the analysis of data stored in the channel of this NDC site establishes that it is possible to anticipate future requests to access the dataset, further analyzing data stored in the channel to determine whether anticipated future requests will soon present this NDC site with another request to access the dataset and whether the projected image of data present in the NDC buffer includes sufficient data to respond immediately to the anticipated request; and

iii. if anticipated future requests to access the dataset will soon present this NDC site with another request to access the dataset and if the projected image of data present in the NDC buffer lacks sufficient data to respond immediately to the anticipated request, this NDC site, before receiving a request therefor, requesting data from the dataset from the next downstream NDC site.

21. The method of claim 20 wherein the NDC site, in requesting data from the next downstream NDC site, requests a quantity of data that is larger than the amount of data returned by the NDC client terminator site to the requesting client site in response to a request to access the dataset received by the NDC client terminator site from the client site.

22. The method of claim 17 wherein each NDC upon receiving the initial request to access the dataset claims a channel and designates the channel for storing various data relevant to processing requests to access the dataset, the method at NDC sites further comprising the step of:

(h) if the NDC is not performing any steps of the method that are required for responding to a request to access a dataset, the method at NDC sites further including;

i. periodically analyzing channels to determine if such channel is presently no longer needed for responding to requests to access the dataset for which the channel was claimed; and

ii. if the channel being analyzed is no longer needed for responding to requests to access the dataset, then processing the no longer needed channel to prepare it for immediate claiming in response to a subsequent request from a client site to access another dataset.

23. The method of claim 22 wherein the periodic analysis of channels to determine if such channel is presently no longer needed for responding to requests is halted if the number of channels available to respond to a future request to access a dataset exceeds a pre-established upper threshold.

24. The method of claim 23 wherein the NDC having stopped periodically analyzing channels, resumes periodically analyzing channels if the number of channels available to respond to a future request to access a dataset drops below a pre-established lower threshold that is less than the pre-established upper threshold.

25. The method of claim 17 further comprising the steps of:

(h) each NDC, upon receiving the initial request to access the dataset from an upstream NDC site, allocating an upstream site structure in which the NDC stores information indicating the type of activity being performed on the dataset at the upstream NDC site from which this NDC site received the request;

(i) each NDC, upon receiving a request to access the dataset if the type of activity occurring on the dataset differs from that already being supported for the dataset, transmitting a message to its downstream NDC site informing the downstream NDC site of the new type of activity now occurring on the dataset; and

(j) each NDC, upon receiving a message from an upstream NDC site informing it of the type of activity being performed on the dataset at that upstream NDC site, storing such information in the upstream site structure for that upstream NDC site.

26. The method of claim 25 further comprising the steps of:

(k) each NDC, upon receiving information from a local client or from an upstream NDC site informing it of the type of activity being performed on the dataset at that location, comparing the information just received with previously stored information recording the type of activity occurring on the dataset; and

(l) if the NDC determines from its comparison of the information just received with the previously stored information that the dataset is being accessed by at least two different NDC sites and one of those NDC sites is accessing the dataset to write data into the dataset, then a concurrent write sharing (“

CWS”

) condition has occurred and the NDC declares itself to be a consistency control site (“

CCS”

) for the dataset.

27. The method of claim 26 after the NDC declares itself to be the CCS further comprising the steps of:

(m) if a request to access the dataset for writing it creates the CWS condition, then the CCS transmitting disable messages to all upstream NDC sites other than the NDC site whose request created the CWS condition instructing such upstream NDC sites to disable any projected image of the dataset stored in the NDC buffer of such NDC sites; and

(n) if a request to access the dataset for reading it creates the CWS condition, then transmitting a recall message to the upstream NDC site that is accessing the dataset to write data to the dataset instructing such upstream NDC site to transmit back to the CCS the projected image of the dataset stored in the NDC buffer of such NDC site.

28. The method of claim 27 further comprising the step of:

(o) the NDC that declared itself to be the CCS for the dataset including with the data returned to all upstream NDC sites that access the dataset an indicator that instructs each such upstream NDC site not to maintain a projected image of the data in their NDC buffer after the data has traversed the NDC site.

29. The method of claim 28 further comprising the steps of:

(p) the NDC that declared itself to be the CCS for the dataset, upon receiving information from a local client or from an upstream NDC site informing it of a cessation of a type of activity being performed on the dataset at that location, comparing the information just received with the previously stored information for the dataset;

(q) if the NDC determines from its comparison of the information just received with the previously stored information for the dataset that this NDC site need no longer function as the CCS for the dataset, then the NDC site ceasing to function as the CCS for the dataset; and

(r) upon this NDC site ceasing to function as the CCS for the dataset, permitting any upstream NDC site that is accessing the dataset to maintain in the NDC buffer of such NDC site an image of any subsequent projections of data from the dataset.

30. The method of claim 26 wherein;

before a disable message or a recall message transmitted upstream by a recently declared CCS reaches an NDC site having a projected image of the dataset present in the NDC buffer of that NDC site;

such NDC site transmits downstream, through a channel toward the NDC site that has declared itself to be the CCS, a request to access the dataset;

the request to access the dataset transmitted downstream arriving at an NDC site that has already transmitted the recall or disable message upstream;

the method further comprising the steps of;

(o) the NDC site, that receives the request to access the dataset after having transmitted the recall or disable message upstream, responds to the request by rejecting the request to access the dataset;

(p) each upstream site upon receiving the response rejecting the request to access the dataset successively retransmitting the rejection response upstream until the rejection message reaches the client terminator site for the request to access the dataset; and

(q) upon receiving the rejection message, the NDC of the NDC client terminator site releasing the channel, waiting a short interval of time to allow the recall or disable message from the downstream CCS to be processed, and then generating and transmitting downstream a new request to access the dataset.

31. The method of claim 25 further comprising the steps of:

(m) the NDC site having declared itself to be the CCS for the dataset, upon receiving information from a local client or from an upstream NDC site informing it of a cessation of a type of activity being performed on the dataset at that location, comparing the information just received with the previously stored information for the dataset; and

(n) if the NDC site determines from its comparison of the information just received with the previously stored information for the dataset that this NDC site need no longer function as the CCS for the dataset, then the NDC site ceasing to function as the CCS for the dataset.

32. A network of digital computers that includes a plurality of client sites which request access to a dataset that is stored at a location that can be accessed through the network, the network comprising:

a plurality of NDC sites, the dataset whose access is requested by the client sites being stored at an NDC server terminator site, a request from the client sites for access to the dataset being received by NDC client terminator sites, each NDC site including;

(a) an NDC that has an NDC buffer;

(b) means for the NDC to receive the request to access the dataset;

(c) means for the NDC to check the NDC buffer at this NDC site to determine if a projected image of data requested from the dataset is already present there wherein;

i. if the NDC buffer of this. NDC site does not contain a projected image of all data requested from the dataset, and if this NDC site is not the NDC server terminator site for the dataset, the NDC includes means for transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

ii. if the NDC buffer of this NDC site does not contain a projected image of all data requested from the dataset, and if this NDC site is the NDC server terminator site for the dataset, the NDC including means for accessing the dataset to project an image of the requested data into its NDC buffer; and

iii. if the NDC buffer of an NDC site contains a projected image of all requested data, the NDC including means for returning the data requested from it upstream to the NDC site from which it received the request, whereby through a succession of such returns of data from one NDC site to the next upstream NDC site the requested data ultimately arrives at the NDC client terminator site; and

(d) means for the NDC client terminator site to return the requested data to the client site.

33. The network of claim 32 wherein, prior to returning the requested data to the client site, said data return means reformats the requested data from the protocol employed for communications between pairs of NDC sites into the protocol in which the client site requested access to the dataset from the NDC client terminator site.

34. The network of claim 33 wherein the data return means selectively reformats the requested data from the protocol employed for communications between pairs of NDC sites into a particular one of a plurality of different protocols that matches the protocol in which the client site requested access to the dataset from the NDC client terminator site.

35. The network of claim 32 wherein each NDC site further comprises:

(e) means for the NDC, upon receiving the initial request to access the dataset, to claim a channel for storing various data relevant to processing requests to access the dataset;

(f) means for the NDC to store in the channel data collected by the NDC while processing requests to access the dataset;

(g) if the NDC site is not responding to a request to access dataset, means for the NDC to periodically analyze data stored in the channel to determine whether it is possible to anticipate future requests to access the dataset;

(h) if the analysis of data stored in the channel by the NDC establishes that it is possible to anticipate future requests to access the dataset, means for the NDC to further analyze data stored in the channel to determine whether anticipated future requests will soon present this NDC with another request to access the dataset and whether the projected image of data present in the NDC buffer includes sufficient data to respond immediately to the anticipated request; and

(i) if anticipated future requests will soon present this NDC site with another request to access the dataset and if the projected image of data present in the NDC buffer lacks sufficient data to respond immediately to the anticipated request, means for the NDC to request from the next downstream NDC site data from the dataset before receiving a request therefor.

36. The network of claim 35 wherein the NDC site requests from the next downstream NDC site a quantity of data from the dataset before receiving a request therefor, the quantity of data requested by the NDC site being larger than the amount of data returned by the NDC client terminator site to the requesting client site in response to a request to access the dataset received by the NDC client terminator site from the client site.

37. The network of claim 32 wherein each NDC site further comprises:

(e) means for the NDC, upon receiving the initial request to access the dataset, to claim a channel for storing various data relevant to processing requests to access the dataset;

(f) if the NDC is not responding to a request to access a dataset, means for the NDC to periodically analyze channels to determine if such channel is presently no longer needed for responding to requests to access the dataset for which the channel was claimed; and

(g) if analysis of the channel establishes that the channel is no longer needed for responding to requests to access the dataset, means for the NDC to process the no longer needed channel to prepare it for immediate claiming in response to a subsequent request from a client site to access another dataset.

38. The network of claim 37 wherein each NDC site further comprises means for halting the analysis of channels to determine if such channels are presently no longer needed for responding to requests if the number of channels available to respond to future requests to access datasets exceeds a pre-established upper threshold.

39. The network of claim 38 wherein each NDC site further comprises means for resuming the previously halted analysis of channels to determine if such channels are presently no longer needed for responding to requests if the number of channels available to respond to future requests to access datasets drops below a pre-established lower threshold that is less than the pre-established upper threshold.

40. The network of claim 32 wherein each NDC site further comprises:

(e) means for the NDC, upon receiving the initial request to access the dataset from an upstream NDC site, to allocate an upstream site structure in which the NDC stores information indicating the type of activity being performed on the dataset at the upstream NDC site from which this NDC site received the request;

(f) means for each NDC, upon receiving a request to access the dataset if the type of activity occurring on the dataset differs from that already being supported for the dataset, to transmit a message to its downstream NDC site informing the downstream NDC site of the new type of activity now occurring on the dataset; and

(g) means for each NDC, upon receiving a message from an upstream NDC site informing it of the type of activity being performed on the dataset at that upstream NDC site, to store such information in the upstream site structure for that upstream NDC site.

41. The network of claim 40 wherein each NDC site further comprises:

(h) means for each NDC, upon receiving information from a local client or from an upstream NDC site informing it of the type of activity being performed on the dataset at that location, to compare the information just received with the previously stored information recording the type of activity occurring on the dataset; and

(i) if the NDC determines from its comparison of the information just received with the previously stored information for the dataset that the dataset is being accessed by at least two different NDC sites and one of those NDC sites is accessing the dataset to write data into the dataset, then means for the NDC to declare itself to be a CCS to respond to the CWS condition that has occurred for the dataset.

42. The network of claim 41 wherein the NDC site that has declared itself to be the CCS further comprises:

(j) if a request to access the dataset for writing it creates the CWS condition, means for the NDC to transmit disable messages to all upstream NDC sites other than the NDC site whose request created the CWS condition instructing such upstream NDC sites to disable any projected image of the dataset stored in the NDC buffer of such NDC sites; and

(k) if a request to access the dataset for reading it creates the CWS condition, means for the NDC to transmit a recall message to the upstream NDC site that is accessing the dataset to write data to the dataset instructing such upstream NDC site to transmit back to the CCS the projected image of the dataset stored in the NDC buffer of such NDC site.

43. The network of claim 42 wherein each NDC site further comprises:

(l) means for the NDC that declared itself to be the CCS for the dataset to include with the data returned to all upstream NDC sites that access the dataset an indicator that instructs each such upstream NDC site not to maintain a projected image of the data in their NDC buffer after the data has traversed the NDC site.

44. The network of claim 43 wherein each NDC site further comprises:

(m) means for the NDC that declared itself to be the CCS for the dataset, upon receiving information from a local client or from an upstream NDC site informing it of a cessation of a type of activity being performed on the dataset at that location, to compare the information just received with the previously stored information for the dataset;

(n) if the NDC determines from its comparison of the information just received with the previously stored information for the dataset that this NDC site need no longer function as the CCS for the dataset, means for the NDC site to cease functioning as the CCS for the dataset; and

(o) upon this NDC site ceasing to function as the CCS for the dataset, means for the NDC site to permit any upstream NDC site that is accessing the dataset to maintain in the NDC buffer of such NDC site an image of any subsequent projections of data from the dataset.

45. The network of claim 41 wherein each NDC site before a disable message or a recall message transmitted upstream by a recently declared CCS reaches an NDC site having a projected image of the dataset present in the NDC buffer of that NDC site;

such NDC site transmits downstream, through a channel toward the NDC site that has declared itself to be the CCS, a request to access the dataset;

the request to access the dataset transmitted downstream arriving at an NDC site that has already transmitted the recall or disable message upstream;

each NDC site further comprising;

(l) means for the NDC site, that receives the request to access the dataset after having transmitted the recall or disable message upstream, to respond to the request by rejecting the request to access the dataset;

(m) means for each upstream site upon receiving the response rejecting the request to access the dataset successively retransmitting the rejection response upstream until the rejection message reaches the client terminator site for the request to access the dataset; and

(n) upon receiving the rejection message, means for the NDC of the NDC client terminator site to release the channel, wait a short interval of time to allow the recall or disable message from the downstream CCS to be processed, and then to generate and to transmit downstream a new request to access the dataset.

46. The network of claim 40 wherein each NDC site further comprises:

(j) means for the NDC site having declared itself to be the CCS for the dataset, upon receiving information from a local client or from an upstream NDC site informing it of a cessation of a type of activity being performed on the dataset at that location, to compare the information just received with the previously stored information for the dataset; and

(k) if the NDC site determines from its comparison of the information just received with the previously stored information for the dataset that this NDC site need no longer function as the CCS for the dataset, means for the NDC site to cease functioning as the CCS for the dataset.

47. In a digital computer system that includes high speed random access memory (“

RAM”

), a method for caching in the RAM of the computer system an image of a portion of data stored in a dataset that the computer system can access in response to a request to access such data, the method comprising the steps of;

(a) claiming a channel in the RAM of the computer system for storing metadata relevant to accessing data in the dataset;

(b) loading metadata into the channel;

(c) from a pool of buffers in the RAM of the computer system, assigning to the channel a buffer for storing an image of a portion of the data from the dataset;

(d) accessing the dataset to load an image of a portion of the data from the dataset into the buffer assigned to the channel; and

(e) responding to the request to access the data by supplying from the buffer an image of the data stored in such buffer together with a portion of the metadata stored in the channel.

48. The method of claim 47 wherein a plurality of buffers are assigned to the channel from the pool of buffers in the RAM of the computer system, each buffer for storing an image of a portion of the data from the dataset, and wherein the dataset is accessed to load images of a portion of the data from the dataset into such plurality of buffers for responding to the request to access data.

49. The method of claim 48 wherein the plurality of buffers that may be assigned to the channel is limited to a maximum number, the method further comprising the steps of:

(f) claiming a subchannel in the RAM of the computer system and associating the subchannel with the previously claimed channel;

(g) assigning a buffer from the pool of buffers in the RAM of the computer system to the subchannel for storing an image of a portion of the data from the dataset;

(h) accessing the dataset to load an image of a portion of the data from the dataset into the buffer assigned to the subchannel; and

(i) responding to the request to access the data by supplying from the buffer assigned to the subchannel an image of the data stored in such buffer together with a portion of the metadata stored in the channel.

50. The method of claim 49 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the method further comprising the steps of:

(j) selecting for further processing the least recently used of the previously claimed channels; and

(k) processing the selected channel to prepare it for immediate claiming in response to a subsequent request to access data stored in another dataset.

51. The method of claim 48 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the method further comprising the steps of:

(f) selecting for further processing the least recently used of the previously claimed channels; and

(g) processing the selected channel to prepare it for immediate claiming in response to a subsequent request to access data stored in another dataset.

52. The method of claim 47 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the method further comprising the steps of:

(f) selecting for further processing the least recently used of the previously claimed channels; and

(g) processing the selected channel to prepare it for immediate claiming in response to a subsequent request to access data stored in another dataset.

53. A cache for a digital computer system having a high speed RAM, said cache caching an image of a portion of data stored in a dataset in response to a request to access such data, said cache comprising:

(a) channel claiming means for claiming a channel in the RAM of the computer system, said channel being adapted for storing metadata relevant to accessing data in the dataset;

(b) means for loading metadata into said channel;

(c) a pool of buffers in the RAM of the computer system from which a buffer is assigned to said channel for storing an image of a portion of the data from the dataset;

(d) data loading means for accessing the dataset to load an image of a portion of the data from the dataset into said buffer assigned to said channel; and

(e) request response means for responding to the request to access the data by supplying from said buffer assigned to said channel an image of the data stored in such buffer together with a portion of the metadata stored in said channel.

54. The cache of claim 53 wherein a plurality of buffers are assigned to said channel from said pool of buffers in the RAM of the computer system, each buffer assigned to said channel storing an image of a portion of the data from the dataset, and wherein said data loading means accesses the dataset to load images of a portion of the data from the dataset-into such plurality of buffers for responding to the request to access data.

55. The cache of claim 54 wherein the plurality of buffers that may be assigned to said channel is limited to a maximum number, the cache further comprising:

(f) subchannel claiming means for claiming a subchannel in the RAM of the computer system and associating said subchannel with said channel; and

(g) a buffer assigned from said pool of buffers in the RAM of the computer system to said subchannel for storing an image of a portion of the data from the dataset, said data loading means accessing the dataset to load an image of a portion of the data from the dataset into said buffer assigned to said subchannel, and said request response means supplying from said buffer assigned to said subchannel an image of the data stored in such buffer together with a portion of the metadata stored in said channel in response to the request to access the data.

56. The cache of claim 55 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the cache further comprising:

(f) channel selection means for selecting for further processing the least recently used of the previously claimed channels; and

(g) channel readying means for processing said channel selected by the channel selection means to prepare it for immediate re-claiming in response to a subsequent request to access data stored in another dataset.

57. The cache of claim 54 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the cache further comprising:

(f) channel selection means for selecting for further processing the least recently used of the previously claimed channels; and

(g) channel readying means for processing said channel selected by the channel selection means to prepare it for immediate re-claiming in response to a subsequent request to access data stored in another dataset.

58. The cache of claim 53 wherein only a limited space in RAM is allocated for channels thereby establishing a maximum number of channels that may be simultaneously claimed for responding to requests to access data stored in datasets, the cache further comprising:

(f) channel selection means for selecting for further processing the least recently used of the previously claimed channels; and

(g) channel readying means for processing said channel selected by the channel selection means to prepare it for immediate re-claiming in response to a subsequent request to access data stored in another dataset.

59. In a network of digital computers that includes a plurality of NDC sites, each NDC site including an NDC that has channels which the NDC claims for storing therein various data relevant to processing requests to access datasets, a method for projecting images of stored datasets from an NDC server terminator site into an NDC client terminator site in response to requests to access such datasets transmitted from a client site to the NDC client terminator site, the method comprising the steps of:

(a) the NDC receiving a request to access data in a stored dataset;

(b) the NDC checking the claimed channels at this NDC site to determine if a channel has already been claimed for the dataset, and to determine from data stored in any channel claimed for the dataset if a projected image of data requested from the dataset is already present at this NDC site;

(c) if the NDC has not already claimed a channel for this dataset, the NDC of this NDC site claiming a channel for this dataset;

(d) if the channel claimed for the dataset at this NDC site indicates that a projected image of all data requested from the dataset is not present at this NDC site, and if the NDC site receiving the request is not the NDC server terminator site for the dataset, the NDC of this NDC site transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

(e) if the channel claimed for the dataset at this NDC site indicates that a projected image of all data requested from the dataset is not present at this NDC site, and if the NDC site receiving the request is the NDC server terminator site for the dataset, the NDC of this NDC site accessing the stored dataset to project an image of the requested data into this NDC site;

(f) storing into the channel for this dataset data collected by the NDC while processing the request to access the dataset;

(g) repeating the steps (a) through (f) until the downstream NDC site receiving the request has a projected image of all requested data;

(h) each successive NDC site, having obtained a projected image of all the requested data, returning data requested from it upstream to the NDC site from which it received the request until the requested data arrives at the NDC client terminator site, and each NDC site upon receiving data from a downstream NDC site also storing data received from such downstream NDC site into the channel for this dataset; and

(i) the NDC client terminator site, upon receiving the requested data, returning the requested data to the client site.

60. The method of claim 59 wherein the NDC of each NDC site checks only a fraction of all the claimed channels at the NDC site to determine if a channel has already been claimed for the dataset.

61. The method of claim 60 further comprising the steps of:

(j) the NDC maintaining a plurality of hash lists in which are included all the claimed channels at this NDC site, each hash list containing only a fraction of the claimed channels at the NDC site; and

(k) the NDC checking a hash list to determine if a channel has already been claimed for the dataset at this NDC site.

62. The method of claim 61 further comprising the step of the NDC assigning a buffer to the channel claimed for the dataset from a pool of buffers, the buffer assigned to the channel storing data from the dataset while such data traverses the NDC site.

63. A network of digital computers that includes a client site which requests access to datasets that are stored at a location that can be accessed through the network, the network comprising:

a plurality of NDC sites, the datasets whose access are requested by the client site being stored at an NDC server terminator site, a request from the client site for access to the datasets being received by an NDC client terminator site, each NDC site including;

(a) an NDC that has channels which the NDC claims for storing therein various data relevant to processing requests to access datasets;

(b) means for the NDC to receive the request to access a dataset;

(c) means for the NDC to check the claimed channels at this NDC site to determine if a channel has already been claimed for the dataset, and to determine from data stored in any channel claimed for the dataset if a projected image of data requested from the dataset is already present at this NDC site wherein;

i. if a channel has not already been claimed at this NDC site for this dataset, the NDC including means for claiming a channel;

ii. if the channel claimed for the dataset at this NDC site indicates that a projected image of all data requested from the dataset is not present at this NDC site, and if this NDC site is not the NDC server terminator site for the dataset, the NDC including means for transmitting a request for data from this NDC site downstream to another NDC site closer to the NDC server terminator site for the dataset than the present NDC site;

iii. if the channel claimed for the dataset at this NDC site indicates that a projected image of all data requested from the dataset is not present at this NDC site, and if this NDC site is the NDC server terminator site for the dataset, the NDC including means for accessing the dataset to project an image of the requested data into this NDC site;

iv. the NDC including means for storing into the channel for this dataset data collected by the NDC while processing the request to access the dataset; and

v. if an NDC site has a projected image of all requested data, the NDC including means for returning data requested from it upstream to the NDC site from which it received the request and for storing data received from any downstream NDC site into the channel for the dataset, whereby through a succession of such returns of data from one NDC site to the next upstream NDC site the requested data ultimately arrives at the NDC client terminator site; and

(d) data return means for returning the requested data from the NDC client terminator site to the client site.

64. The network of claim 63 wherein the NDC of each NDC site checks only a fraction of all the claimed channels at the NDC site to determine if a channel has already been claimed for the dataset.

65. The network of claim 64 in which each NDC further comprises:

vi. a plurality of hash lists in which are included all the claimed channels at this NDC site, each hash list containing only a fraction of the claimed channels at the NDC site; and

vii. means for checking a hash list to determine if a channel has already been claimed for the dataset at this NDC site.

66. The network of claim 65 in which each NDC further comprises:

viii. a pool of buffers; and

ix. means for the NDC to assign a buffer to the channel claimed for the dataset from said pool of buffers, the buffer assigned to the channel storing data from the dataset while such data traverses the NDC site.