On-demand caching in a WAN separated distributed file system or clustered file system cache

• US 9,298,719 B2
• Filed: 09/04/2012
• Issued: 03/29/2016
• Est. Priority Date: 09/04/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Noronha, Ranjit M., Patil, Sandeep R., Singh, Ajay K., Walde, Ravindra P., Zadbuke, Kailas S.
• Primary Examiner(s): RUBIN, BLAKE J
• Application Number:
  US13/602,883
• Publication Number:
  US 20140067988A1
• Time in Patent Office: 1,302 Days
• Field of Search: 709/213
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0888 using selective caching, e.g. bypass
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics

Abstract

A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Claims

1. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device causes the computing device to:

monitor, by a home site in a separated distributed file system or clustered file system, file access by a plurality of cache sites in the separated distributed file system or clustered file system;

record, by the home site, file accesses in an activity table, wherein the activity table is a two-dimensional table indexed by time over a predetermined interval P and a granularity G, wherein each element in the activity table is a linked list of read and write elements;

transform the activity table into a sorted activity table wherein each element in the sorted activity table is a linked list of read and write elements sorted by time;

convert the sorted activity table into a converted activity table by creating a link to a preceding write for each element in the stored activity table having only reads and creating a link to a next set of reads for each element in the sorted activity table having only writes;

identify, by the home site and based on the converted activity table, access patterns by cache sites; and

share the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

2. The computer program product of claim 1, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system.

3. The computer program product of claim 2, wherein the computer readable program further causes the computing device to:

performing analysis to identify at least one active file; and

responsive to identifying at least one active file, informing at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.

4. The computer program product of claim 1, wherein the given cache site performs analysis to identify files to pre-fetch.

5. The computer program product of claim 1, wherein the given cache site performs analysis to generate a pre-fetch schedule.

6. The computer program product of claim 1, wherein the given cache site resets at least one revalidation interval.

7. The computer program product of claim 1, wherein sharing the access patterns comprises sending the converted activity table to the plurality of cache sites.

8. The computer program product of claim 1, wherein the computer readable program further causes the computing device to:

responsive to receiving a write for a data element at the home site, determine subsequent predicted reads from at least one cache site based on the converted activity table; and

push the data dement to the at least one cache site.

9. The computer program product of claim 8, wherein determining subsequent predicted reads comprises weighing data on heuristic patterns across different cache site to schedule predictive prefetching of files.

10. The computer program product of claim 9, wherein determining subsequent predicted reads further comprises:

responsive to a given user accessing a first data element from a given cache site, identifying a second data element that is likely to be accessed by the given user from the given cache site in a predetermined interval of time and nominating the second data element for pre-fetching.

11. The computer program product of claim 1, wherein the computer readable program is stored in a computer readable storage medium in a data processing system and wherein the computer readable program was downloaded over a network from a remote data processing system.

12. The computer program product of claim 1, wherein the computer readable program is stored in a computer readable storage medium in a server data processing system and wherein the computer readable program is downloaded over a network to a remote data processing system for use in a computer readable storage medium with the remote system.

13. A method, in a data processing system, for on-demand caching in a separated distributed file system or clustered file system, the method comprises:

Monitoring file access by a plurality of cache sites in the separated distributed file system or clustered file system;

recording file accesses in an activity table, wherein the activity table is a two-dimensional table indexed by time over a predetermined interval P and a granularity G, wherein each element in the activity table is a linked list of read and write elements;

transforming the activity table into a sorted activity table wherein each element in the sorted activity table is a linked list of read and write elements sorted by time;

convert the sorted activity table into a converted activity table by creating a link to a preceding write for each element in the stored activity table having only reads and creating a link to a next set of reads for each element in the sorted activity table having only writes;

identifying, based on the converted activity table, access patterns by cache sites; and

sharing the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

14. The method of claim 13, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system.

15. The method of claim 14, further comprising:

performing analysis to identify at least one active file; and

responsive to identifying at least one active file, informing at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.

16. The method of claim 13, wherein the given cache site performs analysis to identify files to pre-fetch and generate a pre-fetch schedule.

17. The method of claim 13, wherein the given cache site to reset at least one revalidation interval.

18. The method of claim 13, wherein sharing the access patterns comprises sending the converted activity table to the plurality of cache sites.

19. The method of claim 13, further comprises:

responsive to receiving a write for a data element at the home site, determine subsequent predicted reads from at least one cache site based on the converted activity table; and

push the data dement to the at least one cache site.

20. An apparatus, comprising:

a processor, and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

monitor file access by a plurality of cache sites in the separated distributed file system or clustered file system;

record file accesses in an activity table, wherein the activity table is a two-dimensional table indexed by time over a predetermined interval P and a granularity G, wherein each element in the activity table is a linked list of read and write elements;

transform the activity table into a sorted activity table wherein each element in the sorted activity table is a linked list of read and write elements sorted by time;

convert the sorted activity table into a converted activity table by creating a link to a preceding write for each element in the stored activity table having only reads and creating a link to a next set of reads for each element in the sorted activity table having only writes;

identify, based on the converted activity table, access patterns by cache sites; and

share the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

21. The apparatus of claim 20, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system, wherein instructions further cause the processor to:

perform analysis to identify at least one active file; and

responsive to identifying at least one active file, inform at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.

22. The computer program product of claim 20, wherein the computer readable program further causes the computing device to:

responsive to receiving a write for a data element at the home site, determine subsequent predicted reads from at least one cache site based on the converted activity table; and

push the data dement to the at least one cache site.