On-Demand Caching in a WAN Separated Distributed File System or Clustered File System Cache

• US 20140067988A1
• Filed: 09/04/2012
• Published: 03/06/2014
• Est. Priority Date: 09/04/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Noronha, Ranjit M., Patil, Sandeep R., Singh, Ajay K., Walde, Ravindra P., Zadbuke, Kailas S.
• Granted Patent: US 9,298,719 B2
• Time in Patent Office: Days
• US Class Current: 709/213
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0888 using selective caching, e.g. bypass
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics

Abstract

A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

monitor file access by a plurality of cache sites in a separated distributed file system or clustered file system;

identify access patterns by cache sites; and

share the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

2. The computer program product of claim 1, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system.

3. The computer program product of claim 2, wherein the computer readable program further causes the computing device to:

performing analysis to identify at least one active file; and

responsive to identifying at least one active file, informing at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.

4. The computer program product of claim 1, wherein the given cache site performs analysis to identify files to pre-fetch.

5. The computer program product of claim 1, wherein the given cache site performs analysis to generate a pre-fetch schedule.

6. The computer program product of claim 1, wherein the given cache site to reset at least one revalidation interval.

7. The computer program product of claim 1, wherein monitoring file access comprises recording file accesses in an activity table, wherein the activity table is a two-dimensional table indexed by time running from zero to a predetermined interval P and a granularity G, wherein each element in the activity table is a linked list of read and write elements.

8. The computer program product of claim 7, wherein identifying access patterns by cache sites comprises:

transforming the activity table into a sorted activity table wherein each element in the sorted activity table is a linked list of read and write elements sorted by time;

converting the sorted activity table into a converted activity table by creating a link to a preceding write for each element in the sorted activity table having only reads and creating a link to a next set of reads for each element in the sorted activity table having only writes.

9. The computer program product of claim 8, wherein sharing the access patterns comprises sending the converted activity table to the plurality of cache sites.

10. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

responsive to receiving a write for a data element at the home site, determine subsequent predicted reads from at least one cache site based on the converted activity table; and

push the data element to the at least one cache site.

11. The computer program product of claim 10, wherein determining subsequent predicted reads comprises weighing data on heuristic patterns across different cache site to schedule predictive prefetching of files.

12. The computer program product of claim 11, wherein determining subsequent predicted reads further comprises:

responsive to a given user accessing a first data element from a given cache site, identifying a second data element that is likely to be accessed by the given user from the given cache site in a predetermined interval of time and nominating the second data element for pre-fetching.

13. The computer program product of claim 1, wherein the computer readable program is stored in a computer readable storage medium in a data processing system and wherein the computer readable program was downloaded over a network from a remote data processing system.

14. The computer program product of claim 1, wherein the computer readable program is stored in a computer readable storage medium in a server data processing system and wherein the computer readable program is downloaded over a network to a remote data processing system for use in a computer readable storage medium with the remote system.

15. A method, in a data processing system, for on-demand caching in a separated distributed file system or clustered file system, the method comprises:

monitoring file access by a plurality of cache sites in the separated distributed file system clustered file system;

identifying access patterns by cache sites; and

sharing the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

16. The method of claim 15, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system.

17. The method of claim 16, further comprising:

performing analysis to identify at least one active file; and

responsive to identifying at least one active file, informing at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.

18. The method of claim 17, wherein the given cache site performs analysis to identify files to pre-fetch and generate a pre-fetch schedule.

19. The method of claim 17, wherein the given cache site to reset at least one revalidation interval.

20. The method of claim 17, wherein monitoring file access comprises recording file accesses in an activity table, wherein the activity table is a two-dimensional table indexed by time running from zero to a predetermined interval P and a granularity G, wherein each element in the activity table is a linked list of read and write elements.

21. The method of claim 20, wherein identifying access patterns by cache sites comprises:

transforming the activity table into a sorted activity table wherein each element in the sorted activity table is a linked list of read and write elements sorted by time; and

convening the sorted activity table into a converted activity table by creating a link to a preceding write for each element in the sorted activity table having only reads and creating a link to a next set of reads for each element in the sorted activity table having only writes.

22. The method of claim 21, wherein sharing the access patterns comprises sending the converted activity table to the plurality of cache sites.

23. The method of claim 21, further comprising:

responsive to receiving a write for a data element at the home site, determining subsequent predicted reads from at least one cache site based on the converted activity table; and

pushing the data element to the at least one cache site.

24. An apparatus, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

monitoring file access by a plurality of cache sites in a separated distributed file system or clustered file system;

identifying access patterns by cache sites; and

sharing the access patterns with the plurality of cache sites, wherein a given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

25. The apparatus of claim 24, wherein the home site maintains statistics of file access patterns with respect to files in the separated distributed file system or clustered file system, wherein instructions further cause the processor to:

perform analysis to identify at least one active file; and

responsive to identifying at least one active file, inform at least one cache site within the plurality of cache sites of the at least one active file, wherein the at least one cache site pre-fetches the at least one active file.