DISCONNECTED FILE OPERATIONS IN A SCALABLE MULTI-NODE FILE SYSTEM CACHE FOR A REMOTE CLUSTER FILE SYSTEM

• US 20110145363A1
• Filed: 12/16/2009
• Published: 06/16/2011
• Est. Priority Date: 12/16/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Haskin, Roger L., Ananthanarayanan, Rajagopol, Naik, Manoj P., Eshel, Marc M., Hildebrand, Dean, Schmuck, Frank B., Tewari, Renu
• Granted Patent: US 8,473,582 B2
• Time in Patent Office: Days
• US Class Current: 709/218
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems

Abstract

Facilitating access to data in a network, is provided. One implementation involves maintaining a scalable multi-node file system cache in a local cluster file system, and caching file data in a cache by fetching file data on demand from a remote cluster file system into the local cached file system over the network. The local file data corresponds to file data in the remote cluster file system. Upon disconnection from the remote cluster file system, all file operations are supported at the cache. Then, upon reconnection to the remote cluster file system over the network, the remote cluster file system is synchronized with the information cached in the cache during the disconnection even in the presence of failures.

Claims

1. A method for facilitating access to data in a network, comprising:

maintaining a scalable multi-node file system including a cache in a local cluster file system including multiple computing nodes;

caching file data in the cache by fetching file data on demand from a remote cluster file system into the cache over the network, wherein the local file data corresponds to the file data in the remote cluster file system;

upon disconnection from the remote cluster file system, supporting file data and metadata update and access operations at the cache; and

upon reconnection to the remote cluster file system over the network, synchronizing the remote cluster file system with the information cached in the cache during the disconnection.

2. The method of claim 1, wherein supporting file operations at the cache during said disconnection further comprises:

supporting data and metadata writes to the cache during temporary or long term network outages or remote cluster file system outages.

3. The method of claim 2, wherein supporting file operations at the cache during disconnection further comprises:

maintaining a cache state representing cache updates during a disconnection, wherein the cache state includes information for recreating at the remote cluster file system the entire state of the local data cached during disconnection upon reconnection.

4. The method of claim 3, wherein synchronization upon reconnection further comprises:

recreating the entire state of the local cached data at the remote cluster file system using said cache state information to determine the updates to the cache during the disconnection, without explicitly logging each local cache operation or replaying a local event log.

5. The method of claim 4, wherein the cache state includes type information for each data object inode in the cache, denoting the type of updates performed on each data object in the cache during a disconnection.

6. The method of claim 5, wherein:

supporting file data operations at the cache during disconnection further comprises;

storing file data changes in the cache and maintaining a corresponding cache state;

synchronization upon reconnection further comprises;

a local gateway node application of the local cluster file system updating the remote file cluster system data with the cache data over the network using the cache state information, to recreate the entire state of the local cached data at the remote cluster file system.

7. The method of claim 1, wherein caching local file data in the cache further comprises:

during connection between the local cluster file system and the remote cluster file system;

receiving a request for file access an application node of the local cached file system;

the application node checking the cache for the requested file;

upon detecting a cache hit, the application node accessing the file data in the cache; and

upon detecting a cache miss, fetching file data on demand from the remote cluster file system into the cache over the network, wherein fetching file data on demand includes the application node signaling a gateway node to fetch the file data and the gateway node fetching the file data from the remote cluster file system into the cache over the network for access by the application node, wherein fetching the file data from the remote cluster file system further includes fetching the file data from the remote cluster file system into the cache over a wide area network using a parallel network file system data transfer.

8. The method of claim 1, wherein the local cluster file system and the remote cluster file system comprise General Parallel File Systems, the cache comprises a storage subsystem of the local cluster, and the cache is visible to any file system client node as a Portable Operating System Interface compliant file system, allowing a file system client node to browse the cache and access the cached remote file data in a local file system.

9. A system for facilitating access to data in a network, comprising:

a cache subsystem including a cache for maintaining data for a local cluster file system including multiple computing nodes;

a caching layer function configured for caching file data in the cache including fetching file data on demand from a remote cluster file system into the cache over the network, wherein the local file data corresponds to file data in the remote cluster file system; and

the caching layer function further configured such that upon disconnection from the remote cluster file system, the caching layer function supports file data and metadata update and access operations at the cache, and upon reconnection to the remote cluster file system the caching layer function synchronizes the remote cluster file system with the information cached in the cache during the disconnection.

10. The system of claim 9, wherein the caching layer function is further configured for supporting data and metadata writes to the cache during temporary or long term network outages or remote cluster file system outages.

11. The system of claim 10, wherein the caching layer function is further configured for maintaining a cache state representing cache updates during a disconnection, wherein the cache state includes information for recreating at the remote cluster file system upon reconnection, the entire state of the local data cached during disconnection.

12. The system of claim 11, wherein the caching layer function is further configured for synchronization upon reconnection by recreating the entire state of the local cached data at the remote cluster file system using said cache state information to determine the updates to the cache during the disconnection, without explicitly logging each local cache operation or replaying a local event log.

13. The system of claim 12 wherein the cache state includes type information per data object inode in the cache, denoting the type of updates performed on each data object in the cache during a disconnection.

14. The system of claim 13 wherein:

the caching layer function is further configured for supporting data operations at the local cache during disconnection by storing file data changes in the cache and maintaining a corresponding cache state; and

the caching layer function includes a local gateway node application configured for updating the remote file cluster system data with the cache data over the network, using the cache state information, to recreate the entire state of the local cached data at the remote cluster file system.

15. The system of claim 9, wherein the local cluster file system and the remote cluster file system comprise General Parallel File System, the cache comprises a storage subsystem of the local cluster, and the cache is visible to any file system client node as a Portable Operating System Interface compliant file system, allowing a file system client node to browse the cache and access the cached remote file data as in a local file system.

16. A computer program product for facilitating access to data, the computer program product comprising:

a computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising;

computer usable program code configured to maintain a cache subsystem maintaining data in a cache for a local cluster file system;

computer usable program code configured to provide a caching layer function for caching file data in the cache by fetching file data on demand from a remote cluster file system into the cache over the network, wherein the local file data corresponds to the file data in the remote cluster file system;

computer usable program code configured to, upon disconnection from the remote cluster file system, support file data and metadata update and access operations at the cache; and

computer usable program code configured to, upon reconnection to the remote cluster file system over the network, synchronize the remote cluster file system with the information cached in the cache during the disconnection.

17. The computer program product of claim 16 further comprising:

computer usable program code configured to support data and metadata writes to the cache during temporary or long term network outages or remote cluster file system outages.

18. The computer program product of claim 17 further comprising:

computer usable program code configured to maintain a cache state representing cache updates during a disconnection, wherein the cache state includes information for recreating at the remote cluster file system the entire state of the local data cached during disconnection, upon reconnection; and

computer usable program code configured to, upon reconnection, recreate at the remote cluster file system the entire state of the local cached data using said cache state information to determine the updates to the cache during the disconnection, without explicitly logging each local cache operation or replaying a local event log.

19. The computer program product of claim 18 wherein the cache state includes type information per data object inode in the cache, denoting the type of updates performed on each data object in the cache during a disconnection.

20. The computer program product of claim 19 further comprising:

computer usable program code configured to support data operations at the local cache during disconnection by storing file data changes in the cache and maintaining a corresponding cache state, synchronize upon reconnection wherein a local gateway node application updates the remote file cluster system data with the cache data over the network using the cache state information to recreate the entire state of the local cached data at the remote cluster file system;

wherein the local cluster file system and the remote cluster file system comprise General Parallel File Systems, the cache comprises a storage subsystem of the local cluster, and the cache is visible to any file system client node as a Portable Operating System Interface compliant file system, allowing a file system client node to browse the cache and access the cached remote file data as in a local file system.