Remote access agent for caching in a SAN file system

• US 20060004765A1
• Filed: 06/10/2004
• Published: 01/05/2006
• Est. Priority Date: 06/10/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gill, Binny Sher, Luan, Leo Shyh-Wei, Pereira, Manuel Vasconcellos III, Riegel, Geoffrey Albert, Anderson, Owen Theodore
• Granted Patent: US 7,685,128 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

Addressed is a system and method for remote data caching and replication by local copy maintenance of remote data within a SAN file system. Distributed Storage Tank (DST), an extension to a SAN file system, provides for transparent SAN client access of local copies by importing, exporting, and storing data using network file access protocols as well as by providing assurance of metadata and file content validity. A Remote Access Agent (RAA) handles protocol implementation and conversion necessary for communication with remote data sources. Controlled by a consistency policy, consistency is maintained by RAA fetching and updating local copies if modifications have occurred to a file since it was first stored as a local copy in local storage. Additionally, RAA returns metadata pertaining to the requested data. A SAN client obtains metadata corresponding to the requested data and utilizes it to directly access locally stored copies of remote data.

Claims

1. A method for storing and accessing remotely obtained data as a copy local to a Storage Area Network (SAN), wherein a Remote Access Agent (RAA):

a. obtains remote data from a remote source;

said remote source external to said SAN;

b. decouples attributes from content in said obtained remote data;

c. locally stores said decoupled attributes and a copy of said decoupled content, separately; and

d. utilizes said locally stored attributes to obtain file-level access to said local copy.

2. A method for storing and accessing, as per claim 1, wherein said RAA obtains said remote data from said remote source via protocol-specific communication.

3. A method for storing and accessing, as per claim 1, wherein said decoupled content is stored on a storage device attached to said SAN.

4. A method for storing and accessing, as per claim 3, wherein said decoupled attributes are locally stored as metadata in a metadata server.

5. A method for storing and accessing, as per claim 4, wherein said RAA converts decoupled attributes to a form of metadata compatible with said metadata server and sends said metadata to said metadata server.

6. A method for storing and accessing, as per claim 4, wherein said metadata is comprised of at least:

a content storage attribute, a directory mapping, and object attributes.

7. A method for storing and accessing, as per claim 6, wherein said content storage attribute is either of:

a storage object identifier and offset, or a list of block address;

said content storage attribute corresponding to a physical storage location for said decoupled content.

8. A method for storing and accessing, as per claim 7, wherein said file-level access comprises:

a. obtaining said metadata associated with said local copy;

b. extracting said content storage attribute from said obtained metadata; and

c. reading from or writing to a physical storage location in said SAN disk;

said physical location specified by said content storage attribute.

9. A method for maintaining consistency, in a Storage Area Network (SAN), of a local copy of a remote file system sub-tree obtained from a remote source;

said method comprising;

a. mapping directory structure of said remote file system sub-tree to a remote container;

said remote container attached to said SAN;

b. representing each remote object comprising said remote file system sub-tree as a local object component of said remote container;

c. labeling each of said local objects with attributes associated with said represented remote object;

d. storing metadata describing each of said local objects in a metadata server;

e. associating a consistency policy with each of said local objects in said remote container;

said policy defining conditions for checking freshness of said labeled attributes; and

f. updating said local object components of remote container in accordance with said consistency policy.

10. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 9, wherein said local objects are hierarchically structured within said remote container in a manner corresponding to hierarchical structure of said remote objects comprising said remote file system sub-tree.

11. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 9, wherein said local and remote objects both represent either of:

a directory or a file.

12. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 11, wherein said local object is comprised of at least:

a directory entry and directory attributes;

if said local object represents a directory; and

at least;

file content and file attributes;

if said local object represents a file.

13. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 12, wherein said metadata is comprised of at least a local object attribute;

either of;

a content storage attribute, if said local object is a file;

or at least one directory entry, if said local object is a directory; and

any of;

a consistency guarantee.

14. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 13, wherein representing of said remote object as a local object comprises:

a. storing said file content at a physical location specified by said content storage attribute;

if said local object represents a file; and

b. populating said local object with said directory entries;

if said local object represents a directory.

15. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 13, wherein said freshness checking comprises:

a. triggering a check of said labeled attributes associated with said local object;

b. checking for the existence of said metadata defining a consistency guarantee for said local object;

c. fetching from said remote source, attributes associated with said remote object represented by said local object, based on results of said checking step;

d. creating a consistency guarantee for said local object, based on results of said fetching step; and

e. merging said fetched attributes with said attributes associated with said local object.

16. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 15, wherein said check of labeled attributes is triggered by either of:

synchronous or asynchronous methods.

17. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 15, wherein said fetched attributes are accompanied by either of:

an explicit or an implicit consistency guarantee.

18. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 17, wherein said implicit consistency guarantee exists for a pre-determined period of time.

19. A method for maintaining consistency, in a Storage Area Network (SAN), as per claim 17, wherein said explicit consistency guarantee exists until notification is received from said remote source.

20. A method for maintaining consistency in a Storage Area Network (SAN), as per claim 15, wherein said freshness checking further comprises determining validity of said local object content by comparing said fetched attributes with said labeled attributes, based on results of said checking step;

said comparison validating said local object if said labeled attributes match attributes of said local content, else invalidating said local object content.

21. A method for maintaining consistency in a Storage Area Network (SAN), as per claim 15, wherein said local object is updated with modifications made to a remote object imported from said remote source.

22. A method for maintaining consistency in a Storage Area Network (SAN), as per claim 15, wherein said remote object is updated with modifications made to a local object exported to said remote source.

23. A method for refreshing an object local to a Storage Area Network (SAN) in accordance with a consistency guarantee;

said local object representing a remote file system sub-tree;

said local object refreshing with a remote object imported from a remote source;

said method comprising;

a. retrieving remote data from said remote source;

b. decoupling content and attributes from said remote data;

c. modifying said local object on the basis of said decoupled attributes and content;

d. labeling said local object with said decoupled attributes; and

e. updating metadata associated with said local object.

24. A method for refreshing an object local to a Storage Area Network (SAN), as per claim 23, wherein said retrieval occurs after either of:

the expiration of a time interval or the reception of a notification from said remote source;

based on said consistency guarantee.

25. A method for updating a remote object at a remote source with modifications made to an object local to a Storage Area Network (SAN);

said local object representing a remote file system sub-tree;

said update in accordance with an update consistency policy;

said method comprising;

a. detecting modifications to a local object;

said local object marked as modified;

b. obtaining attributes and content associated with said marked local object;

c. exporting said obtained attributes and content to said remote source; and

d. receiving remote object attributes and a consistency guarantee from said remote source.

26. A method for refreshing an object local to a Storage Area Network (SAN), as per claim 25, wherein said update consistency policy is defined by at least one of:

a delay and a delegation method.

27. A method for refreshing an object local to a Storage Area Network (SAN), as per claim 26, wherein frequency of said updates is determined by frequency at which delegation notification is received from said remote source.

28. A method for refreshing an object local to a Storage Area Network (SAN), as per claim 26, wherein said delay determines time interval between said detection of modifications to said local object.

29. A system for importing, storing, and exporting, in a Storage Area Network (SAN), a local copy of remote file system data;

said system comprising;

a. a client on said SAN requesting, storing, and accessing said local copy;

b. a metadata server, local to said SAN, storing metadata describing attributes of said local copy;

c. a source of remote file system data external to said SAN;

d. a Remote Access Agent (RAA) sending said local copy to and receiving said remote file from, said remote source;

a storage device, attached to said SAN, storing content of said local copy.

30. A system for importing, storing, and exporting, as per claim 29, wherein said storage device attached to SAN is a SAN disk.

31. A system for importing, storing, and exporting, as per claim 29, wherein said RAA stores said content of local copy in said storage device attached to SAN.

32. A system for importing, storing, and exporting, as per claim 29, wherein said RAA converts said attributes of said local copy to a form of metadata compatible with said metadata server and sends said metadata to said metadata server.

33. A system for importing, storing, and exporting, as per claim 29, wherein said metadata is comprised of at least a local copy attribute;

either of;

a content storage attribute, if said local copy represents a file;

or at least one directory entry, if said local copy represents a directory; and

any of;

a consistency guarantee.

34. A system for importing, storing, and exporting, as per claim 29, wherein said RAA obtains said remote file system data from said external, remote source via protocol-specific communication.

35. A system for importing, storing, and exporting, as per claim 29, wherein said client obtains metadata from said metadata server to perform either of:

a read operation or a write operation upon said content of a local copy stored in said storage device attached to SAN.

36. A system for importing, storing, and exporting, as per claim 29, wherein said content of local copy is obtained from said remote source in varying increments of granularity.

37. A system for importing, storing, and exporting, as per claim 33, wherein said content storage attribute is either of:

a storage object identifier and offset, or a list of block address;

said content storage attribute corresponding to a physical storage location in said storage device for said content of a local copy.

38. A system for importing, storing, and exporting, as per claim 35, wherein said client sends to said metadata server, at least one attribute pertaining to said operations performed on said local copy.

39. A system for importing, storing, and exporting, as per claim 37, wherein said RAA stores said content of local copy in said storage device using said content storage attribute.

40. A system for importing, storing, and exporting, as per claim 38, wherein said RAA obtains and forwards to said remote source, modification resulting from said operation performed on said content of a local copy from said storage device and said at least one attribute pertaining to said operations from said metadata server.

41. A system for updating, storing, and accessing, in a Storage Area Network (SAN), a local copy of file system data from a remote source;

said system comprising;

a. a client interface, connected to a SAN client, sending to and receiving from said SAN client, metadata and a request for access to said local copy;

b. a remote container module connected to said client interface;

said remote container module managing consistency of said local copy and processing said request;

c. a cache manager connected to said remote container module managing and allocating storage space for said local copy;

d. a replica manager connected to said remote container module;

said replica manager maintaining consistency of said local copy if validation is asynchronous type;

e. a distributed file manager connected to said remote container module;

said distributed file manager updating said local copy by obtaining said remote source file system data; and

f. a consistency policy maintenance framework connected to;

said remote container module, said distributed file manager;

said replica manager;

said cache manager storing conditions governing consistency management of said local copy.

42. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 41, wherein said consistency policy maintenance framework stores conditions comprising at least:

validation policy and type for said local copy and a policy on storage space replacement.

43. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 42, wherein said a remote container module processes said access request by determining whether access is allowed to said local copy for which said access request was received.

44. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 43, wherein said determination is made by checking metadata to ensure existence of said local copy and if said local copy exists, verifying the validity of said local copy.

45. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 43, wherein said remote container module verifies the validity of said local copy by consulting said consistency policy maintenance framework to obtain said validation policy.

46. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 44, wherein said access request is granted if said local copy exists and is valid;

said access request denied otherwise.

47. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 45, wherein said validation policy comprises checking for either of:

the existence consistency guarantee or proof that said remote file system data has been imported within a time bound specified by said validation policy.

48. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 46, wherein said remote container module manages consistency of said local copy by delegating to said distributed file manager, the task of importing said remote file system data to update said local copy if said access request is denied.

49. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 48, wherein said distributed file manager imports said remote file system data in accordance with at least a local copy update policy obtained from said consistency policy maintenance framework.

50. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 48, wherein said remote container module manages consistency of said local copy by delegating to said cache manager, the task of allocating storage space for said imported remote file system data.

51. A system for updating, storing, and accessing, in a Storage Area Network (SAN), as per claim 50, wherein said cache manager consults said consistency policy maintenance framework to obtain a policy on storage space replacement in service of said storage space allocation task.