File system architecture requiring no direct access to user data from a metadata manager

• US 7,617,321 B2
• Filed: 05/07/2004
• Issued: 11/10/2009
• Est. Priority Date: 05/07/2004
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Clark, Thomas K.
• Primary Examiner(s): Srivastava; Vivek
• Assistant Examiner(s): Goodchild; William J
• Application Number:
  US10/840,983
• Publication Number:
  US 20050251522A1
• Time in Patent Office: 2,013 Days
• Field of Search: 709/229, 707/9, 707/10
• US Class Current: 709/229
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: Y10S 707/99939 Privileged access

Abstract

A system, method and computer program product for implementing a distributed file system architecture requiring no direct access to user data from a metadata manager in a data storage network. Metadata operations requiring user volume access are performed via a proxy on a storage manager server that is a distributed file system client. Management of metadata can be isolated from user data because the metadata manager requires no capability to see the storage for user volumes. This allows for a distributed file management system to support heterogeneous environments without requiring any single point in the system to see all the storage.

Claims

1. A method for implementing a data storage network distributed file system without requiring direct access to user data from a metadata manager, comprising:

establishing one or more metadata storage volumes that are accessible by said metadata manager on a single homogeneous type of one or more data storage devices;

storing all distributed file system metadata on said one or more metadata storage volumes on said single homogeneous type of one or more data storage devices;

establishing one or more user data storage volumes that are accessible by one or more storage manager servers on one or more types of data storage device;

said metadata manager only accessing said one or more metadata storage volumes in said data storage network, and not directly accessing said one or more user data storage volumes;

generating administrative requests from said metadata manager to a proxy in one of said storage manager servers for metadata manager access to said one or more user data volumes in said data storage network for performing administrative actions specified by a data storage network administrator or administration program, said administrative requests involving said metadata manager requesting manipulation of data stored on said one or more user data volumes on its own behalf in support of said administrative actions; and

accessing said one or more user data volumes from said storage manager server in response to said requests to said proxy.

2. A method in accordance with claim 1 wherein said requests comprise volume commission requests and include parameters for consumption by said proxy that include a volume identifier and a volume label to be written.

3. A method in accordance with claim 2 wherein said proxy causes a volume label to be written to a data volume according to said parameters passed by said metadata manager.

4. A method in accordance with claim 1 wherein said requests include volume drain and data move requests and include parameters for consumption by said proxy that include a read volume identifier, a read volume offset, a write volume identifier, a write volume offset, and a data size indicator.

5. A method in accordance with claim 4 wherein said proxy causes data to be read from a read volume and said data to be written to a write volume according to said parameters passed by said metadata manager.

6. A method in accordance with claim 1 wherein said requests comprise remote procedure calls.

7. A method in accordance with claim 1 wherein said requests include volume commission requests, volume decommission requests, volume drain requests, data move requests, data backup requests and data restore requests.

8. A method system in accordance with claim 1 wherein said storage manager server implements logical volume management.

9. A system for implementing a data storage network distributed file system without requiring direct access to user data from a metadata manager, comprising:

a metadata manager;

one or more storage manager servers;

one or more metadata storage volumes that are accessible by said metadata manager on a single homogeneous type of one or more data storage devices;

one or more user data storage volumes that are accessible by said one or more storage manager servers on one or more types of data storage device;

all metadata for said distributed file system being stored on said one or more metadata storage volumes on said single homogeneous type of one or more data storage devices;

said metadata manager comprising one or more processors and a memory coupled to said processor(s), said memory including a computer readable storage medium tangibly embodying at least one program of instructions executable by said processor(s) to perform operations, comprising;

accessing said one or more metadata storage volumes in said data storage network, and not directly accessing said one or more user data storage volumes; and

generating administrative requests from said metadata manager to a proxy in one of said storage manager servers for metadata manager access to said one or more user data volumes in said data storage network for performing administrative actions specified by a data storage network administrator or administration program, said administrative requests involving said metadata manager requesting manipulation of data stored on said one or more user data volumes on its own behalf in support of said administrative actions; and

said storage manager servers comprising one or more processors and a memory coupled to said processor(s), said memory including a computer readable storage medium tangibly embodying at least one program of instructions executable by said processor(s) to perform operations, comprising;

implementing a proxy for receiving said administrative requests from said metadata manager for metadata manager access to said one or more user data volumes in said data storage network;

receiving said administrative requests from said metadata manager; and

accessing said user data volumes in response to said requests received by said proxy.

10. A system in accordance with claim 9 wherein said requests comprise volume commission requests and include parameters for consumption by said proxy that include a volume identifier and a volume label to be written.

11. A system in accordance with claim 10 wherein said storage manager server causes a volume label to be written to a data volume according to said parameters passed by said metadata manager.

12. A system in accordance with claim 9 wherein said requests comprise volume drain and data move requests and include parameters for consumption by said proxy that include a read volume identifier, a read volume offset, a write volume identifier, a write volume offset, and a data size indicator.

13. A system in accordance with claim 12 wherein said storage manager server causes data to be from a read volume and said data to be written to a write volume according to said parameters passed by said metadata manager.

14. A system in accordance with claim 9 wherein said requests comprise remote procedure calls.

15. A system in accordance with claim 9 wherein said requests include volume commission requests, volume decommission requests, volume drain requests, data move requests, data backup requests and data restore requests.

16. A system in accordance with claim 9 wherein said storage manager server implements logical volume management.

17. A metadata manager for implementing a data storage network distributed file system without requiring direct access to user data from said metadata manager, comprising:

one or more processors;

a memory coupled to said processor(s), said memory including a computer readable storage medium tangibly embodying at least one program of instructions executable by said processor(s) to perform operations, comprising;

managing one or more metadata storage volumes that are accessible by said metadata manager on a single homogeneous type of one or more data storage devices;

storing all distributed file system metadata on said one or more metadata storage volumes on said single homogeneous type of one or more data storage devices;

said metadata storage volumes being distinct from one or more user data storage volumes that are accessible by one or more storage manager servers on one or more types of data storage device;

said metadata manager only accessing said one or more metadata storage volumes in said data storage network, and not directly accessing said one or more user data storage volumes; and

generating administrative requests from said metadata manager to a proxy in one of said storage manager servers for metadata manager access to said one or more user data volumes in said data storage network for performing administrative actions specified by a data storage network administrator or administration program, said administrative requests involving said metadata manager requesting manipulation of data stored on said one or more user data volumes on its own behalf in support of said administrative actions.

18. A storage manager server for implementing a data storage network distributed file system without direct access to user data from a metadata manager, comprising:

one or more processors;

a memory coupled to said processor(s), said memory including a computer readable storage medium tangibly embodying at least one program of instructions executable by said processor(s) to perform operations, comprising;

supporting access to one or more user data storage volumes by said storage manager server on one or more types of data storage device and which are not directly accessible by a metadata manager that manages one or more metadata storage volumes that are accessible by said metadata manager on a single homogeneous type of one or more data storage devices, said metadata storage volumes storing all distributed file system metadata on said single homogeneous type of one or more data storage devices;

said metadata manager only accessing said one or more metadata storage volumes in said data storage network, and not directly accessing said one or more user data storage volumes;

implementing a proxy for receiving administrative requests from said metadata manager for metadata manager access to said one or more user data volumes in said data storage network for performing administrative actions specified by a data storage network administrator or administration program, said administrative requests involving said metadata manager requesting manipulation of data stored on said one or more user data volumes on its own behalf in support of said administrative actions;

receiving said administrative requests from said metadata manager; and

accessing said user data volumes in response to said administrative requests received by said proxy.

19. A computer program product for implementing a data storage network distributed file system without direct access to user data from a metadata manager, comprising:

one or more computer readable data storage media;

program instructions recorded on said data storage media for programming data processing platforms in said data storage network to operate by;

supporting access to one or more metadata storage volumes by said metadata manager on a single homogeneous type of one or more data storage devices;

storing all distributed file system metadata on said one or more metadata storage volumes on said single homogeneous type of one or more data storage devices;

supporting access to one or more user data storage volumes by one or more storage manager servers on one or more types of data storage device;

said metadata manager only accessing said one or more metadata storage volumes in said data storage network, and not directly accessing said one or more user data storage volumes;

generating administrative requests from said metadata manager to a proxy in one of said storage manager servers for metadata manager access to said one or more user data volumes in said data storage network for performing administrative actions specified by a data storage network administrator or administration program, said administrative requests involving said metadata manager requesting manipulation of data stored on said one or more user data volumes on its own behalf in support of said administrative actions; and

accessing said one or more user data volumes from said storage manager server in response to said administrative requests to said proxy.