Distributed file serving architecture system with metadata storage virtualization and data access at the data server connection speed

• US 7,610,348 B2
• Filed: 05/07/2003
• Issued: 10/27/2009
• Est. Priority Date: 05/07/2003
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines SA (International Business Machines Corporation)
• Inventors: Kisley, Richard Victor, Knight, Philip Douglas
• Primary Examiner(s): Vaughn, Jr.; William C
• Assistant Examiner(s): CHRISTENSEN, SCOTT B
• Application Number:
  US10/431,413
• Publication Number:
  US 20040225719A1
• Time in Patent Office: 2,365 Days
• Field of Search: 709/201, 709/203, 709/212, 709/217, 709/219, 709/227, 709/228, 709/229, 709/230, 709/238, 709/245, 709/250, 707/100, 707/200, 707/205, 711/100
• US Class Current: 709/212
• CPC Class Codes: G06F 13/385 for adaptation of a particular data processing system to different peripheral devices
• CPC Class Codes: G06F 13/4282 on a serial bus, e.g. I2C bus, SPI bus on daisy chain buses G06F13/4247
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Method, apparatus and program storage device that provides a distributed file serving architecture with metadata storage virtualization and data access at the data server connection speed is provided. A host issues a file access request including data target locations. The file access request including data target locations is processed. Remote direct memory access (RDMA) channel endpoint connection are issued in response to the processing of the file access request. An RDMA transfer of the file-block data associated with the file access request is made directly between a memory at the host and a data server.

Claims

1. A method for providing a distributed file serving architecture with metadata storage virtualization and direct data access, comprising:

storing metadata associated with data on a data server in memory at a metadata server;

establishing a file-based remote direct memory access protocol session with a host and a data server;

receiving a data request at the metadata server from the host through an operations channel established between the host and the metadata server for providing asynchronous communication initiated only by the host;

controlling virtualization of the data server file system at the metadata server, wherein the controlling is performed by;

providing data structures containing information about block files represented by logical files on the data server that forms the directory structure; and

providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server;

satisfying the data request under the control of the metadata server, said satisfying including;

providing from the metadata server to the data server, connection and host target buffer information through a back control channel between the metadata server and the data server using the remote direct memory access protocol;

processing the received connection and host target buffer information;

using, by said data server, the processed connection and host target buffer information to directly access memory allocated in the host for the data request to satisfy the data request; and

transferring the data requested by said data request from the data server to the host through said file-based remote direct memory access protocol session.

2. The method of claim 1 further comprising providing remote direct memory access I/O operation details to the data server from the metadata server.

3. The method of claim 1 wherein transferring of data associated with the data request directly between the host memory and the data server is initiated by the data server.

4. The method of claim 1 farther comprising implementing an FRP Client and Adaptation Layer, and an RDMA Access Driver/DS Address Virtualization at the host.

5. The method of claim 1 farther comprising implementing an FRP server, an RDMA Exchange Protocol Engine and an RDMA Access Driver/DS Address Virtualization at the metadata server.

6. The method of claim 1 farther comprising implementing an RDMA Exchange Protocol Engine, an RDMA Initiator Engine and an RDMA Access Driver/DS Address Virtualization at the data server.

7. The method of claim 1 further providing direct memory-to-memory transfer and direct application access for queuing data transfer operations directly to the host without involvement of the host operating system.

8. The method of claim 1 farther comprising implementing an RDMA Access Driver for enabling direct access of buffers of the host.

9. The method of claim 1 farther comprising providing data structures that contain information about block files represented by logical files on the data server and that forms the directory structure and providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server.

10. A program storage readable by one or more computers, the program storage having media tangibly embodying one or more programs of instructions executable by a computer to perform a method for satisfying a request in a distributed file serving architecture with metadata storage virtualization and direct data access, comprising:

storing metadata associated with data on a data server in memory at a metadata server;

establishing a file-based remote direct memory access protocol session with a host and a data server;

receiving a data request at the metadata server from the host through an operations channel established between the host and the metadata server for providing asynchronous communication initiated only by the host;

controlling virtualization of the data server file system at the metadata server, wherein the controlling is performed by;

providing data structures containing information about block files represented by logical files on the data server that forms the directory structure; and

providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server;

satisfying the data request under the control of the metadata server, said satisfying including;

providing from the metadata server to the data server, connection and host target buffer information through a back control channel between the metadata server and the data server using the remote direct memory access protocol;

processing the received connection and host target buffer information;

using, by said data server, the processed connection and host target buffer information to directly access memory allocated in the host for the data request to satisfy the data request; and

transferring the data requested by said data request from the data server to the host through said file-based remote direct memory access protocol session.

11. The program storage of claim 10 wherein the method further comprises providing remote direct memory access I/O operation details to the data server from the metadata server.

12. The program storage of claim 10 wherein the transferring of data associated with the data request directly between the host memory and the data server is initiated by the data server.

13. The program storage of claim 10 wherein the method further comprises implementing an FRP Client and Adaptation Layer, and an RDMA Access Driver/DS Address Virtualization at the host.

14. The program storage of claim 10 wherein the method further comprises implementing an FRP server, an RDMA Exchange Protocol Engine and an RDMA Access Driver/DS Address Virtualization at the metadata server.

15. The program storage of claim 10 wherein the method further comprises implementing an RDMA Exchange Protocol Engine, an RDMA Initiator Engine and an RDMA Access Driver/DS Address Virtualization at the data server.

16. The program storage of claim 10 wherein the method further comprises providing direct memory-to-memory transfer and direct application access for queuing data transfer operations directly to the host without involvement of the host operating system.

17. The program storage of claim 10 wherein the method further comprises implementing an RDMA Access Driver for enabling direct access of buffers of the host.

18. The program storage of claim 10 wherein the method further comprises providing data structures that contain information about block files represented by logical files on the data server and that forms the directory structure and providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server.

19. A system, comprising:

a data server that stores data;

a host comprising;

a first memory,

and instructions, stored on said first memory, for establishing a file-based remote direct memory access protocol session between the host and the data server;

a metadata server comprising a second memory storing metadata associated with data on the data server;

wherein said metadata server controls virtualization of the data server file system, wherein the metadata server controls virtualization by;

providing data structures containing information about block files represented by logical files on the data server that forms the directory structure; and

providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server;

wherein said host comprises instructions, stored on said first memory, for establishing an operations channel between the host and the metadata server for providing asynchronous communication initiated only by the host and to provide a data request at the metadata server from the host through the operations channel;

wherein said metadata server controls the satisfying of the data request, said satisfying including;

establishing a back control channel between the metadata server and the data server using the remote direct memory access protocol;

providing from the metadata server to the data server, connection and host target buffer information through the back control channel between the metadata server and the data server using the remote direct memory access protocol;

processing the received connection and host target buffer information; and

using, by said data server, the processed connection and host target buffer information to directly access memory allocated in the host for the data request to satisfy the data request.

20. The system of claim 19 further wherein the metadata server is further adapted to provide remote direct memory access I/O operation details to the data server from the metadata server.

21. The system of claim 19 wherein transfer of data associated with the data request directly between the host memory and the data server is initiated by the data server.

22. The system of claim 19 wherein the host is further adapted to implement an FRP Client and Adaptation Layer, and an RDMA Access Driver/DS Address Virtualization at the host.

23. The system of claim 19 wherein the metadata server is further adapted to implement an FRP server, an RDMA Exchange Protocol Engine and an RDMA Access Driver/DS Address Virtualization at the metadata server.

24. The system of claim 19 wherein the data server is further adapted to implement an RDMA Exchange Protocol Engine, an RDMA Initiator Engine and an RDMA Access Driver/DS Address Virtualization at the data server.

25. The system of claim 19 wherein the metadata server is further adapted to satisfy requests for direct memory-to-memory transfer and direct application access for queuing data transfer operations directly to the host without involvement of the host operating system.

26. The system of claim 19 wherein the host and the data server are each further adapted to implement an RDMA Access Driver for enabling direct access of buffers of the host.

27. The system of claim 19 wherein the metadata server is further adapted to provide data structures that contain information about block files represented by logical files on the data server and that forms the directory structure and providing block data virtualization metadata comprising a mapping of the file system provided by the data server to a metafile system of the metadata server.