PRE-REGISTERING MEMORY REGIONS FOR REMOTE DIRECT MEMORY ACCESS IN A DISTRIBUTED FILE SYSTEM

• US 20170199841A1
• Filed: 01/13/2016
• Published: 07/13/2017
• Est. Priority Date: 01/13/2016
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Chundattu Parambil, Mohammed Rafi Kavunga, Talur, Raghavendra
• Granted Patent: US 10,713,211 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 49/9005 using dynamic buffer space allocation
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The subject matter of this specification can be implemented in, among other things, a method that includes pre-registering, by a processing device at a client device, multiple input/output (IO) buffers at the client device with a remote direct memory access (RDMA) interface at the client device. The client device accesses multiple server devices of a distributed file system using the RDMA interface. The method further includes receiving a request to access a file in the distributed file system from an application at the client device. The method further includes designating a first IO buffer among the IO buffers as a cache for data from the file. The method further includes receiving the data for the file in the first IO buffer from the distributed file system using the RDMA interface.

Claims

1. A method comprising:

pre-registering, by a processing device at a client device, multiple memory regions at the client device for input/output (IO) buffers of a remote direct memory access (RDMA) interface at the client device, wherein the client device accesses multiple server devices of a distributed file system using the RDMA interface;

receiving a request to access a file in the distributed file system from an application at the client device;

designating a first IO buffer among the IO buffers as a cache for data from the file; and

receiving the data for the file in the first IO buffer from the distributed file system using the RDMA interface.

2. The method of claim 1, wherein the IO buffers comprise a first set of IO buffers used to cache data received from the distributed file system and a second set of IO buffers that are not used to cache data received from the distributed file system, and wherein the first set of IO buffers comprises the first IO buffer.

3. The method of claim 1, wherein the IO buffers are created and pre-registered during initialization of the RDMA interface prior to providing the access to the distributed file system by the application.

4. The method of claim 1, wherein the request to access the file comprises a system call or a file operation of an operating system.

5. The method of claim 4, wherein the system call or the file operation accesses a virtual file system.

6. The method of claim 5, wherein the operating system directs the system call or the file operation for the virtual file system to a kernel module in kernel space that provides access to the distributed file system from user space.

7. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processing device, cause the processing device to:

pre-register, by the processing device at a client device, multiple memory regions at the client device for input/output (IO) buffers of a remote direct memory access (RDMA) interface at the client device, wherein the client device accesses multiple server devices of a distributed file system using the RDMA interface;

receive a request to access a file in the distributed file system from an application at the client device;

receive data for the file in a first IO buffer among the IO buffers from the distributed file system using the RDMA interface; and

copying the data from the first IO buffer to a cache at the client device.

8. The computer-readable medium of claim 7, wherein the processing device is to copy the data in the background using a thread that is separate from a thread that receives the data from the distributed file system.

9. The computer-readable medium of claim 7, wherein the IO buffers are created and pre-registered during initialization of the RDMA interface prior to providing the access to the distributed file system by the application.

10. The computer-readable medium of claim 7, wherein the request to access the file comprises a system call or a file operation of an operating system.

11. The computer-readable medium of claim 10, wherein the system call or the file operation accesses a virtual file system.

12. The computer-readable medium of claim 11, wherein the operating system directs the system call or the file operation for the virtual file system to a kernel module in kernel space that provides access to the distributed file system from user space.

13. A system comprising:

a remote direct memory access (RDMA) interface at a client device;

a memory, at the client device, comprising multiple memory regions; and

a processing device, at the client device, to communicate with the RDMA interface and the memory to;

pre-register the memory regions for input/output (IO) buffers of the RDMA interface, wherein the client device accesses multiple server devices of a distributed file system using the RDMA interface;

receive a request to access a file in the distributed file system from an application at the client device;

designate a first IO buffer among the IO buffers as a cache for data from the file; and

receive the data for the file in the first IO buffer from the distributed file system using the RDMA interface.

14. The system of claim 13, wherein the IO buffers comprise a first set of IO buffers used to cache data received from the distributed file system and a second set of IO buffers that are not used to cache data received from the distributed file system, and wherein the first set of IO buffers comprises the first IO buffer.

15. The system of claim 13, wherein the IO buffers are created and pre-registered during initialization of the RDMA interface prior to providing the access to the distributed file system by the application.

16. The system of claim 13, wherein the request to access the file comprises a system call or a file operation of an operating system.

17. The system of claim 16, wherein the system call or the file operation accesses a virtual file system.

18. The system of claim 17, wherein the operating system directs the system call or the file operation for the virtual file system to a kernel module in kernel space that provides access to the distributed file system from user space.