Scalable data using RDMA and MMIO
First Claim
1. A computerized method for retrieving data from a memory mapped input/output (MMIO) enabled storage device of a control server communicatively coupled with a client device through a remote direct memory access (RDMA) enabled network interface controller (rNIC), the method comprising:
- receiving from the client device, through a first communication protocol, a first entry for a submission queue, the first entry using a second communication protocol that permits communication between the control server and the MMIO enabled storage device;
receiving from the client device, through the first communication protocol, a second entry for a send queue, the second entry using the first communication protocol to send a data block from a memory reserved for the client device to the client device;
receiving a trigger for the first entry, the trigger being a doorbell; and
receiving, using the second communication protocol, the data block from the MMIO enabled storage device in response to executing the first entry; and
transmitting, using the first communication protocol, the data block to the client device in response to the data block being received from the MMIO enabled storage device.
3 Assignments
0 Petitions
Accused Products
Abstract
To improve upon some of the characteristics of current storage systems in general and block data storage systems in particular, exemplary embodiments combine state-of-the art networking techniques with state-of-the-art data storage elements in a novel way. To accomplish this combination in a highly effective way, it is proposed to combine networking remote direct memory access (RDMA) technique and storage-oriented memory mapped input output (MMIO) technique in a system to provide direct access from a remote storage client to a remote storage system with little to no central processing unit (CPU) intervention of the remote storage server. In some embodiments, this technique may reduce the required CPU intervention on the client side. These reductions of CPU intervention potentially reduce latency while providing performance improvements, and/or providing more data transfer bandwidth and/or throughput and/or more operations per second compared to other systems with equivalent hardware.
19 Citations
25 Claims
-
1. A computerized method for retrieving data from a memory mapped input/output (MMIO) enabled storage device of a control server communicatively coupled with a client device through a remote direct memory access (RDMA) enabled network interface controller (rNIC), the method comprising:
-
receiving from the client device, through a first communication protocol, a first entry for a submission queue, the first entry using a second communication protocol that permits communication between the control server and the MMIO enabled storage device; receiving from the client device, through the first communication protocol, a second entry for a send queue, the second entry using the first communication protocol to send a data block from a memory reserved for the client device to the client device; receiving a trigger for the first entry, the trigger being a doorbell; and receiving, using the second communication protocol, the data block from the MMIO enabled storage device in response to executing the first entry; and transmitting, using the first communication protocol, the data block to the client device in response to the data block being received from the MMIO enabled storage device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computerized method for storing data in a memory mapped input/output (MMIO) enabled storage device of a control server communicatively coupled with a client device through a remote direct memory access (RDMA) enabled network interface controller (rNIC), the method comprising:
-
receiving through a first communication protocol from the client device a first data block for storing on the storage device; receiving through the first communication protocol from the client device a first entry for a submission queue, the first entry for communicating using a second communication protocol; receiving through the first communication protocol from the client device a second entry for a send queue to send a second data block from a memory reserved for the client device to the client device over the first communication protocol; and receiving a trigger in response to receiving the second entry, and sending the second data block to the client device using the first communication protocol, the trigger being a doorbell. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
23. A non-transitory computer readable medium having stored thereon instructions that when executed by one or more processing elements perform a method of retrieving data from a memory mapped input/output (MMIO) enabled storage device of a control server communicatively coupled with a client device through a remote direct memory access (RDMA) enabled network interface controller (rNIC), the method comprising:
-
receiving from the client device, through a first communication protocol, a first entry for a submission queue, the first entry using a second communication protocol that permits communication between the control server and the MMIO enabled storage device; receiving from the client device, through the first communication protocol, a second entry for a send queue, the second entry using the first communication protocol to send a data block from a memory reserved for the client device to the client device; receiving a trigger for the first entry, the trigger being a doorbell; and receiving, using the second communication protocol, the data block from the MMIO enabled storage device in response to executing the first entry; and transmitting, using the first communication protocol, the data block to the client device in response to the data block being received from the MMIO enabled storage device.
-
-
24. A non-transitory computer readable medium having stored thereon instructions that when executed by one or more processing elements perform a method for storing data in a memory mapped input/output (MMIO) enabled storage device of a control server communicatively coupled with a client device through a remote direct memory access (RDMA) enabled network interface controller (rNIC), the method comprising:
-
receiving through a first communication protocol from the client device a first data block for storing on the storage device; receiving through the first communication protocol from the client device a first entry for a submission queue, the first entry for communicating using a second communication protocol; receiving through the first communication protocol from the client device a second entry for a send queue to send a second data block from a memory reserved for the client device to the client device over the first communication protocol; and receiving a trigger in response to receiving the second entry, and sending the second data block to the client device using the first communication protocol, the trigger being a doorbell.
-
-
25. A server including:
-
a remote direct memory access (RDMA) enabled network interface controller (rNIC) configured to communicate with a client device using a first communication protocol and to communicate with a storage device using a second communication protocol, the storage device including a memory reserved for the client device; a memory configured to store a first data block that is for storing on the storage device, in response to the rNIC receiving the data block from the client device; a submission queue configured to store a first entry in response to receiving the first entry from the client device, the first entry facilitating communication between the rNIC and the storage device using the second communication protocol; a send queue configured to store a second entry in response to receiving the second entry from the client device, the second entry facilitating sending a second data block from the memory reserved for the client device to the client device over the first communication protocol; and a doorbell configured to send the second data block to the client device using the first communication protocol in response to being triggered.
-
Specification