System and method for efficient access for remote storage devices

• US 9,971,519 B2
• Filed: 06/01/2015
• Issued: 05/15/2018
• Est. Priority Date: 07/30/2014
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: Excelero Storage Ltd.
• Inventors: Romem, Yaniv, Mann, Omri, Oshri, Ofer
• Primary Examiner(s): Sall, El Hadji
• Application Number:
  US14/726,919
• Publication Number:
  US 20160034419A1
• Time in Patent Office: 1,079 Days
• Field of Search: 709203, 709217, 709200
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 15/167 using a common memory, e.g. mailbox
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 2212/65 Details of virtual memory and virtual address translation
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 12/06 Answer-back mechanisms or circuits
• CPC Class Codes: H04L 12/6418 Hybrid transport
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A network interface controller (NIC) for use in a server controlling a plurality of storage devices provides remote access to a plurality of clients each accessing one or more storage devices of the plurality of storage devices. The NIC is configured to receive from at least a client device an input/output (I/O) request for a storage device of the plurality of storage devices and store the I/O request in a secondary memory thereof. The I/O request is sent to the storage device and a response respective of the I/O request intended for the primary processing unit of the server is received by the NIC. The client device is sent an indication that the I/O request is completed. In some exemplary embodiments, the NIC determines if the client device has appropriate I/O permissions to access a specific data block.

Claims

1. A network interface controller (NIC) for use in a server controlling a plurality of storage devices, the NIC comprising:

a network interface configured to communicatively connect the NIC to a network comprising a client device and the server;

a communication interface configured to communicatively connect the NIC to a primary processing unit of the server;

a secondary processing unit;

a first memory containing instructions for execution by the secondary processing unit; and

a second memory;

wherein the instructions, when executed by the secondary processing unit, configure the NIC to;

receive, from the client device, an input/output (I/O) request for a storage device from among the plurality of storage devices;

store the I/O request in the second memory;

send the I/O request to the storage device;

receive, from the storage device, a response for the I/O request intended for the primary processing unit; and

send the client device an indication that the I/O request is completed;

wherein the I/O request contains an address of a data block for the storage device, and the first memory further contains instructions that when executed by the secondary processing unit configure the NIC to;

access a table of permissions to determine if the client device has permission to access the data block of the storage device, the table containing a list of data blocks of the storage device, and for each client device accessing the storage device, per instruction type permissions per each data block.

2. The NIC of claim 1, wherein the second memory comprises a queue configured to store a plurality of received I/O requests.

3. The NIC of claim 1, wherein the first memory further contains instructions that when executed by the secondary processing unit configure the NIC to:

send again an I/O request stored in the second memory, in response to determining that the I/O request was not executed by the storage device.

4. The NIC of claim 1, wherein the first memory further contains instructions that when executed by the secondary processing unit configure the NIC to:

erase the I/O request from the secondary memory in response to determining that the received response indicates a successful completion of the I/O request.

5. The NIC of claim 1, wherein the secondary processing unit, the first memory and the second memory are implemented from one of:

a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), and a system on a chip (SoC).

6. The NIC of claim 1, wherein the first memory and the second memory are implemented as a single logic circuit.

7. The NIC of claim 1, wherein the received response for the I/O request includes an interrupt request.

8. The NIC of claim 1, wherein the indication sent to the client device includes an interrupt request.

9. The NIC of claim 1, wherein the NIC is further configured to send the indication to the client device before receiving an interrupt message for the I/O request from the storage device.

10. The NIC of claim 1, wherein the I/O request comprises a block of data and an address for the storage device.

11. The NIC of claim 1, wherein the network interface and the communication interface are implemented as a single interface.

12. The NIC of claim 2, wherein the NIC provides support for remote direct memory access (RDMA) protocol, and the I/O request is stored in an I/O queue of the second memory using RDMA protocol.

13. The NIC of claim 1, wherein the NIC provides support for remote direct memory access (RDMA) protocol and the response for the I/O request is stored directly in a memory of the client device.

14. The NIC of claim 1, wherein the first memory further contains instructions that when executed by the secondary processing unit configure the NIC to:

send the client device an indication that the received I/O request was not completed, in response to determining that the client device does not have permission to access the data block of the storage device.

15. A computerized method for providing a client device remote access to a server controlling a plurality of storage devices, by a network interface controller (NIC) comprising a secondary processing unit, a first memory and a second memory, the method comprising:

receiving from the client device an input/output (I/O) request for a storage device from among the plurality of storage devices, wherein the I/O request contains an address of a data block for the storage device;

storing the I/O request in the secondary memory of the NIC;

sending the I/O request from the NIC to the storage device;

receiving, from the storage device, a response for the I/O request intended for a primary processing unit of the server;

sending, from the NIC to the client device, an indication that the I/O request is completed; and

accessing a table of permissions to determine if the client device has permission to access the data block of the storage device, the table containing a list of data blocks of the storage device, and for each client device accessing the storage device, per instruction type permissions per each data block.

16. The computerized method of claim 15, storing a plurality of received I/O requests, using a queue of the secondary memory.

17. The computerized method of claim 15, further comprising:

in response to determining that the I/O request was not executed by the storage device, sending an I/O request stored in the second memory again.

18. The computerized method of claim 15, further comprising:

in response to determining that the received response indicates a successful completion of the I/O request, erasing the I/O request from the secondary memory.

19. The computerized method of claim 15, wherein the secondary processing unit, the first memory and the second memory are implemented from one of:

a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), and a system on a chip (SoC).

20. The computerized method of claim 15, wherein the first memory and the second memory are implemented as a single logic circuit.

21. The computerized method of claim 15, wherein the received response for the I/O request includes an interrupt request.

22. The computerized method of claim 15, wherein the indication sent to the client device includes an interrupt request.

23. The computerized method of claim 15, further comprising sending the indication to the client device before receiving the response for the I/O request from the storage device.

24. The computerized method of claim 15, wherein the I/O request comprises a block of data and an address for the storage device.

25. The computerized method of claim 16 wherein the NIC provides support for remote direct memory access (RDMA) protocol, and the I/O requests are stored in a queue of the second memory using RDMA protocol.

26. The computerized method of claim 15, wherein the NIC provides support for remote direct memory access (RDMA) protocol and the I/O response is stored directly in a memory of the client device.

27. The computerized method of claim 15, further comprising:

in response to determining that the client device does not have permission to access the data block of the storage device, sending an indication that the received I/O request was not completed to the client device.

28. The computerized method of claim 15, wherein the instruction type is any of:

read, write, and discard.

29. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute a method for providing a client device remote access to a server controlling a plurality of storage devices, by a network interface controller (NIC) comprising a secondary processing unit, a first memory and a second memory, the method comprising:

receiving from the client device an input/output (I/O) request for a storage device from among the plurality of storage devices, wherein the I/O request contains an address of a data block for the storage device;

storing the I/O request in the secondary memory of the NIC;

sending the I/O request from the NIC to the storage device;

receiving, from the storage device, a response for the I/O request intended for a primary processing unit of the server; and

sending, from the NIC to the client device, an indication that the I/O request is completed; and

accessing a table of permissions to determine if the client device has permission to access the data block of the storage device, the table containing a list of data blocks of the storage device, and for each client device accessing the storage device, per instruction type permissions per each data block.