NVM express controller for remote access of memory and I/O over ethernet-type networks

• US 10,063,638 B2
• Filed: 11/30/2015
• Issued: 08/28/2018
• Est. Priority Date: 06/26/2013
• Status: Active Grant

• Current Assignee: Point Financial, Inc.
• Original Assignee: Cnex Labs, Inc.
• Inventors: Huang, Yiren Ronnie
• Primary Examiner(s): Christensen, Scott B
• Assistant Examiner(s): Do, Lam T
• Application Number:
  US14/954,691
• Publication Number:
  US 20160085718A1
• Time in Patent Office: 1,002 Days
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: H04L 65/40 Support for services or applications
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method and system for enabling Non-Volatile Memory express (NVMe) for accessing remote solid state drives (SSDs) (or other types of remote non-volatile memory) over the Ethernet or other networks. An extended NVMe controller is provided for enabling CPU to access remote non-volatile memory using NVMe protocol. The extended NVMe controller is implemented on one server for communication with other servers or non-volatile memory via Ethernet switch. The NVMe protocol is used over the Ethernet or similar networks by modifying it to provide a special NVM-over-Ethernet frame.

Claims

1. An extended NVMe storage network comprising:

at least one NVMoE initiator node comprising;

a host processor;

an extended NVMe controller coupled to the host processor via a host interface, the extended NVMe controller functioning as an NVMoE initiator that receives from the host processor NVMe commands directed to access remote namespaces coupled to an Ethernet network, the NVMoE initiator translating the received NVMe commands to a format suitable for transmission over the Ethernet network to the remote namespaces, the converted NVMe commands passed to a direct network interface to be transmitted over the Ethernet network, wherein the extended NVMe controller cannot access local non-volatile memories using NVMe commands; and

the direct network interface adapted to couple the NVMoE initiator to the Ethernet network; and

a plurality of NVMoE target nodes comprising;

local non-volatile memories corresponding to the remote namespaces;

extended NVMe controllers coupled to the local non-volatile memories, the extended NVMe controllers functioning as NVMoE targets; and

direct network interfaces adapted to couple the Ethernet network to the NVMoE targets, the NVMoE targets receiving the translated NVMe commands via the Ethernet network according to which remote namespaces the translated NVMe commands are directed, the NVMoE targets executing the received NVMe commands to access the local non-volatile memories corresponding to the remote namespaces.

2. The extended NVMe storage network of claim 1, wherein the NVMoE initiator comprises a command translator that translates the received NVMe commands to NVM-over-Ethernet (NVMoE) commands suitable for transmission over Ethernet.

3. The extended NVMe storage network of claim 2, wherein the command translator translates a host identifier (HSID) and/or a name space identifier (NSID) of the NVMe commands to Ethernet media access control (MAC) addresses of the NVMoE commands.

4. The extended NVMe storage network of claim 1, wherein the NVMoE targets cannot transmit NVMe commands directed to access remote namespaces.

5. The extended NVMe storage network of claim 1, wherein all of the direct network interfaces include an Ethernet MAC interface.

6. The extended NVMe storage network of claim 1, wherein the NVMe commands include read and write commands.

7. The extended NVMe storage network of claim 1, wherein the remote non-volatile memories include solid state drives.

8. The extended NVMe storage network of claim 1, wherein the external network includes an InfiniBand network.

9. The extended NVMe storage network of claim 1, wherein the external network includes a fibre channel network.

10. A unit for use in a rack-mounted system, the unit comprising:

an Ethernet switch;

a NVM blade, comprising;

local non-volatile memories corresponding to remote namespaces; and

an extended NVMe controller coupled to the local non-volatile memories the extended NVMe controller functioning as an NVMoE target to connect the local non-volatile memories to the Ethernet switch via a direct network interface of the NVMe target, such that the NVMoE target does not transmit NVMe commands directed to access remote namespaces, the NVMoE target receiving translated NVMe commands via the direct network interface according to which remote namespaces the translated NVMe commands are directed, and the NVMoE targets converting the translated NVMe commands to native NVMe commands and executing the native NVMe commands to access the local non-volatile memories corresponding to the remote namespaces; and

wherein the Ethernet switch is coupled to a direct network interface of a NVMoE initiator node, the NVMoE initiator node comprising;

a host processor;

an extended NVMe controller coupled to the host processor via a host interface, the extended NVMe controller functioning as an NVMoE initiator that receives from the host processor NVMe commands directed to access remote namespaces coupled to the Ethernet switch, the NVMoE initiator translating the received NVMe commands to a format suitable for transmission via the Ethernet switch to the remote namespaces, wherein the converted NVMe commands are passed to the direct network interface of the NVMoE initiator to be transmitted via the Ethernet switch, and wherein the extended NVMe controller cannot access local non-volatile memories using NVMe commands.

11. The unit of claim 10, wherein the unit is a shelf unit mountable in a rack.

12. The rack-mountable shelf unit of claim 11, wherein the plurality of NVM blades are mounted in slots in the shelf unit.

13. The rack-mountable shelf unit of claim 11, wherein the plurality of NVM blades are SSD blades.

14. The rack-mountable shelf unit of claim 11, wherein the shelf unit is a 2 U-high or a 3 U-high shelf unit.

15. The rack-mountable shelf unit of claim 11, further comprising:

a blade server that provides NVMoE administrative functions.

16. The rack-mountable shelf unit of claim 11, further comprising:

a bridge between an NVMoE protocol and another network protocol.

17. The rack-mountable shelf unit of claim 16, wherein the other network protocol is one of RDMA, NVMf, NVMoF, iSCSI and NFS.

18. A rack-mounted system comprising:

a plurality of the rack-mountable shelf units of claim 11; and

an Ethernet fabric connecting the plurality of rack-mountable shelf units, wherein the Ethernet fabric is either top-of-rack or end-of-row.