Non-volatile memory express over ethernet

• US 9,565,269 B2
• Filed: 11/04/2014
• Issued: 02/07/2017
• Est. Priority Date: 11/04/2014
• Status: Active Grant

• Current Assignee: Pavilion Data Systems, Inc.
• Original Assignee: Pavilion Data Systems, Inc.
• Inventors: Malwankar, Kiron Balkrishna, Govindasamy, Raghuraman, Melnic, Dan M.
• Primary Examiner(s): Pollack, Melvin H
• Application Number:
  US14/532,790
• Publication Number:
  US 20160127492A1
• Time in Patent Office: 826 Days
• Field of Search: 709/212
• US Class Current: 1/1
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A processing device receives a message encapsulating an input/output (I/O) command from a remote computing device. The processing device identifies one or more physical storage devices to be accessed to satisfy the I/O command. The processing device then sends, to each physical storage device of the one or more physical storage devices, one or more non-volatile memory express (NVMe) commands directed to that physical storage device.

Claims

1. A method comprising:

receiving, by a processing device of a storage server comprising an array of physical storage devices, a message encapsulating an input/output (I/O) command from a remote computing device, wherein the I/O command is directed to a virtual storage device that is associated with a plurality of physical storage devices of the array of physical storage devices;

identifying, by the processing device, the plurality of physical storage devices to be accessed to satisfy the I/O command;

generating, by the processing device, for each physical storage device of the plurality of physical storage devices, one or more non-volatile memory express (NVMe) commands directed to that physical storage device based on the I/O command; and

sending, by the processing device, to each physical storage device of the plurality of physical storage devices, the one or more non-volatile memory express (NVMe) commands directed to that physical storage device.

2. The method of claim 1, wherein the message comprises an Ethernet packet encapsulating the I/O command, and wherein the I/O command is included in a payload portion of the Ethernet packet.

3. The method of claim 2, wherein the I/O command comprises a write command, and wherein the one or more NVMe commands directed to the plurality of physical storage devices comprise one or more NVMe write commands, the method further comprising:

receiving one or more additional Ethernet packets encapsulating data to be stored for the write command; and

removing the data from the one or more additional Ethernet packets, wherein the one or more NVMe write commands cause the data to be stored in the plurality of physical storage devices.

4. The method of claim 1, wherein the I/O command is directed to a plurality of logical block addresses of the virtual storage device, and wherein identifying the plurality of physical storage devices comprises identifying addresses of the plurality of physical storage devices that the plurality of logical block addresses map to, wherein the plurality of physical storage devices comprise a plurality of physical NVMe drives and the virtual storage device comprises a virtual NVMe drive.

5. The method of claim 4, further comprising:

validating the I/O command based on determining whether the remote computing device has access to the plurality of logical block addresses; and

sending the one or more NVMe commands to the plurality of physical storage devices responsive to validating the I/O command.

6. The method of claim 4, further comprising:

responsive to creation of a new virtual NVMe drive, notifying a plurality of remote computing devices of the new virtual NVMe drive;

receiving a discovery request from at least one remote computing device of the plurality of remote computing devices; and

connecting the at least one remote computing device to the new virtual NVMe drive.

7. The method of claim 1, wherein the I/O command comprises a read command, and wherein the one or more NVMe commands directed to the plurality of physical storage devices comprise one or more NVMe read commands, the method further comprising:

receiving data from the plurality of physical storage devices responsive to the one or more NVMe read commands;

generating one or more Ethernet packets encapsulating the received data; and

sending the one or more Ethernet packets encapsulating the received data to the remote computing device to satisfy the I/O command.

8. The method of claim 1, further comprising:

receiving an additional message from the remote computing device, the additional message encapsulating a task management command;

determining one or more previously received I/O commands identified in the task management command; and

canceling the one or more previously received I/O commands.

9. A system comprising:

an array of physical storage devices; and

a storage controller coupled to the array of physical storage devices via at least one of a switch or a bus, the storage controller comprising a processing device that is to;

receive a message encapsulating an input/output (I/O) command from a remote computing device, wherein the I/O command is directed to a virtual storage device that is associated with a plurality of physical storage devices of the array of physical storage devices;

identify the plurality of physical storage devices to be accessed to satisfy the I/O command;

generate, for each physical storage device of the plurality of physical storage devices, one or more non-volatile memory express (NVMe) commands directed to that physical storage device based on the I/O command; and

send, to each physical storage device of the plurality of physical storage devices, the one or more non-volatile memory express (NVMe) commands directed to that physical storage device.

10. The system of claim 9, wherein the storage controller further comprises a network interface controller (NIC) to connect the system to the remote computing device via a network, wherein the message comprises an Ethernet packet encapsulating the I/O command, and wherein the I/O command is included in a payload portion of the Ethernet packet.

11. The system of claim 10, wherein the I/O command comprises a write command, and wherein the one or more NVMe commands directed to the plurality of physical storage devices comprise one or more NVMe write commands, wherein the processing device is further to:

receive one or more additional Ethernet packets encapsulating data to be stored for the write command; and

remove the data from the one or more additional Ethernet packets, wherein the one or more NVMe write commands cause the data to be stored in the plurality of physical storage devices.

12. The system of claim 9, wherein the I/O command is directed to a plurality of logical block addresses of the virtual storage device, and wherein identifying the plurality of physical storage devices comprises identifying addresses of the plurality of physical storage devices that the plurality of logical block addresses map to, wherein the plurality of physical storage devices comprise a plurality of physical NVMe drives and the virtual storage device comprises a virtual NVMe drive.

13. The system of claim 9, wherein the processing device is further to:

responsive to creation of a new virtual storage, notify a plurality of remote computing devices of the new virtual storage;

receive a discovery request from at least one remote computing device of the plurality of remote computing devices; and

connect the at least one remote computing device to the new virtual storage.

14. The system of claim 9, wherein the processing device is further to:

generate, for each physical storage device of the plurality of physical storage devices, the one or more NVMe commands directed to that physical storage device based on the I/O command.

15. The system of claim 9, wherein the I/O command comprises a read command, and wherein the one or more NVMe commands directed to the plurality of physical storage devices comprise one or more NVMe read commands, the processing device further to:

receive data from the plurality of physical storage devices responsive to the one or more NVMe read commands;

generate one or more Ethernet packets encapsulating the received data; and

send the one or more Ethernet packets encapsulating the received data to the remote computing device to satisfy the I/O command.

16. The system of claim 9, wherein the plurality of storage devices comprise a plurality of NVMe solid state storage devices coupled to the storage controller via the switch, the system further comprising:

one or more additional storage controllers that are also coupled to the plurality of solid state storage devices via the switch.

17. A non-transitory computer readable storage medium having instructions that, when executed by a processing device of a storage server comprising an array of physical storage devices, cause the processing device to perform operations comprising:

receiving, by the processing device of the storage server comprising the array of physical storage devices, an Ethernet packet encapsulating an input/output (I/O) command from a remote computing device, wherein the I/O command is directed to a virtual non-volatile memory express (NVMe) drive that is associated with a plurality of physical storage devices of the array of physical storage devices;

identifying, by the processing device, a plurality of physical NVMe drives to be accessed to satisfy the I/O command;

generating, by the processing device, for each physical NVMe drive of the plurality of physical NVMe drives, one or more non-volatile memory express (NVMe) commands directed to that physical NVMe drive based on the I/O command; and

sending, by the processing device, to each physical NVMe drive of the plurality of physical NVMe drives, the one or more NVMe commands directed to that physical NVMe drive.

18. The non-transitory computer readable storage medium of claim 17, wherein the I/O command is directed to a plurality of logical block addresses of the virtual NVMe drive, and wherein identifying the plurality of physical NVMe drives comprises identifying addresses of the plurality of physical NVMe drives that the plurality of logical block addresses map to, the operations further comprising:

generating, for each physical NVMe drive of the plurality of physical NVMe drives, the one or more NVMe commands directed to that physical NVMe drive based on the I/O command.