Processor agnostic data storage in a PCIE based shared storage enviroment

• US 8,966,172 B2
• Filed: 01/23/2012
• Issued: 02/24/2015
• Est. Priority Date: 11/15/2011
• Status: Active Grant

• Current Assignee: Pavilion Data Systems, Inc.
• Original Assignee: Pavilion Data Systems, Inc.
• Inventors: Malwankar, Kiron Balkrishna
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): DOAN, HAN V
• Application Number:
  US13/355,823
• Publication Number:
  US 20130191590A1
• Time in Patent Office: 1,128 Days
• US Class Current: 711/114
• CPC Class Codes: G06F 12/10 Address translation
• CPC Class Codes: G06F 13/1668 Details of memory controller
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 2212/65 Details of virtual memory and virtual address translation
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Disclosed are a system, a method and/or an apparatus of processor agnostic data storage in a PCIE based shared storage environment. In one aspect, a method includes processing a storage based request received at an adapter circuit of a controller device associated with a disk array to direct the storage based request to at least one of a processor of the disk array and a plurality of storage devices of the disk array. The method also includes routing, through an interface circuit of the controller device, the data request in the other format compatible with the storage device directly to at least one storage device of the plurality of storage devices of the disk array coupled to the controller device agnostic to a processor of the disk array to store a data associated with the data request based on a mapping table.

Claims

1. A method comprising:

receiving, by a first controller device of a plurality of controller devices associated with a disk array that comprises a plurality of storage devices and a processor, a request from a host machine that is at a remote location from the disk array, wherein the plurality of storage devices comprise Peripheral Component Interconnect Express (PCIE) based solid state storage devices;

determining, by the first controller device, a nature of the request to be at least one of a data request or a control request based on metadata of the request;

responsive to determining that the request is a data request, performing the following comprising;

converting, by the first controller device, a format of the data request from a first format to a second format that is compatible with the plurality of storage devices, wherein the first format is a format other than a non-volatile memory express (NVMe) format, and wherein converting the data request to the second format comprises converting the data request to the NVMe format; and

routing the data request in the second format directly to at least one storage device of the plurality of storage devices, bypassing the processor of the disk array, based on a mapping table, residing in a memory of the first controller device, that represents an association of the at least one storage device to the first controller device, the association comprising first locations on the at least one storage device that are accessible to the first controller device, wherein at least one of the mapping table or an additional mapping table further represents an additional association of the at least one storage device to a second controller device of the plurality of controller devices, the additional association comprising second locations on the at least one storage device that are accessible to the second controller device.

2. The method of claim 1, further comprising:

associating the at least one storage device to the first controller device;

recording an association of the at least one storage device to the first controller device in the mapping table; and

storing the mapping table representing the association of the at least one storage device to the first controller device in the memory of the first controller device, wherein the memory of the first controller device is mapped in a memory of the disk array.

3. The method of claim 2, further comprising:

dynamically updating the association of the at least one storage device to the first controller device based on a status of the at least one storage device;

directing, by the first controller device, a control request from the host machine to the processor; and

processing the control request by the processor.

4. The method of claim 2, wherein associating the at least one storage device to the first controller device comprises:

associating a pair of queues comprising a submission queue of the storage device that represents a submitted storage request and a completion queue of the storage device that represents a completed storage request with the first controller device, wherein the pair of queues is to reside in the memory of the first controller device.

5. The method of claim 1, further comprising:

adding the data request from the host machine to a submission queue of the at least one storage device, wherein the submission queue resides in the memory of the first controller device;

sending an alert of the data request in the submission queue to the at least one storage device, wherein the at least one storage device is to process the data request entered in the submission queue responsive to receiving the alert.

6. The method of claim 5, further comprising:

issuing an interrupt from the at least one storage device to the first controller device responsive to the at least one storage device processing the data request entered in the submission queue;

writing a completion status associated with the data request in a completion queue that is associated with the submission queue responsive to the at least one storage device of the plurality of storage devices processing the data request; and

updating the submission queue and the completion queue.

7. The method of claim 1, wherein converting the format of the data request from the first format to the second format comprises translating, by the first controller device, a virtual address associated with the at least one storage device that is included in the storage based request to a physical address of the at least one storage device based on an address translation table stored in the memory of the first controller device, the method further comprising:

communicatively coupling the first controller device to the at least one storage device through an interface circuit of the first controller device via a Peripheral Component Interconnect Express (PCIE) switch.

8. A controller device, comprising:

an adapter circuit to receive a request from a host machine;

a logic circuit, coupled to the adapter circuit, to;

determine a nature of the request to be at least one of a data request or a control request based on metadata of the request; and

convert a format of the request from a first format to a second format compatible with a plurality of storage devices of a disk array responsive to a determination that the request is a data request, wherein the plurality of storage devices comprise Peripheral Component Interconnect Express (PCIE) based solid state storage devices, wherein the first format is a format other than a non-volatile memory express (NVMe) format, and wherein converting the data request to the second format comprises converting the data request to the NVMe format; and

an interface circuit, coupled to the logic circuit, to route the data request in the second format to at least one storage device of the plurality of storage devices of the disk array, bypassing a processor of the disk array, based on a mapping table, residing in a memory of the controller device that is mapped in a memory of the disk array, that represents an association of the at least one storage device to the controller device, the association comprising first locations on the at least one storage device that are accessible to the controller device, wherein at least one of the mapping table or an additional mapping table further represents an additional association of the at least one storage device to an additional controller device, the additional association comprising second locations on the at least one storage device that are accessible to the additional controller device.

9. A disk array comprising:

a plurality of storage devices, wherein the plurality of storage devices comprise Peripheral Component Interconnect Express (PCIE) based solid state storage devices;

a processor; and

a first controller device coupled to the processor and to the plurality of storage devices, the first controller device comprising;

an adapter circuit to receive a request from a host machine;

a logic circuit to distinguish a nature of the request to be at least one of a data request or a control request based on metadata of the request, and to convert a format of the request from a first format to a second format that is compatible with the plurality of storage devices responsive to a determination that the request is a data request, wherein the first format is a format other than a non-volatile memory express (NVMe) format, and wherein converting the data request to the second format comprises converting the data request to the NVMe format; and

an interface circuit to route the data request in the second format to at least one storage device of the plurality of storage devices, bypassing the processor, based on a mapping table, residing in a memory of the first controller device, that represents an association of the at least one storage device to the first controller device, the association comprising first locations on the at least one storage device that are accessible to the first controller device; and

a second controller device coupled to the processor and to the plurality of storage devices, the second controller device comprising an additional interface circuit to route an additional data request in the second format to the at least one storage device of the plurality of storage devices, bypassing the processor, based on at least one of the mapping table or an additional mapping table, residing in an additional memory of the second controller device, that represents an additional association of the at least one storage device to the second controller device, the additional association comprising second locations on the at least one storage device that are accessible to the second controller device.

10. The disk array of claim 9, wherein the processor is to:

associate the at least one storage device to the first controller device;

record an association of the at least one storage device to the first controller device in the mapping table; and

store the mapping table representing the association of the at least one storage device to the first controller device in the memory of the first controller device, wherein the memory of the first controller device is mapped in the memory of the disk array.

11. The disk array of claim 9, wherein:

the first controller device is further to direct the storage based request to the processor responsive to a determination that the storage based request is a control request; and

the processor is further to process the control request.

12. The disk array of claim 9, wherein associating the at least one storage device to the first controller device comprises:

associating a pair of queues comprising a submission queue of the storage device that represents a submitted storage request and a completion queue of the storage device that represents a completed storage request with the first controller device, wherein the pair of queues is to reside in the memory of the first controller device that is mapped in the memory of the disk array.

13. The disk array of claim 9, wherein:

the first controller device is further to add the data request from the host machine to a submission queue of the at least one storage device, wherein the submission queue resides in the memory of the first controller device, and alert the at least one storage device of the data request entry in the submission queue; and

the at least one storage device of the plurality of storage devices is to process the data request entered in the submission queue responsive to receiving the alert from the first controller device.

14. The disk array of claim 13, wherein one of the processor or the at least one storage device is to:

issue an interrupt to the first controller device in response to the at least one storage device processing the data request entered in the submission queue;

write a completion status associated with the data request in a completion queue that is associated with the submission queue in response to the at least one storage device of the plurality of storage devices processing the data request; and

update the submission queue and the completion queue.

15. The disk array of claim 9, wherein converting the format of the data request from the first format to the second format comprises translating a virtual address associated with the at least one storage device that is included in the storage based request to a physical address of the at least one storage device based on an address translation table stored in the memory of the first controller device.

16. The disk array of claim 9, wherein the interface circuit comprises a Peripheral Component Interconnect Express (PCIE) switch that communicatively couples the first controller device to the at least one storage device.