Systems and methods for implementing an enterprise-class converged compute-network-storage appliance

• US 9,521,198 B1
• Filed: 12/19/2013
• Issued: 12/13/2016
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Springpath, Incoporated (Cisco Systems, Inc.)
• Inventors: Zabarsky, Jeffrey A., Agarwala, Sandip, Chaturvedi, Abhishek, Gaonkar, Shravan, Mahalingam, Mallikarjunan, Reddy, Sazzala, Shah, Smit, Shaikh, Faraz, Vegulla, Praveen, Yadappanavar, Krishna
• Primary Examiner(s): Scott, Randy
• Application Number:
  US14/135,485
• Time in Patent Office: 1,090 Days
• Field of Search: 709/212, 709/213, 709/214, 709/219, 711/114, 711/162, 711/165
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 12/0292 using tables or multilevel address translation means G06F12/023 takes precedence; address translation in virtual memory systems G06F12/10
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11C 7/1072 : for memories with random access ports synchronised on clock signal pulse trains, e.g. synchronous memories, self timed memories
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed storage system that dispatches an input/output request is described. In an exemplary embodiment, a storage controller client receives the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers. The storage controller client further computes a target virtual node for the input/output request, where the target virtual node is one of the plurality of virtual nodes. Using the computed target virtual node, the storage controller client determines a target physical node that corresponds to the target virtual node, where the target physical node is one of the plurality of physical nodes. In addition, the storage controller client dispatches the input/output request to a target storage controller that corresponds to the target physical node, wherein the target storage controller server is one of the plurality of storage controller servers. In addition, the virtual node includes a set of one or more mirrored copies across different fault domains for ensuring resiliency and high availability.

Claims

1. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to dispatch an input/output request in a distributed storage system, the method comprising:

receiving, at a storage controller client, the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers;

computing a target virtual node for the input/output request, wherein the target virtual node is one of the plurality of virtual nodes and the target virtual node is computed using a deterministic placement function without storing a global map;

determining a target physical node that corresponds to the target virtual node, wherein the target physical node is one of the plurality of physical nodes; and

dispatching the input/output request to a target storage controller server that corresponds to the target physical node, wherein the target storage controller server is one of the plurality of storage controller servers.

2. The non-transitory machine-readable medium of claim 1, wherein the input/output request is selected from the group consisting of a write operation request, a read operation request, and a query operation request.

3. The non-transitory machine-readable medium of claim 1, wherein the storage controller client and the target storage controller are on different devices that are coupled together by a network.

4. The non-transitory machine-readable medium of claim 1, wherein the determining a target physical node comprises:

performing a lookup with the target virtual node for the target physical node using a storage routing table.

5. The non-transitory machine-readable medium of claim 1, wherein the dispatching comprises:

scheduling a dispatch of the input/output request to the target using a policy based on a characteristic of the input/output request; and

transmitting the input/output request to the target storage controller server based on the scheduling.

6. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to write data in a distributed storage system, the method comprising:

receiving, at one of a plurality of storage controller servers, a write operation request to store data at a virtual node, wherein the write operation request is dispatched from a storage controller client to the one of the plurality of storage controller servers using a deterministic placement function without storing a global map, and the distributed storage system includes the storage controller client, and the plurality of storage controller servers; and

in response to receiving the write operation request, writing the data to a mirror set of the virtual node.

7. The non-transitory machine-readable medium of claim 6, wherein a mirror set includes one or more mirrors that is used to replicate data and metadata of the virtual node.

8. The non-transitory machine-readable medium of claim 7, wherein the writing the data comprises:

determining a primary mirror of the mirror set; and

storing the data from a segment store of the primary mirror.

9. The non-transitory machine-readable medium of claim 8, wherein if the writing of the data is not successful,

marking the primary mirror as inactive;

and

electing a new primary mirror.

10. The non-transitory machine-readable medium of claim 8, wherein the segment store is thin provisioned logical partition of distributed storage system.

11. The non-transitory machine-readable medium of claim 8, wherein the writing of the data further comprises:

storing the data in an active mirror of the mirror set.

12. The non-transitory machine-readable medium of claim 11, wherein if the storing of the data in the active mirror is not successful,

marking the active mirror as inactive.

13. The non-transitory machine-readable medium of claim 6, wherein the data includes an object to be stored and a key to reference the object.

14. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to process a read operation request in a distributed storage system, the method comprising:

receiving, at one of a plurality of storage controller servers, the read operation request for a virtual node, wherein the write operation request is dispatched from a storage controller client to the storage controller server using a deterministic placement function without storing a global map, and the distributed storage system includes the storage controller client and the plurality of storage controller servers; and

in response to receiving the read operation request,

determining a mirror set for the read operation request, and

reading data indicated in the read operation request from the mirror set.

15. The non-transitory machine-readable medium of claim 14, wherein a mirror set includes one or more mirrors that is used to replicate data and metadata of the virtual node.

16. The non-transitory machine-readable medium of claim 15, wherein the reading the data comprises:

determining a primary mirror of the mirror set; and

fetching the data from a segment store of the primary mirror.

17. The non-transitory machine-readable medium of claim 16, wherein if the fetching of the data is not successful, fetching the data from an active mirror of the mirror set.

18. The non-transitory machine-readable medium of claim 16, wherein if the fetching of the data is not successful, repairing the primary mirror.

19. The non-transitory machine-readable medium of claim 16, wherein if the fetching of the data is not successful,

marking the primary mirror as inactive;

and

electing a new primary mirror from an active mirror of the mirror set.

20. A method to dispatch an input/output request in a distributed storage system, the method comprising:

receiving, at a storage controller client, the input/output request, wherein the distributed storage system includes the storage controller client, a plurality of storage controller servers, a plurality of virtual nodes distributed among a plurality of physical nodes, and each of the plurality of physical nodes is hosted on one of the plurality of storage controller servers;

computing a target virtual node for the input/output request, wherein the target virtual node is one of the plurality of virtual nodes and the target virtual node is computed using a deterministic placement function without storing a global map;

determining a target physical node that corresponds to the target virtual node, wherein the target physical node is one of the plurality of physical nodes; and

dispatching the input/output request to a target storage controller server that corresponds to the target physical node, wherein the target storage controller server is one of the plurality of storage controller servers.

21. A method to write data in a distributed storage system, the method comprising:

receiving, at one of a plurality of storage controller servers, a write operation request to store data at a virtual node, wherein the write operation request is dispatched from a storage controller client to the one of the plurality of storage controller servers using a deterministic placement function without storing a global map, and the distributed storage system includes the storage controller client, and the plurality of storage controller servers; and

in response to receiving the write operation request, writing the data to a mirror set of the virtual node.

22. A method to process a read operation request in a distributed storage system, the method comprising:

receiving, at one of a plurality of storage controller servers, the read operation request for a virtual node, wherein the write operation request is dispatched from a storage controller client to the storage controller server using a deterministic placement function without storing a global map, and the distributed storage system includes the storage controller client and the plurality of storage controller servers; and

in response to receiving the read operation request,

determining a mirror set for the read operation request, and

reading data indicated in the read operation request from the mirror set.