System and method for reliable peer communication in a clustered storage system

• US 7,716,323 B2
• Filed: 07/18/2003
• Issued: 05/11/2010
• Est. Priority Date: 07/18/2003
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: NetApp, Inc.
• Inventors: Gole, Abhijeet, Sarma, Joydeep sen
• Primary Examiner(s): Nguyen; Thu
• Assistant Examiner(s): Hussain; Tauqir
• Application Number:
  US10/622,558
• Publication Number:
  US 20050015460A1
• Time in Patent Office: 2,489 Days
• Field of Search: 709/224, 370/214, 370/400, 370/464, 370/471, 710/38, 711/162
• US Class Current: 709/224
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/10015 Access to distributed or replicated servers, e.g. using brokers
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A system and method for reliable peer-to-peer communication is provided. The system and method includes a cluster connection manager that acts as a unified resource for opening, closing and maintaining peer-to-peer communication channels between applications executing on storage systems comprising a storage system cluster. The cluster connection manager communicates with cluster connection manager clients to maintain and manage the communication channels. The cluster connection manager may also be operatively interconnected with a plurality of cluster interconnect devices, thereby enabling failover operation in the event that a cluster interconnect device suffers an error condition.

Claims

1. A method for initiating a peer-to-peer communication session, comprising:

creating, using a cluster connection manager executing on a first server, an initial connection with a cluster partner on a second server;

exchanging a set of peer connection information between the first and second servers;

passing a set of cluster connection manager client information to the cluster partner, wherein the set of cluster connection manager client information includes at least one virtual interface and any memory requirements for a cluster manager client;

creating a set of communication ports using the set of cluster connection manager client information, wherein the at least one virtual interface connection allows remote direct memory access (RDMA) operations that allow the cluster connection manager executing on the first server to directly access memory regions of the cluster partner operating on the second server before a storage operating system executing on the cluster partner is fully active, the RDMA read operation bypassing the operating system;

alerting the cluster partner of a ready status; and

alerting a set of cluster connection manager clients that the cluster partner is in a ready state.

2. The method of claim 1 wherein the set of clients comprises a failover monitor process.

3. The method of claim 1 wherein the set of peer connection information comprises a version number.

4. The method of claim 1 wherein the step of passing a set of client information to the cluster partner further comprises:

collecting, from a set of clients, the set of client information; and

transferring the collected set of client information to the cluster.

5. The method of claim 4 wherein the client information comprises a number of communication ports required.

6. The method of claim 4 wherein the set of client information further comprises an amount of memory requested by a particular client.

7. The method of claim 1 wherein the step of creating an initial connection further comprises using remote direct memory access primitives to create the initial connection.

8. The method of claim 1 wherein the step of creating an initial connection further comprises performing a series of remote direct memory access operations to create the initial connection.

9. A system configured to manage reliable peer communication among storage systems in a clustered environment, the system comprising:

one or more peer processes executing on each storage system partner; and

a cluster connection manager executing on each storage system partner, the cluster connection manager creating a set of peer-to-peer connections between the one or more peer processes executing on each storage system, wherein the cluster connection manager is provided to reliably create virtual interface connections between peer processes executing on the storage system partners over a cluster interconnect without requiring a storage operating system executing on each storage system to be fully active or functioning, wherein the virtual interface connection allows remote direct memory access (RDMA) operations that allow the cluster connection manager operating on the first server to directly access memory regions of the cluster partner operating on the second server before the storage operating system executing on the cluster partner is fully active, the RDMA read operation bypassing the operating system.

10. A computer readable medium storing executable program instructions executed by a processor, comprising:

program instructions that create, using a cluster connection manager executing on a first server, an initial connection with a cluster partner on a second server;

program instructions that exchange a set of peer connection information between the first server and the second server;

program instructions that pass a set of cluster connection manager client information to the cluster partner, wherein the set of cluster connection manager client information includes at least one virtual interface and any memory requirements for a cluster manager client;

program instructions that create a set of communication ports using the set of cluster connection manager client information, wherein the at least one virtual interface connection allows remote direct memory access (RDMA) operations that allow the cluster connection manager executing on the first server to directly access memory regions of the cluster partner operating on the second server before a storage operating system executing on the cluster partner is fully active, the RDMA read operation bypassing the operating system;

program instructions that alert the cluster partner of a ready status; and

program instructions that alert a set of cluster connection manager clients that the cluster partner is in a ready state.