Scalable transport method for multicast replication

• US 9,338,019 B2
• Filed: 12/03/2013
• Issued: 05/10/2016
• Est. Priority Date: 01/23/2013
• Status: Active Grant

• Current Assignee: Nexenta by DDN, Inc.
• Original Assignee: Nexenta Systems Incorporated (DataDirect Networks Incorporated)
• Inventors: Novak, Robert E., Bestler, Caitlin, Aizman, Alexander
• Primary Examiner(s): Mered, Habte
• Application Number:
  US14/095,839
• Publication Number:
  US 20140204940A1
• Time in Patent Office: 889 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/185 with management of multicast group membership
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 12/1877 Measures taken prior to transmission
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/164 Adaptation or special uses of UDP protocol

Abstract

Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Claims

1. A method of distributing a chunk which encodes data or object metadata within a cluster of storage servers, wherein distributing the chunk within the cluster of storage servers comprises performing a chunk put transaction, the method comprising:

negotiating a rendezvous group by exchanging unreliable datagrams amongst an initiating client and a negotiating group to determine the rendezvous group, wherein the negotiating group comprises a subset of the storage servers,

wherein said negotiating uses a cluster-consensus procedure where each member of the negotiating group evaluates delivery options for the chunk put transaction, wherein the delivery options are evaluated consistently by members of the negotiating group, and

wherein said exchanging comprises multicasting the unreliable datagrams from the initiating client to the negotiating group and multicasting put accept responses from each storage server in the negotiating group to all other storage servers in the negotiating group;

encoding the chunk in a sequence of unreliable datagrams; and

multicasting the chunk by transmitting the sequence of unreliable datagrams in a rendezvous transfer to the rendezvous group, which is a multicast group, such that a single transmission of the sequence of the unreliable datagrams results in reception of the chunk by multiple members of the rendezvous group.

2. The method of claim 1 further comprising:

determining a designated time and a designated transmission rate for the rendezvous transfer.

3. The method of claim 2 further comprising:

offering a reservation by a storage server granting permission to use an ingress reserved bandwidth on the storage server when the storage server has made a determination to respond to a request from the initiating client and has determined that an aggregate offered bandwidth will not exceed a previously-provisioned ingress reserved bandwidth.

4. The method of claim 1, wherein the rendezvous group is a non-empty subset of the negotiating group.

5. The method of claim 1 further comprising:

evaluating the put accept responses by each storage server in the negotiating group to select which storage server in the negotiating group is to send a consensus put accept to the initiating client;

receiving the consensus put accept by the initiating client; and

performing the rendezvous transfer by the initiating client multicasting a payload of the chunk to the rendezvous group.

6. The method of claim 1 further comprising:

evaluating the put accept responses by each storage server in the negotiating group to select which storage server in the negotiating group is to send a consensus put accept to the initiating client, determine members of the rendezvous group, and determine a designated time and a designated transmission rate for the rendezvous transfer;

receiving the consensus put accept by the initiating client; and

performing the rendezvous transfer by the initiating client multicasting a payload of the chunk to the rendezvous group at the designated time at the designated transmission rate.

7. A method of distributing a chunk which encodes data or object metadata within a cluster of storage servers, wherein distributing the chunk within the cluster of storage servers comprises performing a chunk get transaction, the method comprising:

negotiating a rendezvous group by exchanging unreliable datagrams amongst an initiating client and a negotiating group to determine the rendezvous group, wherein the negotiating group comprises a subset of the storage servers, and wherein said exchanging comprises multicasting the unreliable datagrams from the initiating client to the negotiating group,

wherein the rendezvous group includes at least an initiating client, which is a chunk sink that initiated the chunk get transaction,

wherein said negotiating the rendezvous group uses a cluster-consensus procedure where each member of the negotiating group evaluates delivery options for the chunk get transaction, wherein the delivery options are evaluated consistently by members of the negotiating group, and

wherein said exchanging unreliable datagrams comprises multicasting a get request message from the initiating client to the negotiating group, and multicasting get responses from each storage server in the negotiating group to all other storage servers in the negotiating group;

encoding the chunk in a sequence of unreliable datagrams; and

multicasting the chunk by transmitting the sequence of unreliable datagrams in a rendezvous transfer to the rendezvous group which is a multicast group, such that a single transmission of the sequence of the unreliable datagrams results in reception of the chunk by multiple members of the rendezvous group.

8. The method of claim 7, wherein the rendezvous group further includes a storage server that volunteers to store the chunk.

9. The method of claim 7 further comprising:

evaluating the get responses by each storage server in the negotiating group to select a designated storage server in the negotiating group to perform the rendezvous transfer; and

performing the rendezvous transfer by the designated storage server at a designated time at a designated transmission rate.