SCALABLE TRANSPORT SYSTEM FOR MULTICAST REPLICATION

• US 20140204941A1
• Filed: 12/03/2013
• Published: 07/24/2014
• Est. Priority Date: 01/23/2013
• Status: Active Grant

• Current Assignee: Nexenta by DDN, Inc.
• Original Assignee: Nexenta Systems Incorporated (DataDirect Networks Incorporated)
• Inventors: BESTLER, Caitlin, NOVAK, Robert E., AIZMAN, Alexander
• Granted Patent: US 9,344,287 B2
• Time in Patent Office: Days
• US Class Current: 370/390
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/185 with management of multicast group membership
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 12/1877 Measures taken prior to transmission
• CPC Class Codes: H04L 47/15 in relation to multipoint traffic arrangements for broadcast or multicast in data networks H04L12/18
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/164 Adaptation or special uses of UDP protocol

Abstract

Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Claims

1. A distributed data storage system that allows for available storage capacity and workload of an individual storage server to impact distribution of chunks within a cluster of storage servers, the distributed data storage system comprising:

a network of non-blocking switches communicatively interconnecting the storage servers of the cluster;

an initiating client comprising a client system that communicatively interconnects to the network; and

a transport protocol implemented by the storage servers of the cluster and the initiating client for distributing a chunk within the cluster of storage servers, wherein the chunk is encoded in a sequence of unreliable datagrams, and the chunk is multicast by transmitting the sequence of unreliable datagrams in a rendezvous transfer to a rendezvous group, which is a multicast group, such that a single transmission of the sequence of the unreliable datagrams results in reception of the chunk by multiple members of the rendezvous group.

2. The distributed data storage system of claim 1, wherein the rendezvous group is negotiated by exchanging unreliable datagrams amongst the initiating client and a negotiating group to determine the rendezvous group, and the negotiating group comprises a subset of the storage servers, and wherein said exchanging comprises multicasting the unreliable datagrams from the initiating client to the negotiating group.

3. The distributed data storage system of claim 2, wherein the transport protocol determines a designated time and a designated transmission rate for the rendezvous transfer.

4. The distributed data storage system of claim 3, wherein a reservation offered by a storage server grants permission to use an ingress reserved bandwidth on the storage server when the storage server has made a determination to respond to a request from the initiating client and has determined that an aggregate offered bandwidth will not exceed a previously-provisioned ingress reserved bandwidth.

5. The distributed data storage system of claim 2, wherein distributing the chunk within the cluster of storage servers comprises performing a chunk put transaction.

6. The distributed data storage system of claim 5, wherein the rendezvous group is a non-empty subset of the negotiating group.

7. The distributed data storage system of claim 5, wherein a client-consensus procedure is used in which an initiating client, which is a chunk source, evaluates delivery options for the chunk put transaction.

8. The distributed data storage system of claim 7, wherein a put proposal message is multicast from the initiating client to the negotiating group, and put accept responses are unicast from the storage servers in the negotiating group to the initiating client.

9. The distributed data storage system of claim 8, wherein the put accept responses are evaluated by the initiating client to determine whether the rendezvous transfer is to be performed based on a number of existing replicas of the chunk indicated by the put accept responses.

10. The distributed data storage system of claim 8, wherein the put accept responses are evaluated by the initiating client to determine members of the rendezvous group and the designated time and the designated transmission rate for the rendezvous transfer, and a payload of the chunk is multicast by the initiating client to perform the rendezvous transfer to the rendezvous group.

11. The distributed data storage system of claim 5, wherein each member of the negotiating group evaluates delivery options for the chunk put transaction, wherein the delivery options are evaluated consistently by members of the negotiating group.

12. The distributed data storage system of claim 11, wherein a put proposal message is multicast from the initiating client, which is a chunk source, to the negotiating group, and put accept responses are multicast from each storage server in the negotiating group to all other storage servers in the negotiating group.

13. The distributed data storage system of claim 12, wherein the put accept responses is evaluated by each storage server in the negotiating group to select which storage server in the negotiating group is to send a consensus put accept response to the initiating client, wherein the consensus put accept is received by the initiating client, and wherein the rendezvous transfer is performed by the initiating client multicasting a payload of the chunk to the rendezvous group.

14. The distributed data storage system of claim 12, wherein the put accept responses are evaluated by each storage server in the negotiating group to select which storage server in the negotiating group is to send a consensus put accept response to the initiating client, determine members of the rendezvous group, and determine a designated time and a designated transmission rate for the rendezvous transfer, wherein the consensus put accept is received by the initiating client, and wherein the rendezvous transfer is performed by the initiating client multicasting a payload of the chunk to the rendezvous group at the designated time at the designated transmission rate.

15. The distributed data storage system of claim 2, wherein distributing the chunk within the cluster of storage servers comprises performing a chunk get transaction.

16. The distributed data storage system of claim 15, wherein the rendezvous group includes at least an initiating client, which is a chunk sink that initiated the chunk get transaction.

17. The distributed data storage system of claim 16, wherein the rendezvous group further includes a storage server that volunteers to store the chunk.

18. The distributed data storage system of claim 16, wherein a client-consensus procedure is used in which the initiating client evaluates delivery options for the chunk get transaction.

19. The distributed data storage system of claim 18, wherein a get request message is multicast from the initiating client to the negotiating group, and get responses are unicast from the storage servers in the negotiating group to the initiating client.

20. The distributed data storage system of claim 19, wherein the get responses are evaluated by the initiating client to determine which get response is to be accepted.

21. The distributed data storage system of claim 20, wherein a get accept message is multicast from the initiating client to the negotiating group, and the get accept message indicates the storage server that is to perform the rendezvous transfer, and wherein the rendezvous transfer is performed by the designated storage server at the designated time at the designated transmission rate.

22. The distributed data storage system of claim 16, wherein a cluster-consensus procedure is used in which members of the negotiating group evaluate which of the members is to respond to the initiating client.

23. The distributed data storage system of claim 22, wherein a get request message is multicast from the initiating client to the negotiating group, and get responses are multicast from each storage server in the negotiating group to all other storage servers in the negotiating group.

24. The distributed data storage system of claim 23, wherein the get responses are evaluated by each storage server in the negotiating group to select a designated storage server in the negotiating group to perform the rendezvous transfer, and the rendezvous transfer is performed by the designated storage server at a designated time at a designated transmission rate.