Synchronized data transfer system
First Claim
1. A method of synchronously transferring distributed subchunks of data between a plurality of processor nodes coupled to a network switch, said method comprising:
- periodically broadcasting, by a management process executed on at least one of the processor nodes to each of the plurality of processor nodes via the network switch, a transmit command to initiate each of a plurality of sequential transmit periods;
sending to the management process, by each processor node having at least one subchunk to send, a message for each subchunk to be sent, each message identifying a source processor node and a destination processor node;
selecting, by the management process, at least one message received from the processor nodes to ensure that each processor node scheduled to send at least one subchunk sends only one subchunk during a following transmit period and that each processor node scheduled to receive at least on subchunk receives only one subchunk during the following transmit period;
sending, by the management process, a plurality of transmit requests, each transmit request sent to a processor node that had sent a corresponding message that was selected; and
transmitting, by each processor node receiving a transmit request, a subchunk identified by the received transmit request to a destination processor node in response to the following transmit command.
1 Assignment
0 Petitions
Accused Products
Abstract
A synchronized data transfer system including storage processor nodes, a backbone communication switch, multiple titles subdivided into subchunks and distributed across the nodes, a user process executed on a first node, a transfer process executed on a second node, and a management process executed on a management node. The user process sends a timestamped read request to request a corresponding subchunk. The transfer process sends a message to the management process for each received read request, where each message identifies source and destination nodes. The management process broadcasts transmit commands to initiate sequential transmit periods, receives the messages, and selects from among the received messages to ensure that each node sends and/or receives no more than one subchunk at a time. The management process sends a transmit request for each selected message, and the transfer process sends a subchunk identified by a transmit request in response to the transmit command.
61 Citations
24 Claims
-
1. A method of synchronously transferring distributed subchunks of data between a plurality of processor nodes coupled to a network switch, said method comprising:
-
periodically broadcasting, by a management process executed on at least one of the processor nodes to each of the plurality of processor nodes via the network switch, a transmit command to initiate each of a plurality of sequential transmit periods; sending to the management process, by each processor node having at least one subchunk to send, a message for each subchunk to be sent, each message identifying a source processor node and a destination processor node; selecting, by the management process, at least one message received from the processor nodes to ensure that each processor node scheduled to send at least one subchunk sends only one subchunk during a following transmit period and that each processor node scheduled to receive at least on subchunk receives only one subchunk during the following transmit period; sending, by the management process, a plurality of transmit requests, each transmit request sent to a processor node that had sent a corresponding message that was selected; and transmitting, by each processor node receiving a transmit request, a subchunk identified by the received transmit request to a destination processor node in response to the following transmit command. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A synchronized data transfer system, comprising:
-
a plurality of storage processor nodes including first and second user nodes and a management node; a backbone communication switch coupled to said plurality of storage processor nodes; a plurality of titles each subdivided into a plurality of subchunks which are distributed across said plurality of storage processor nodes; a user process, executed on said first user node, which sends a plurality of timestamped read requests, each timestamped read request for requesting a corresponding subchunk; a transfer process, executed on said second user node, which sends a message to a management process for each received timestamped read request requesting a locally stored subchunk, each message including a source node identifier and a destination node identifier; and a said management process, executed on said management node, which periodically broadcasts a transmit command to said plurality of storage processor nodes via said switch to initiate each of a plurality of sequential transmit periods, which receives a plurality of messages, and, prior to each transmit period, which selects at least one of said plurality of messages to ensure that each storage processor node sends no more than one subchunk and receives no more than one subchunk during each transmit period in which at least one subchunk is transferred, and which sends a plurality of transmit requests corresponding to selected messages; and wherein said transfer process, in response to receiving a transmit request from said management process, sends a corresponding subchunk to a storage processor node identified by said destination node identifier during the next transmit period initiated by the next broadcasted transmit command. - View Dependent Claims (12, 13)
-
-
14. A synchronized data transfer system, comprising:
-
a plurality of processor nodes; a backbone network switch coupled to said plurality of processor nodes to enable communication between said plurality of processor nodes; a plurality of storage devices coupled to and distributed across said plurality of processor nodes and storing a plurality of titles, each title divided into a plurality of subchunks that are distributed across said plurality of storage devices; a plurality of transfer processes, each executed on a corresponding one of said plurality of processor nodes and operative to send a message to a synchronous switch manager process for each subchunk that is to be transferred from a source processor node to a destination processor node, each message including a source node identifier and a destination node identifier; and said synchronous switch manager process, executed on at least one of said plurality of processor nodes, which periodically broadcasts a transmit command to said plurality of processor nodes to initiate each of a plurality of sequential transmit periods, which receives a plurality of messages, and, prior to each transmit period, which selects at least one of said plurality of messages to ensure that each processor node sends no more than one subchunk and receives no more than one subchunk during each transmit period in which at least one subchunk is transferred, and which sends a plurality of transmit requests corresponding to selected messages; and wherein each transfer process which sent at least one message and which received a transmit request from said synchronous switch manager process identifying a corresponding subchunk sends said corresponding subchunk to said destination processor node during the next transmit period initiated by a broadcasted transmit command. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
-
Specification