Synchronized data transfer system
First Claim
1. A synchronized data transfer system, comprising:
- a plurality of processor nodes;
a backbone network switch coupled to said plurality of processor nodes to enable communication between said plurality of processor nodes;
a plurality of storage devices distributed across said plurality of processor nodes and storing a plurality of titles, each title divided into a plurality of subchunks that are distributed across said plurality of storage devices;
a plurality of transfer processes, each executed on a corresponding one of said plurality of processor nodes and operative to send a message for each subchunk that is to be transferred from a local storage device to a destination processor node, each message including a source node identifier and a destination node identifier; and
a synchronous switch manager process, executed on at least one of said plurality of processor nodes, which periodically broadcasts a transmit command to initiate each of a plurality of sequential transmit periods, which receives a plurality of messages, and, prior to each transmit period, which selects from among said plurality of messages to ensure that each processing node sends up to one subchunk and receives up to one subchunk during each transmit period, and which sends a plurality of transmit requests corresponding to selected messages; and
wherein each transfer process which sent at least one message and which received a transmit request from said synchronous switch manager process identifying a corresponding subchunk sends said corresponding subchunk during the next transmit period initiated by a broadcasted transmit command.
1 Assignment
0 Petitions
Accused Products
Abstract
A synchronized data transfer system including storage processor nodes, a backbone communication switch, multiple titles subdivided into subchunks and distributed across the nodes, a user process executed on a first node, a transfer process executed on a second node, and a management process executed on a management node. The user process sends a timestamped read request to request a corresponding subchunk. The transfer process sends a message to the management process for each received read request, where each message identifies source and destination nodes. The management process broadcasts transmit commands to initiate sequential transmit periods, receives the messages, and selects from among the received messages to ensure that each node sends or receives only one subchunk during each transmit period. The management process sends a transmit request for each selected message, and the transfer process sends a subchunk identified by a transmit request in response to the transmit command.
272 Citations
24 Claims
-
1. A synchronized data transfer system, comprising:
-
a plurality of processor nodes;
a backbone network switch coupled to said plurality of processor nodes to enable communication between said plurality of processor nodes;
a plurality of storage devices distributed across said plurality of processor nodes and storing a plurality of titles, each title divided into a plurality of subchunks that are distributed across said plurality of storage devices;
a plurality of transfer processes, each executed on a corresponding one of said plurality of processor nodes and operative to send a message for each subchunk that is to be transferred from a local storage device to a destination processor node, each message including a source node identifier and a destination node identifier; and
a synchronous switch manager process, executed on at least one of said plurality of processor nodes, which periodically broadcasts a transmit command to initiate each of a plurality of sequential transmit periods, which receives a plurality of messages, and, prior to each transmit period, which selects from among said plurality of messages to ensure that each processing node sends up to one subchunk and receives up to one subchunk during each transmit period, and which sends a plurality of transmit requests corresponding to selected messages; and
wherein each transfer process which sent at least one message and which received a transmit request from said synchronous switch manager process identifying a corresponding subchunk sends said corresponding subchunk during the next transmit period initiated by a broadcasted transmit command. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A method of synchronously transferring distributed subchunks of data between a plurality of processor nodes coupled to a network switch, said method comprising:
-
periodically broadcasting, by a management process executed on at least one of the processor nodes, a transmit command to initiate each of a plurality of sequential transmit periods;
sending to the management process, by each processor node having at least one subchunk to send, a message for each subchunk to be sent, each message identifying a source processor node and a destination processor node;
selecting, by the management process, messages received from the processor nodes to ensure that each processor node sends up to one subchunk during a following transmit period and that each processor node receives up to one subchunk during the following transmit period;
sending, by the management process, a plurality of transmit requests, each transmit request sent to a processing node that had sent a corresponding message that was selected; and
transmitting, by each processor node receiving a transmit request, a subchunk identified by the received transmit request to a destination processor node in response to the following transmit command. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A synchronized data transfer system, comprising:
-
a plurality of storage processor nodes including first and second user nodes and a management node;
a backbone communication switch coupled to said plurality of storage processor nodes;
a plurality of titles each subdivided into a plurality of subchunks which are distributed across said plurality of storage processor nodes;
a user process, executed on said first user node, which sends a plurality of timestamped read requests, each timestamped read request for requesting a corresponding subchunk;
a transfer process, executed on said second user node, which sends a message for each received timestamped read request requesting a locally stored subchunk, each message including a source node identifier and a destination node identifier; and
a management process, executed on said management node, which periodically broadcasts a transmit command via said switch to initiate each of a plurality of sequential transmit periods, which receives a plurality of messages, and, prior to each transmit period, which selects from among said plurality of messages to ensure that each storage processing node sends up to one subchunk and receives up to one subchunk during each transmit period, and which sends a plurality of transmit requests corresponding to selected messages; and
wherein said transfer process, in response to receiving a transmit request from said management process, sends a corresponding subchunk during the next transmit period initiated by the next broadcasted transmit command. - View Dependent Claims (23, 24)
-
Specification