Method and apparatus for scalable distribution of information in a distributed network

• US 6,748,447 B1
• Filed: 04/07/2000
• Issued: 06/08/2004
• Est. Priority Date: 04/07/2000
• Status: Expired due to Term

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Karge, Leroy R., Revsin, Vitaly S., Basani, Vijay R., Crovella, Mark E., LaRosa, Domenic J., Bestavros, Azer, Murach, Lynne M., Mangiapudi, Krishna
• Primary Examiner(s): Luu, Le Hien
• Application Number:
  US09/545,337
• Time in Patent Office: 1,523 Days
• Field of Search: 709/203, 709/212, 709/213, 709/217, 709/219, 709/231, 709/244, 709/238, 370/429, 370/236
• US Class Current: 709/244
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

The present invention provides a system and apparatus for efficient and reliable, control and distribution of data files or portions of files, applications, or other data objects in large-scale distributed networks. A unique content-management front-end provides efficient controls for triggering distribution of digitized data content to selected groups of a large number of remote computer servers. Network distribution messages are dispatched according to a sorted list of servers, based on factors such as nearness, server processor speed, reliability, and CPU Usage. For large numbers of servers, a store-and-forward approach becomes much more efficient. A first selected server receives the message from a content control manager (CCM). The first server requests instructions for the next server listed on an ordered list in the CCM and forwards a copy of that message to the next identified server. Each server reports its completion and requests further instructions from the CCM. This mechanism permits highly efficient and robust distribution of assignments and data content from the CCM to each required GL using a store-and-forward tree structure.

Claims

1. A system for distributing information to a set of destination nodes connected via a communication network comprising:

a reporting process in each destination node for generating and transmitting a report to a distribution manager, said report containing an identification of said destination node and corresponding destination node parameters, whereby said destination node offers to become a participant in a distribution job;

said distribution manager connected to said network and configured to receive said reports from said destination nodes and to create a prioritized list of destination nodes selected as participants in a distribution job according to said destination node parameters, said prioritized list adapted to represent a transmission tree structure in which each destination node on said prioritized list becomes an information source for any later destination node occurring later in said list, said distribution manager having a management process for sending information to each participant;

said management process adapted to send each participant instructions to obtain a copy of said information either from said distribution manager or from another identified participant, and said management process determines when each participant has received a copy of said information;

each participant having a store-and-forward process configured to receive instructions from a prior participant or from said distribution manager and to request a copy of said information from said prior participant or from said distribution manager, and to thereafter request further distribution instructions from said distribution manager until instructed that no other participants require said information;

whereby each participant obtains a copy of the information and the distribution manager obtains confirmation that each destination node has obtained said information.

2. The system of claim 1 in which the destination node parameters, by which said prioritized list is created, are selected from the set of parameters consisting of location, destination node performance, distance from other destination nodes, transmission speed of said destination node'"'"'s network connection, and reliability of said network connection.

3. A method of distributing information to a set of destination nodes connected via a communication network comprising the steps of:

obtaining a list of destination nodes desiring to participate in a distribution;

prioritizing said list according to parameters associated with each destination node on said list;

issuing instructions to each destination node according to the prioritized list order, comprising the steps of;

A) obtaining an address of a target node on said prioritized list;

(B) sending a notification message to said target node, said notification containing the address of a source node having an information file to distribute;

(C) receiving said notification message at said first destination node and requesting a copy of said information from said identified source node;

(D) reporting when said target node has successfully received said copy of said information;

(E) responding to said report by selecting a next target node address on said prioritized list and repeating steps (A) through (E) until said prioritized list is exhausted;

whereby said copy of said information is sent to each target node on said list in an order determined according to the order of the list for each successful copy; and

notifying each destination node when the prioritized list is exhausted.

4. The method of claim 3 in which said step of prioritizing further comprises the steps of:

obtaining a set of parameters corresponding to said list of destination nodes; and

sorting said list according to at least some of said set of parameters;

whereby the sorted list provides a prioritized ordering of destination nodes to receive said information being distributed.

5. The method of claim 3 in which said steps of sending and receiving are carried out using a reliable transport protocol across said communication network.

6. A method of distributing information to a set of destination nodes connected via a communication network comprising the steps of:

electing a group leader among a set of destination nodes on a network segment;

sending reports of the respective address and capabilities of each destination node to said group leader;

collecting and processing said reports in said group leader;

generating a network segment discovery report in said group leader;

sending said network segment discovery report to a distribution manager; and

arranging a list of destination nodes according to at least one of said network segment discovery reports;

prioritizing said list according to parameters associated with each destination node on said list;

issuing instructions to each destination node according to the prioritized list order, said instructions including the identification of a source for obtaining said information and an identification of the next destination node on the prioritized list;

distributing said information according to said instructions; and

notifying each destination node when the prioritized list is exhausted.

7. The method of claim 6 in which said distribution server further comprises a database for storing said reports and a graphical user interface for providing an interface for system administration personnel.

8. The method of claim 6 in which said step of arranging said list further comprises the steps of:

building a database of reports collected from said group leaders;

collecting distribution policies;

administratively dividing said destination nodes into content groups;

collecting information about the network communication delays;

creating a list of group leaders corresponding to each content group; and

sorting each said list of group leaders according to at least one of said reported destination node capabilities, distribution policies, and information about the network communication delays.

9. A system for distributing information to a set of destination nodes connected via a communication network comprising:

a discovery process for discovering destination nodes in a network, said discover process including at least one network cluster comprised of destination node connected to each other with a multicast communication medium, an election mechanism for dynamically selecting one destination node of each said at least one network cluster to be group leader, and a reporting process in which each said group leader solicits reports from each destination node in the corresponding cluster, creates a profile report for the cluster, and transmits it to said distribution server;

a distribution server for receiving the results of said discovery process, comprising

at least one network group comprised of a set of destination nodes;

a list of destination nodes comprising at least one user-defined content group;

a user interface for obtaining distribution policies; and

a distribution process for managing the transmission of said information to each of said network groups.

10. The system of claim 9 in which said discovery process further comprises:

a group leader for collecting reports from at least one destination node in a network segment and for processing said reports into a network segment report and for sending said report to said distribution server.

11. The system of claim 9 in which said network group is created through the process of receiving reports from a set of group leaders, in which at least some reports identify destination nodes under the control of the corresponding group leader.

12. The system of claim 9 in which said distribution process further comprises a mechanism for:

arranging a list of network groups according to at least one parameter selected from the set of;

physical location, transmission delay for reaching said network group from said distribution server, and a measure of destination node capabilities of at least one destination node within each network group; and

transmitting said information to each network group according to said arranged list.

13. The system of claim 12 in which said system further comprises:

a report generator for generating a report from any destination node that has changed its availability status or system capabilities, or has successfully received an information transmission from said distribution process; and

a report transmitter, in a group leader for each cluster of destination nodes, for collecting reports from any destination node in the corresponding cluster, and for processing said reports and transmitting a consolidated report to the distribution server.

14. The system of claim 9 in which said reporting process is triggered by an event selected from the set of:

a timer, a group leader receiving a registration request from a destination node in said network cluster, a change of group leader, a user input, an information transfer from said distribution server, and an instruction from said distribution server.

18. The system according to claim 9 in which each network group further comprises:

a group leader for receiving information from an information source and for instructing other group leaders to request information from an information source;

a local distribution mechanism in which a group leader transmits said information to each destination node in its corresponding network group using a multicast protocol; and

an interpreter in each said destination node for executing a set of commands in said information, and for reporting completion status of said execution.

19. The system according to claim 9 in which said distribution process is responsive to said distribution policies and identifies which network groups correspond to any destination made in said content group.

15. A system for distributing information to a set of destination nodes connected via a communication network comprising:

a discovery process for discovering destination nodes in a network;

a distribution server for receiving the results of said discovery process, comprising

at least one network group comprised of a set of destination nodes;

a list of destination nodes comprising at least one user-defined content group;

a user interface for obtaining distribution policies; and

a distribution process for managing the transmission of said information to each of said network groups, said distribution process comprising;

a list of destinations, each corresponding to a network group, said list adapted to represent a transmission tree structure in which each destination on said list becomes an information source for any later destination occurring later in said list;

a point-to-point transmission process for sending information from one of said information sources to any of said later destinations;

a distribution manager containing an information source, and for obtaining said list of destinations and requesting a point-to-point transmission of said information to each destination according to said list, and for determining when transmission has been completed for each destination.

16. The system of claim 15 in which said distribution manager further comprises:

a process for creating an information notification message for transmission, said notification including an earlier destination address from said list and an information identifier, and for triggering transmission to said next destination;

whereby said next destination receives a notification that information corresponding to said information identifier is to be obtained from one of said information sources.

17. The system of claim 15 in which said list is ordered according to parameters selected from the set of:

the CPU power of the destination, the network location of each destination, and user inputs.

20. A method for determining the completion status of processing by individual nodes of a group of destination nodes on a multicast data channel, said group having a group leader, comprising the steps of:

notifying an assignment processor in at least one of said destination nodes that an assignment has been received for processing;

determining completion status of said processing by at least one of said destination nodes;

notifying said group leader of said completion status by a message from said at least one destination node such that the group leader acquires a collective indication of status from nodes within said group;

determining whether the number of non-reporting nodes in the group exceeds a configured number or proportion of nodes;

and transmitting a multicast request from said group leader (GL) to nodes in said group where said request is formatted according to said number of non-reporting nodes, and otherwise transmitting a request to each non-reporting node; and

repeating said determination of the number of non-reporting nodes and transmitting a multicast or unicast request until either each node has reported its status or a preconfigured number of requests has been transmitted.

21. The method of claim 20 further comprising the steps of:

when the number of non-reporting nodes is less than the number of reporting nodes, listing said non-reporting nodes in said multicast request;

when the number of non-reporting nodes is not less than the number of reporting nodes, listing said reporting nodes in said multicast request;

interpreting said multicast request in each node to determine whether any given node in the group should transmit a missing report to said GL; and

transmitting said missing report according to said determination, after a time delay configurable for each node.