System and method for distributing data in a parallel processing system
First Claim
1. A parallel-processing system, including memory and at least one processor, capable of dynamically creating a distributed tree for distributing data to the parallel-processing system, comprising:
- a plurality of child master nodes operable to;
establish connections among the plurality of child master nodes, anddistribute a received pre-compiled query to the plurality of child master nodes; and
a plurality of parent master nodes operable to;
receive the pre-compiled query from a client for distribution within the parallel-processing system,identify at least two child master nodes allocated to the parent master node for the dynamically created distribution tree,establish connections with the identified at least two child nodes to form the dynamically created distribution tree,receive from the identified at least two child master nodes polling information,determine, based on the received polling information, a percentage of master nodes in the parallel-processing system that are connected to the dynamically created distribution tree,determine whether the percentage of master nodes in the parallel-processing system connected to the dynamically created distribution tree is above a threshold percentage, andresponsive to a determination that the percentage of master nodes in the parallel-processing system connected to the distribution tree is above the threshold percentage, pre-key the identified at least two child nodes by distributing the pre-compiled query to the identified at least two child master nodes,wherein the connections between the plurality of parent master nodes and the identified at least two child master nodes operate to form the dynamically created distribution tree,the identified at least two child master nodes operate to further distribute the pre-compiled query to additional master nodes through the dynamically created distribution tree, andthe pre-compiled query is distributed in advance of a user providing a request to the parallel-processing system.
3 Assignments
0 Petitions
Accused Products
Abstract
A parallel-processing system that is capable of dynamically creating a distributed tree for distributing data. The system includes a plurality of first nodes. Each of the plurality of first nodes is capable of establishing a connection with at least one of the plurality of first nodes to form at least a portion of a dynamically created distribution tree. The system also includes a second node that is capable of receiving data for distribution within the parallel-processing system. The second node is also capable of establishing a connection with at least two of the plurality of first nodes. In this particular embodiment, the second node and the plurality of first nodes operate to form the dynamically created distribution tree. Moreover, the second node also operates to distribute the data to each of the plurality of first nodes through the dynamically created distribution tree.
-
Citations
18 Claims
-
1. A parallel-processing system, including memory and at least one processor, capable of dynamically creating a distributed tree for distributing data to the parallel-processing system, comprising:
-
a plurality of child master nodes operable to; establish connections among the plurality of child master nodes, and distribute a received pre-compiled query to the plurality of child master nodes; and a plurality of parent master nodes operable to; receive the pre-compiled query from a client for distribution within the parallel-processing system, identify at least two child master nodes allocated to the parent master node for the dynamically created distribution tree, establish connections with the identified at least two child nodes to form the dynamically created distribution tree, receive from the identified at least two child master nodes polling information, determine, based on the received polling information, a percentage of master nodes in the parallel-processing system that are connected to the dynamically created distribution tree, determine whether the percentage of master nodes in the parallel-processing system connected to the dynamically created distribution tree is above a threshold percentage, and responsive to a determination that the percentage of master nodes in the parallel-processing system connected to the distribution tree is above the threshold percentage, pre-key the identified at least two child nodes by distributing the pre-compiled query to the identified at least two child master nodes, wherein the connections between the plurality of parent master nodes and the identified at least two child master nodes operate to form the dynamically created distribution tree, the identified at least two child master nodes operate to further distribute the pre-compiled query to additional master nodes through the dynamically created distribution tree, and the pre-compiled query is distributed in advance of a user providing a request to the parallel-processing system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 16)
-
-
9. A computer-implemented method for distributing data within a parallel-processing system including a plurality of parent master nodes and a plurality of child master nodes, the method performed by at least one of the parent master nodes and comprising:
-
receiving a pre-complied query representation for distribution within the parallel-processing system; dynamically creating a distribution tree for distributing the precompiled query within the parallel-processing system, comprising; identifying at least two of the plurality of child master nodes allocated to the parent master node for the dynamically created distribution tree; and establishing connections with the identified at least two child master nodes to form at least a portion of a dynamically created distribution tree; receiving from the identified at least two child master nodes polling information; determining, based on the received polling information, a percentage of master nodes in the parallel-processing system that are connected to the dynamically created distribution tree; determining whether the percentage of master nodes in the parallel processing system connected to the dynamically created distribution tree is above a threshold percentage; and when it is determined that the percentage of master nodes in the parallel processing system connected to the dynamically created distribution tree is above the threshold percentage, pre-keying the identified at least two child master nodes by distributing to the identified at least two child master nodes the pre-compiled query, wherein the connections between the at least one parent master node and the identified at least two child master nodes operate to form the dynamically created distribution tree, and the at least one parent master node and the identified at least two child master nodes operate to store the pre-compiled query in advance of a user providing a request to the parallel-processing system. - View Dependent Claims (10, 11, 12, 13, 14, 17)
-
-
15. A system for dynamically creating a distribution tree for distributing data within a parallel-processing system including a plurality of parent master nodes and a plurality of child master nodes, the system comprising:
-
one or more computer processors associated with at least one of the plurality of parent master nodes operable to; receive a pre-complied query representation for distribution within the parallel-processing system; create the dynamic distribution tree, including; identifying at least two of the plurality of child master nodes allocated to the at least one parent master node for the dynamically created distribution tree; establishing connections with the identified at least two child master nodes to form at least a portion of a dynamically created distribution tree; receiving from the identified at least two child master nodes polling information; determining, based on the received polling information, a percentage of master nodes in the parallel-processing system that are connected to the dynamically created distribution tree; determining whether the percentage of master nodes in the parallel-processing system connected to the dynamically created distribution tree is above a threshold percentage; and in response to a determination that the percentage of master nodes in the parallel-processing system connected to the dynamically created distribution tree is above the threshold percentage, pre-keying the identified at least two child master nodes by distributing to the identified at least two child nodes the pre-compiled query, wherein the connections between the at least one parent master node and the identified at least two child master nodes operate to form the dynamically created distribution tree, and the at least one parent master node and the identified at least two child master nodes operate to store the pre-compiled query in advance of a user providing a request to the parallel-processing system. - View Dependent Claims (18)
-
Specification