Document store with non-uniform memory access aware high performance query processing
First Claim
1. A method for implementation by a computing system having a non-uniform memory access (NUMA) architecture comprising a plurality of NUMA nodes, the method comprising:
- receiving, from a client, a query of a document store storing a collection of slices each comprising one or more documents, wherein memory associated with each slice is assigned to one of the plurality of NUMA nodes;
determining which of the slices within the document store are required for execution of the query to form a list of slices;
generating, using the query, an execution plan comprising a plurality of nodes including a source node and a plurality of child nodes each specifying at least one database operation to execute a portion of the query, the execution plan comprising a plurality of pipelines of nodes, the execution plan assigning one of the plurality of NUMA nodes to each slice determined to be required for execution of the query, wherein each slice is aware of the NUMA node assigned thereto;
executing the database operations specified by the nodes of the execution plan using the corresponding assigned NUMA nodes for the associated slices, the executing comprising the source node going through the list of slices and forwarding each of the slices including the corresponding one or more documents as a data package to a corresponding child node of the execution plan; and
providing data responsive to the query to the client;
wherein;
each pipeline is assigned to a different one of the plurality of NUMA nodes; and
only one thread executes a node of the execution graph at any given time.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods are described for implementation by a computing system having a non-uniform memory access (NUMA) architecture comprising a plurality of NUMA nodes. A query of a document store storing a collection of slices each comprising one or more documents is received from a client. Thereafter, it is determined which of the slices within the document store are required for execution of the query. An execution plan is then generated, using the query, that comprises a plurality of nodes each specifying at least one database operation to execute a portion of the query. The execution plan assigns one of the plurality of NUMA nodes to each slice determined to be required for execution of the query. The database operations specified by the nodes of the execution plan are then executed using the corresponding assigned NUMA nodes for the associated slice. Data responsive to the query is then provided to the client.
-
Citations
20 Claims
-
1. A method for implementation by a computing system having a non-uniform memory access (NUMA) architecture comprising a plurality of NUMA nodes, the method comprising:
-
receiving, from a client, a query of a document store storing a collection of slices each comprising one or more documents, wherein memory associated with each slice is assigned to one of the plurality of NUMA nodes; determining which of the slices within the document store are required for execution of the query to form a list of slices; generating, using the query, an execution plan comprising a plurality of nodes including a source node and a plurality of child nodes each specifying at least one database operation to execute a portion of the query, the execution plan comprising a plurality of pipelines of nodes, the execution plan assigning one of the plurality of NUMA nodes to each slice determined to be required for execution of the query, wherein each slice is aware of the NUMA node assigned thereto; executing the database operations specified by the nodes of the execution plan using the corresponding assigned NUMA nodes for the associated slices, the executing comprising the source node going through the list of slices and forwarding each of the slices including the corresponding one or more documents as a data package to a corresponding child node of the execution plan; and providing data responsive to the query to the client; wherein; each pipeline is assigned to a different one of the plurality of NUMA nodes; and only one thread executes a node of the execution graph at any given time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A system comprising:
-
a document store storing a collection of slices each comprising one or more documents; and a plurality of non-uniform memory access (NUMA) nodes each comprising at least one data processor and memory for storing instructions for execution by the associated at least one data processor; wherein; memory associated with each slice is assigned to one of the plurality of NUMA nodes; the document store receives a query from a client; it is determined which of the slices within the document store are required for execution of the query to form a list of slices; an execution plan is generated, using the query, the execution plan comprising a plurality of nodes including a source node and a plurality of child nodes each specifying at least one database operation to execute a portion of the query, the execution plan comprising a plurality of pipelines of nodes, the execution plan assigning one of the plurality of NUMA nodes to each slice determined to be required for execution of the query, wherein each slice is aware of the NUMA node assigned thereto; the database operations specified by the nodes of the execution plan are executed using the corresponding assigned NUMA nodes for the associated slices, the executing comprising the source node going through the list of slices and forwarding each of the slices including the corresponding one or more documents as a data package to a corresponding child node of the execution plan; data responsive to the query is provided to the client; each pipeline is assigned to a different one of the plurality of NUMA nodes; and only one thread executes a node of the execution graph at any given time. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. A non-transitory computer program product storing instructions for implementation by a computing system having a non-uniform memory access (NUMA) architecture comprising a plurality of NUMA nodes, the instructions, when executed, result in operations comprising:
-
receiving, from a client, a query of a document store storing a collection of slices each comprising one or more documents, wherein memory associated with each slice is assigned to one of the plurality of NUMA nodes; determining which of the slices within the document store are required for execution of the query to form a list of slices; generating, using the query, an execution plan comprising a plurality of nodes including a source node and a plurality of child nodes each specifying at least one database operation to execute a portion of the query, the execution plan comprising a plurality of pipelines of nodes, the execution plan assigning one of the plurality of NUMA nodes to each slice determined to be required for execution of the query, wherein each slice is aware of the NUMA node assigned thereto; executing the database operations specified by the nodes of the execution plan using the corresponding assigned NUMA nodes for the associated slices, the executing comprising the source node going through the list of slices and forwarding each of the slices including the corresponding one or more documents as a data package to a corresponding child node of the execution plan; providing data responsive to the query to the client; each pipeline is assigned to a different one of the plurality of NUMA nodes; only one thread executes a node of the execution graph at any given time; for a node of the execution graph, internal data structures are initiated lazily when such node is first triggered; a tick is called by a node of the execution graph for each new incoming data package; and a finishing event is invoked by a node of the execution graph after all of its corresponding parent nodes of the execution graph have been executed. - View Dependent Claims (19, 20)
-
Specification