Tuning memory across database clusters for distributed query stability

• US 10,620,837 B2
• Filed: 04/27/2017
• Issued: 04/14/2020
• Est. Priority Date: 03/30/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Finlay, Ian R., Garcia-Arellano, Christian M., Storm, Adam J., Yakub, Gennady
• Primary Examiner(s): Park, Jeong S
• Application Number:
  US15/498,610
• Publication Number:
  US 20170285965A1
• Time in Patent Office: 1,083 Days
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

A method for optimizing query compilation by tuning memory across a database cluster is provided. The method may include receiving, by a global memory tuner, memory configuration information from a plurality of nodes within the database cluster. The method may also include determining a node within the plurality of nodes having a least available memory value relative to a remainder of nodes within the plurality of nodes based on the received memory configuration information. The method may then include calculating a globally tuned memory value based on a memory value associated with the determined node. The method may further include determining a memory distribution based on the calculated globally tuned memory value. The method may also include sending the determined memory distribution to the plurality of nodes.

Claims

1. A method for tuning memory across a database cluster, the method comprising:

receiving, by a global memory tuner, memory configuration information from a plurality of nodes within the database cluster, wherein the received memory configuration includes an available physical memory value for each node within the database cluster;

determining a node within the plurality of nodes having a least available memory value relative to a remainder of nodes within the plurality of nodes based on the received memory configuration information;

generating a globally tuned memory value to match the available physical memory value associated with the determined node;

determining a memory distribution based on the generated globally tuned memory value;

sending the determined memory distribution to the plurality of nodes, wherein each node within the plurality of nodes designates a memory portion equal to the calculated globally tuned memory value as a globally tuned memory partition; and

generating at least one query plan based on the sent memory distribution, wherein the generated at least one query plan is restricted to using a memory amount specified by the globally tuned memory partition within the sent memory distribution for each node within the plurality of nodes.

2. The method of claim 1, wherein calculating the globally tuned memory value based on the memory value associated with the determined node comprises setting the globally tuned memory value to equal the memory value associated with the determined node.

3. The method of claim 1, wherein calculating the globally tuned memory value based on the memory value associated with the determined node comprises setting the globally tuned memory value to be less than the memory value associated with the determined node.

4. The method of claim 1, wherein sending the determined memory distribution to the plurality of nodes further includes each node within the plurality of nodes applying the sent memory distribution.

5. The method of claim 1, wherein, each node within the plurality of nodes having additional memory beyond the globally tuned memory partition further partitions the additional memory as locally tuned memory that is tuned by a local memory tuner, and wherein the local memory tuner determines how to distribute the locally tuned memory and apply configuration changes locally to the locally tuned memory.

6. The method of claim 1, wherein determining the memory distribution based on the calculated globally tuned memory value further comprises the global memory tuner using aggregated cost/benefit data collected from a local benefit collector associated with each node within the plurality of nodes.