Non-Uniform Memory Access (NUMA) Database Management System

• US 20170031718A1
• Filed: 08/20/2015
• Published: 02/02/2017
• Est. Priority Date: 08/20/2015
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Wagle, Mehul, Booss, Daniel, Schreter, Ivan
• Granted Patent: US 9,697,048 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 12/1458 by checking the subject access rights
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 9/5016 the resource being the memory

Abstract

Disclosed herein are system, method, and computer program product embodiments for managing non-uniform memory access (NUMA) in a database management system. An embodiment operates by receiving a request to load data from a disk into an in-memory database. An optimal one of the plurality of nodes onto which to load the data from the disk based on a system state is determined. It is determined whether the optimal node has sufficient free local memory in a free list to store the data. If the optimal node does not have sufficient free list memory, a memory allocation is requested from an operating system. An address of the memory storing the data is provided.

Claims

1. A computer implemented method, comprising:

receiving a request to load data from a storage device into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determining, by at least one processor, an optimal one of the plurality of nodes onto which to load the data from the storage device based on a system state, wherein the system state indicates a concurrency of threads executing on the plurality of nodes;

determining whether the optimal node has sufficient free local memory in a free list to store the data, wherein if the optimal node does not have sufficient free list memory;

requesting an allocation of heap memory to be allocated to the optimal node,

storing the data in the requested heap memory, and

binding the requested heap memory storing the data to the optimal node; and

providing, responsive to the request, an address of the allocated heap memory by which the data is accessed;

wherein at least one of the receiving, determining the optimal node, determining whether the optimal node has sufficient free memory, and providing are performed by one or more computers.

2. The method of claim 1, wherein the concurrency of threads comprises non-uniform memory access to the local memory of the plurality of nodes by the threads, wherein a first local memory of a first node of the plurality of nodes is accessed more frequently by the threads than a second local memory of a second node of the plurality of nodes

3. The method of claim 1, wherein the determining a preferred one of the plurality of nodes, comprises determining a secondary node onto which to load the data from the disk in case the optimal node is unavailable.

4. The method of claim 1, wherein the free list comprises a portion of local memory allocated prior to the determining for storing the data.

5. The method of claim 3, further comprising:

receiving a bitmap indicating the optimal node and the secondary node.

6. The method of claim 1, further comprising:

determining that the free list of the optimal node does not have sufficient free memory to store the data;

determining whether the local memory of the optimal node outside the free list has sufficient free local memory to store the data;

allocating a portion of the local memory of the optimal node outside the free list to the free list memory of the optimal node; and

storing the data in the allocated portion of local memory of the optimal node.

7. The method of claim 1, wherein the allocation of heap memory is requested from an operating system.

8. The method of claim 1, wherein the binding occurs when a client requests access to the data stored at the address of the allocated heap memory.

9. The method of claim 1, further comprising:

storing the data in the memory of the optimal node if the optimal node has sufficient free local memory.

10. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to;

receive a request to load data from a storage device into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determine, by at least one processor, an optimal one of the plurality of nodes onto which to load the data from the storage device based on a system state, wherein the system state a concurrency of threads executing on the plurality of nodes;

determine whether the optimal node has sufficient free local memory in a free list to store the data, wherein if the optimal node does not have sufficient free list memory;

request an allocation of heap memory to be allocated to the optimal node,

store the data in the requested heap memory, and

bind the requested heap memory storing the data to the optimal node; and

provide, responsive to the request, an address of the allocated heap memory by which the data is accessed.

11. The system of claim 10, wherein the concurrency of threads comprises non-uniform memory access to the local memory of the plurality of nodes by the threads, wherein a first local memory of a first node of the plurality of nodes is accessed more frequently by the threads than a second local memory of a second node of the plurality of nodes

12. The system of claim 10, wherein the processor configured to determine a preferred one of the plurality of nodes, is configured to determine a secondary node onto which to load the data from the disk in case the optimal node is unavailable.

13. The system of claim 12, wherein the processor is further configured to:

receive a bitmap indicating the optimal node and the secondary node.

14. The system of claim 10, wherein the processor is further configured to:

determine that the free list of the optimal node does not have sufficient free memory to store the data;

determine whether the local memory of the optimal node outside the free list has sufficient free local memory to store the data;

allocate a portion of the local memory of the optimal node outside the free list to the free list memory of the optimal node; and

store the data in the allocated portion of local memory of the optimal node.

15. The system of claim 10, wherein the allocation of heap memory is requested from an operating system.

16. The system of claim 10, wherein the binding occurs when a client requests access to the data stored at the address of the allocated heap memory.

17. The system of claim 10, wherein the free list comprises a portion of local memory allocated prior to the determining for storing the data.

18. A tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

receiving a request to load data from a disk into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determining, by at least one processor, an optimal one of the plurality of nodes onto which to load the data from the disk based on a system state, wherein the system state a concurrency of threads executing on the plurality of nodes;

determining whether the optimal node has sufficient free local memory to store the data, wherein if the optimal node does not have sufficient free local memory;

requesting an allocation of heap memory to be allocated to the optimal node,

storing the data in the requested heap memory, and

binding the requested heap memory storing the data to the optimal node; and

providing, responsive to the request, an address of the allocated heap memory by which the data is accessed.