Non-uniform memory access (NUMA) database management system

• US 9,697,048 B2
• Filed: 08/20/2015
• Issued: 07/04/2017
• Est. Priority Date: 08/20/2015
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Schreter, Ivan, Booss, Daniel, Wagle, Mehul
• Primary Examiner(s): Tang, Kenneth
• Application Number:
  US14/830,874
• Publication Number:
  US 20170031718A1
• Time in Patent Office: 684 Days
• Field of Search: None
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 12/1458 by checking the subject access rights
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 9/5016 the resource being the memory

Abstract

Disclosed herein are system, method, and computer program product embodiments for managing non-uniform memory access (NUMA) in a database management system. An embodiment operates by receiving a request to load data from a disk into an in-memory database. An optimal one of the plurality of nodes onto which to load the data from the disk based on a system state is determined. It is determined whether the optimal node has sufficient free local memory in a free list to store the data. If the optimal node does not have sufficient free list memory, a memory allocation is requested from an operating system. An address of the memory storing the data is provided.

Claims

1. A computer implemented method, comprising:

receiving a request to load data from a storage device into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determining, by at least one processor, a preferred one of the plurality of nodes onto which to load the data from the storage device based on a bitmap indicating the preferred node based on a workload of the plurality of nodes and a relative distance from a client;

determining whether the preferred node has sufficient free local memory in a free list to store the data, wherein if the preferred node does not have sufficient free list memory;

requesting an allocation of heap memory to be allocated to the preferred node,

storing the data in the requested heap memory, and

binding the requested heap memory storing the data to the preferred node; and

providing, responsive to the request, an address of the allocated heap memory by which the data is accessed;

wherein at least one of the receiving, determining the preferred node, determining whether the preferred node has sufficient free memory, and providing are performed by one or more computers.

2. The method of claim 1, wherein the concurrency of threads comprises non-uniform memory access to the local memory of the plurality of nodes by the threads, wherein a first local memory of a first node of the plurality of nodes is accessed more frequently by the threads than a second local memory of a second node of the plurality of nodes.

3. The method of claim 1, wherein the determining a preferred one of the plurality of nodes, comprises determining a secondary node onto which to load the data from the disk in case the preferred node is unavailable.

4. The method of claim 1, wherein the free list comprises a portion of local memory allocated prior to the determining for storing the data.

5. The method of claim 3, wherein the bitmap indicates both the preferred node and the secondary node.

6. The method of claim 1, further comprising:

determining that the free list of the preferred node does not have sufficient free memory to store the data;

determining whether the local memory of the preferred node outside the free list has sufficient free local memory to store the data;

allocating a portion of the local memory of the preferred node outside the free list to the free list memory of the preferred node; and

storing the data in the allocated portion of local memory of the preferred node.

7. The method of claim 1, wherein the allocation of heap memory is requested from an operating system memory heap which is distinguishable from borrowing memory from nodes from the plurality of nodes.

8. The method of claim 1, wherein the binding occurs when a client requests access to the data stored at the address of the allocated heap memory.

9. The method of claim 1, further comprising:

storing the data in the memory of the preferred node if the preferred node has sufficient free local memory.

19. The method of claim 7, wherein the requesting an allocation of heap memory comprises:

requesting the allocation of heap memory from the operating system heap memory based on a determination that there is insufficient memory remaining on the at least a subset of the plurality of nodes to process the request.

10. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to;

receive a request to load data from a storage device into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determine, by at least one processor, a preferred one of the plurality of nodes onto which to load the data from the storage device based on a bitmap indicating the preferred node based on a workload of the plurality of nodes and a relative distance from a client;

determine whether the optimal node has sufficient free local memory in a free list to store the data, wherein if the preferred node does not have sufficient free list memory;

request an allocation of heap memory to be allocated to the preferred node,

store the data in the requested heap memory, and

bind the requested heap memory storing the data to the preferred node; and

provide, responsive to the request, an address of the allocated heap memory by which the data is accessed.

11. The system of claim 10, wherein the concurrency of threads comprises non uniform memory access to the local memory of the plurality of nodes by the threads, wherein a first local memory of a first node of the plurality of nodes is accessed more frequently by the threads than a second local memory of a second node of the plurality of nodes.

12. The system of claim 10, wherein the processor configured to determine a preferred one of the plurality of nodes, is configured to determine a secondary node onto which to load the data from the disk in case the preferred node is unavailable.

13. The system of claim 12, wherein the bitmap indicates both the preferred node and the secondary node.

14. The system of claim 10, wherein the processor is further configured to:

determine that the free list of the preferred node does not have sufficient free memory to store the data;

determine whether the local memory of the preferred node outside the free list has sufficient free local memory to store the data;

allocate a portion of the local memory of the preferred node outside the free list to the free list memory of the preferred node; and

store the data in the allocated portion of local memory of the preferred node.

15. The system of claim 10, wherein the allocation of heap memory is requested from an operating system.

16. The system of claim 10, wherein the binding occurs when a client requests access to the data stored at the address of the allocated heap memory.

17. The system of claim 10, wherein the free list comprises a portion of local memory allocated prior to the determining for storing the data.

18. A tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

receiving a request to load data from a disk into an in-memory database, wherein the in-memory database is distributed across a plurality of nodes, each node comprising a local memory;

determining, by at least one processor, a preferred one of the plurality of nodes onto which to load the data from the storage device based on a bitmap indicating the preferred node based on a workload of the plurality of nodes and a relative distance from a client;

determining whether the preferred node has sufficient free local memo store the data, wherein if the preferred node does not have sufficient free local memory;

requesting an allocation of heap memory to be allocated to the preferred node,

storing the data in the requested heap memory, and

binding the requested heap memory storing the data to the preferred node; and

providing, responsive to the request, an address of the allocated heap memory by which the data is accessed.