Database with NVDIMM as persistent storage

• US 10,802,766 B2
• Filed: 09/29/2017
• Issued: 10/13/2020
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Choudhury, Nilesh, Umamageswaran, Kothanda, Meiyyappan, Krishnan, Shi, Jia, Sridharan, Vijay, Tao, Zuoyu, Zhang, Kai, Ustimenko, Semen, Kowsalya, Salini Selvaraj, Sardashti, Somayeh
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US15/720,959
• Publication Number:
  US 20190102113A1
• Time in Patent Office: 1,110 Days
• Field of Search: None
• CPC Class Codes: G06F 16/1847 specifically adapted to static storage, e.g. adapted to flash memory or SSD
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/90339 by using parallel associative memories or content-addressable memories
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Abstract

A shared storage architecture persistently stores database files in non-volatile random access memories (NVRAMs) of computing nodes of a multi-node DBMS. The computing nodes of the multi-node DBMS not only collectively store database data on NVRAMs of the computing nodes, but also host database server instances that process queries in parallel, host database sessions and database processes, and together manage access to a database stored on the NVRAMs of the computing nodes. To perform a data block read operation from persistent storage, a data block may be transferred directly over a network between NVRAM of a computing node that persistently stores the data block to a database buffer in non-volatile RAM of another computing node that requests the data block. The transfer is accomplished using remote direct memory access (“RDMA).

Claims

1. A method, comprising:

storing a database in a plurality of database files in non-volatile random access memory (NVRAM), of a plurality of computing nodes in a multi-node database management system (DBMS), each computing node of said plurality of computing nodes hosting a database server instance;

wherein each computing node of said plurality of computing nodes comprises NVRAM and stores a respective subset of said plurality of database files in the NVRAM of said each computing node;

a database process on a first computing node of said plurality of computing nodes executing a query, wherein executing said query comprises;

determining that a first data block is stored in the NVRAM of a second computing node of said plurality of computing nodes;

issuing a RDMA read request to a first RDMA hardware mechanism on said first computing node to read said first data block from the NVRAM of the second computing node;

a second RDMA hardware mechanism on said second computing node retrieving said first data block from said NVRAM and transmitting said first data block to said first RDMA hardware mechanism;

said first RDMA hardware mechanism storing said first data block in a database buffer of pool of database buffers on said first computing node.

2. The method of claim 1, wherein:

the method includes each computing node of said plurality of computing nodes storing mapping data that maps a plurality of database tables of said data database to storage locations within the NVRAM of said plurality of said computing nodes, said storage locations storing data for said plurality of database tables;

wherein for said first computing node of said plurality of computing nodes, the mapping data stored on said first computing node maps a first database table to a storage location within the NVRAM of said second computing node;

wherein determining that a first data block is stored in the NVRAM of said second computing node includes examining the mapping data stored on said first computing node to determine that a storage location of said first data block is at the NVRAM of said second computing node.

3. The method of claim 1, wherein:

the method includes each computing node of said plurality of computing nodes storing mapping data that maps a plurality of database tables of said database to storage locations within the NVRAM of said plurality of said computing nodes, said storage locations storing data for said plurality of database tables;

said first data block stores data for a first database table of said plurality of database tables;

wherein the mapping data stored on said first computing node comprises first mapping data;

wherein said first mapping data maps as a primary storage location of a first portion of data for said first database table to a storage location in the NVRAM of said second computing node;

wherein determining that a first data block is stored in the NVRAM of said second computing node includes examining the mapping data stored on said first computing node to determine that said primary storage location of said first data block is at the NVRAM of said second computing node.

4. The method of claim 3,

wherein said first mapping data maps as a secondary storage location of a second portion of data for said first database table to a storage location at the NVRAM of said first computing node;

wherein said first database process executing said query includes;

based on the first mapping data, determining that a secondary storage location of a second data block is at the NVRAM of said first computing node;

reading said second data block from the NVRAM of said first computing node;

storing said second data block in said database buffer of pool of database buffers on said first computing node.

5. The method of claim 1, further including:

each computing node of said plurality of computing nodes running a storage server process that responds to filtered data block requests by returning only data blocks that contain at least one row that satisfies one or more filtering criteria specified by said filtered data block requests;

wherein a database process on a first computing node of said plurality of computing nodes executing a query comprises;

said database process on said first computing node sending a first filtered data block request to a storage server process on said second computing node, said first filtered data block request identifying data blocks stored in the NVRAM of said second computing node; and

the storage server process on said second computing node returning data blocks that satisfy one or more filtering criteria specified by said filtered data block request.

6. The method of claim 5, wherein:

a database process on a first computing node of said plurality of computing nodes executing a query comprises said database process on said first computing node sending a second filtered data block request to a second storage server process on said first computing node, said second filtered data block request identifying data blocks stored in the NVRAM of said first computing node;

the method further comprises said second storage server process on said first computing node returning data blocks that satisfy one or more filtering criteria specified by said second filtered data block request.

7. The method of claim 5, wherein the method further comprises:

said second computing node loading a particular data block from the NVRAM of said second computing node to a database buffer in volatile RAM of said second computing node;

said second computing node examining said particular data block in said database buffer to determine whether any row in said particular data block satisfies the one or more filtering criteria specified by said first filtered data block request.

8. The method of claim 5, wherein the method further comprises:

said second computing node examining a particular data block in-place in said NVRAM of said second computing node to determine whether any row in said particular data block satisfies the one or more filtering criteria specified by said first filtered data block request.

9. The method of claim 1, wherein database files of said plurality of database files are stored in the NVRAM of said first computing node in non-interleaved mode.

10. The method of claim 1, wherein executing said query further comprises:

determining that a particular data block is stored in the NVRAM of a storage cell;

issuing a RDMA read request to the first RDMA hardware mechanism on said first computing node to read said particular data block from the NVRAM of the storage cell; and

a particular RDMA hardware mechanism on said storage cell retrieving said particular data block from said NVRAM of said storage cell and transmitting said particular data block to said first RDMA hardware mechanism.

11. One or more non-transitory computer-readable media storing sequences of instructions that, when executed by one or more processors, cause:

storing a database in a plurality of database files in non-volatile random access memory (NVRAM) of a plurality of computing nodes in a multi-node database management system (DBMS), each computing node of said plurality of computing nodes hosting a database server instance;

wherein each computing node of said plurality of computing nodes comprises NVRAM and stores a respective subset of said plurality of database files in the NVRAM of said each computing node;

a database process on a first computing node of said plurality of computing nodes executing a query, wherein executing said query comprises;

determining that a first data block is stored in the NVRAM of a second computing node of said plurality of computing nodes;

issuing a RDMA read request to a first RDMA hardware mechanism on said first computing node to read said first data block from the NVRAM of the second computing node;

a second RDMA hardware mechanism on said second computing node retrieving said first data block from said NVRAM and transmitting said first data block to said first RDMA hardware mechanism;

said first RDMA hardware mechanism storing said first data block in a database buffer of pool of database buffers on said first computing node.

12. The one or more non-transitory computer-readable media of claim 11, Wherein:

the sequences of instructions include instructions that, when executed by said one or more processors, cause each computing node of said plurality of computing nodes storing mapping data that maps a plurality of database tables of said data database to storage locations within the NVRAM of said plurality of said computing nodes, said storage locations storing data for said plurality of database tables;

wherein for said first computing node of said plurality of computing nodes, the mapping data stored on said first computing node maps a first database table to a storage location within the NVRAM of said second computing node;

wherein determining that a first data block is stored in the NVRAM of said second computing node includes examining the mapping data stored on said first computing node to determine that a storage location of said first data block is at the NVRAM of said second comp in computing node.

13. The one or more non-transitory computer-readable media of claim 11, wherein:

the sequences of instructions include instructions that, when executed by said one or more processors, cause each computing node of said plurality of computing nodes storing mapping data that maps a plurality of database tables of said database to storage locations within the NVRAM of said plurality of said computing nodes, said storage locations storing data for said plurality of database tables;

said first data block stores data for a first database table of said plurality of database tables;

wherein the mapping data stored on said first computing node comprises first mapping data;

wherein said first mapping data maps as a primary storage location of a first portion of data for said first database table to a storage location in the NVRAM of said second computing node;

wherein determining that a first data block is stored in the NVRAM of said second computing node includes examining the mapping data stored on said first computing node to determine that said primary storage location of said first data block is at the NVRAM of said second computing node.

14. The one or more non-transitory computer-readable media of claim 13,

wherein said first mapping data maps as a secondary storage location of a second portion of data for said first database table to a storage location at the NVRAM of said first computing node;

wherein said first database process executing said query includes;

based on the first mapping data, determining that a secondary storage location of a second data block is at the NVRAM of said first computing node;

reading said second data block from the NVRAM of said first computing node;

storing said second data block in said database buffer of pool of database buffers on said first computing node.

15. The one or more non-transitory computer-readable media of claim 11, wherein:

the sequences of instructions include instructions that, when executed by said one or more processors, cause each computing node of said plurality of computing nodes running a storage server process that responds to filtered data block requests by returning only data blocks that contain at least one row that satisfies one or more filtering criteria specified by said filtered data block requests;

wherein a database process on a first computing node of said plurality of computing nodes executing a query comprises;

said database process on said first computing node sending a first filtered data block request to a storage server process on said second computing node, said first filtered data block request identifying data blocks stored in the NVRAM of said second computing node; and

the sequences of instructions include instructions that, when executed by said one or more processors, cause the storage server process on said second computing node returning data blocks that satisfy one or more filtering criteria specified by said filtered data block request.

16. The one or more non-transitory computer-readable media of claim 15, wherein:

a database process on a first computing node of said plurality of computing nodes executing a query comprises;

said database process on said first computing node sending a second filtered data block request to a second storage server process on said first computing node, said second filtered data block request identifying data blocks stored in the NVRAM of said first computing node;

the sequences of instructions include instructions that, when executed by said one or more processors, cause said second storage server process on said first computing node returning data blocks that satisfy one or more filtering criteria specified by said second filtered data block request.

17. The one or more non-transitory computer-readable media of claim 15, the sequences of instructions include instructions that, when executed by said one or more processors, cause:

said second computing node loading a particular data block from the NVRAM of said second computing node to a database buffer in volatile RAM of said second computing node;

said second computing node examining said particular data block in said database buffer to determine whether any row in said particular data block satisfies the one or more filtering criteria specified by said first filtered data block request.

18. The one or more non-transitory computer-readable media of claim 15, the sequences of instructions include instructions that, when executed by said one or more processors, cause said second computing node examining a particular data block in-place in said NVRAM of said second computing node to determine whether any row in said particular data block satisfies the one or more filtering criteria specified by said first filtered data block request.

19. The one or more non-transitory computer-readable media of claim 11, wherein database files of said plurality of database files are stored in the NVRAM of said first computing node in non-interleaved mode.

20. The one or more non-transitory computer-readable media of claim 11, wherein executing said query comprises:

determining that a particular data block is stored in the NVRAM of a storage cell;

issuing a RDMA read request to the first RDMA hardware mechanism on said first computing node to read said particular data block from the NVRAM of the storage cell; and

a particular RDMA hardware mechanism on said storage cell retrieving said particular data block from said NVRAM of said storage cell and transmitting said particular data block to said first RDMA hardware mechanism.