Scalable database management software on a cluster of nodes using a shared-distributed flash memory

• US 8,667,001 B2
• Filed: 06/20/2012
• Issued: 03/04/2014
• Est. Priority Date: 03/20/2008
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: SanDisk Enterprise IP LLC (Western Digital Corporation)
• Inventors: Dinker, Darpan, Eckhardt, Andrew David, Ouye, Darryl Manabu, O'Krafka, Brian Walter, Cohen, Earl T., McWilliams, Thomas M.
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Kim, Paul
• Application Number:
  US13/528,064
• Publication Number:
  US 20120259889A1
• Time in Patent Office: 622 Days
• Field of Search: 707/764, 707/758, 707/781, 707/802
• US Class Current: 707/764
• CPC Class Codes: G06F 12/0842 for multiprocessing or multitasking
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2212/2022 Flash memory
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A distributed database system has multiple compute nodes each running an instance of a database management system (DBMS) program that accesses database records in a local buffer cache. Records are persistently stored in distributed flash memory on multiple storage nodes. A Sharing Data Fabric (SDF) is a middleware layer between the DBMS programs and the storage nodes and has API functions called by the DBMS programs when a requested record is not present in the local buffer cache. The SDF fetches the requested record from flash memory and loads a copy into the local buffer cache. The SDF has threads on a home storage node that locate database records using a node map. A global cache directory locks and pins records to local buffer caches for updating by a node'"'"'s DBMS program. DBMS operations are grouped into transactions that are committed or aborted together as a unit.

Claims

1. A non-transitory computer-readable storage medium storing computer-executable programs, which when executed on a computing device manages a distributed database stored across multiple nodes of flash memory, the computer-executable programs comprising:

query processing software for processing a database query, the database query specifying a target database record in a database;

table reader software for reading the target database record from a local buffer cache when a hit occurs, the local buffer cache storing cached database records in volatile memory;

table writer software for writing to the target database record in the local buffer cache when a hit occurs and the target database record is in the local buffer cache;

Sharing Data Fabric (SDF) thread software, activated by a GET command from the table reader software when the hit does not occur in the local buffer cache, for sending a request message over an interconnect from a first node to a target storage node in a plurality of nodes, and for identifying the target storage node using an identifier of the target database record;

storage thread software, for reading a requested database record from the flash memory at the first node, and for sending the requested database record over the interconnect to the SDF thread software on another node of the multiple nodes; and

miss loader software, in the SDF thread software, for receiving the target database record from the target storage node over the interconnect, and for writing the target database record into the local buffer cache, and for activating the table reader software to read the target database record as a hit,

whereby the target database record is retrieved from the target storage node using the SDF thread software transparently to the table reader software.

2. The non-transitory computer-readable storage medium of claim 1 wherein the computer-executable programs further comprise:

flush software, in the SDF thread software, for sending a flush message to the target storage node, the flush message communicating the target database record modified by the table writer means; and

persistent store software, in the storage thread software, for writing the requested database record to the flash memory at the first node in response to a flush message received from another node of the multiple nodes.

3. The non-transitory computer-readable storage medium of claim 1 wherein the computer-executable programs further comprise:

lock request software, in the SDF thread software, for sending a lock message to the target storage node in response to the table writer means attempting to write to the target database record in the local buffer cache;

wherein the target storage node includes a directory for indicating that the target database record is locked by the first node in response to the lock message, the directory means preventing other nodes from writing to the target database record when locked; and

unlock software, in the SDF thread software, for sending an unlock message to the target storage node in response to the table writer means finishing writing to the target database record in the local buffer cache.

4. The non-transitory computer-readable storage medium of claim 3 wherein the computer-executable program code means further comprises:

sync software, in the SDF thread software, for sending a sync message to the target storage node;

wherein the storage thread software further comprises sync update software for allowing other nodes in the plurality of nodes to access a database record modified by the table writer software;

cache pin software, in the SDF thread software, for marking for retention the target database record in the local buffer cache means during a current transaction.

5. A transactional distributed database system comprising:

a plurality of compute nodes;

a plurality of database management system (DBMS) programs executing on a plurality of processors on the plurality of compute nodes, each DBMS program receiving database queries from clients that identify target data items in a database;

a plurality of storage nodes each having a non-volatile memory that stores a portion of the database, wherein the database is partitioned across the plurality of storage nodes;

wherein the non-volatile memory is a solid-state memory that retains data when power is disconnected;

an interconnect that passes messages between the plurality of compute nodes and the plurality of storage nodes; and

a plurality of Sharing Data Fabric (SDF) threads executing on the plurality of processors on the plurality of compute nodes, a SDF thread being activated by a respective DBMS program of the plurality of DBMS programs when the target data items are not present in a local buffer cache, the SDF thread identifying a target storage node in the plurality of storage nodes, the target storage node being a home node that persistently stores the target data items, the SDF thread sending a request message to the target storage node;

wherein each storage node further comprises a processor executing a storage thread, the storage thread reading the target data items from the non-volatile memory in response to the request message received over the interconnect, the storage thread sending the target data items over the interconnect in a reply message sent in response to the request message;

wherein each of the plurality of DBMS programs operates in a shared address space that includes the database stored on the plurality of storage nodes, the plurality of SDF threads transparently accessing data items stored on the plurality of storage nodes.

6. The transactional distributed database system of claim 5 wherein an action node in the plurality of compute nodes comprises:

a transaction table, wherein operations performed by a DBMS program executing on the action node are grouped into a transaction having a transaction entry in the transaction table, wherein a first operation in a new transaction causes a SDF thread executing on the action node to create a new entry in the transaction table for the new transaction;

wherein the SDF thread adds a list of touched data items to the new entry in the transaction table when subsequent operations in the transaction access the touched data items in the database;

wherein a final commit operation in the new transaction causes the SDF thread to flush modified data items in the list of touched data items in the new entry, wherein modified data items are sent over the interconnect to at least a subset of the plurality of storage nodes for storage in the non-volatile memory of the storage nodes storing the modified data items;

wherein when an error occurs during processing of the new transaction, an abort operation is processed instead of the final commit operation, the abort operation preventing any of the modified data items from updating the non-volatile memory in the plurality of storage nodes, whereby operations in the new transaction are committed together or aborted together as an atomic transaction.