INSERTING DATA INTO AN IN-MEMORY DISTRIBUTED NODAL DATABASE

• US 20090083276A1
• Filed: 09/26/2007
• Published: 03/26/2009
• Est. Priority Date: 09/26/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Santosuosso, John Matthew, Darrington, David L., Barsness, Eric L., Peters, Amanda
• Granted Patent: US 8,892,558 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/35 Clustering; Classification

Abstract

A method and apparatus loads data to an in-memory database across multiple nodes in a parallel computing system. A database loader uses SQL flags, historical information gained from monitoring prior query execution times and patterns, and node and network configuration to determine how to effectively cluster data attributes across multiple nodes. The database loader may also allow a system administrator to force placement of database structures in particular nodes.

Claims

1. A parallel computer system comprising:

a plurality of fully functional compute nodes; and

a database loader for pre-loading an in-memory database to optimize database efficiency by clustering database attributes, wherein clustering database attributes comprises placing data corresponding to a first attribute for a plurality of records together on a first node in the database and placing data corresponding to a second attribute for the plurality of records together on a second node in the database.

2. The parallel computer system of claim 1 wherein the parallel computer system is a massively parallel computer system.

3. The parallel computer system of claim 1 wherein the database loader determines to cluster the database attributes based on a flag in an SQL statement that initiates placing the record in the database.

4. The parallel computer system of claim 1 wherein the database loader determines to cluster the database attributes based on historical information for accessing the database.

5. The parallel computer system of claim 4 wherein the historical information includes information chosen from the following:

node information, network information and query historical information.

6. The parallel computer system of claim 5 wherein the node information includes node identification, timestamp, current utilization, future utilization and availability.

7. The parallel computer system of claim 5 wherein the network information includes network identification, timestamp, current utilization future utilization and availability.

8. The parallel computer system of claim 5 wherein the query information includes query identification, network used, elapsed time, node list and priority.

9. A computer implemented method for pre-loading an in-memory database into the compute nodes of a parallel computer system, the method comprising the steps of:

receiving a database structure to load into the in-memory database;

determining an optimized data node mapping to cluster database attributes across multiple nodes, wherein clustering database attributes across multiple nodes comprises placing data corresponding to a first attribute for a plurality of records together on a first node in the database and placing data corresponding to a second attribute for the plurality of records together on a second node in the database; and

loading the database structure into the in-memory database with the determined optimized data node mapping.

10. The computer implemented method of claim 9 further comprises the step of determining the data is accessed often.

11. The computer implemented method of claim 9 further comprising the step of determining if there is a force location for the database structure in the in-memory database as indicated by a system administrator input.

12. The computer implemented method of claim 9 wherein it is determined to cluster the database attributes based on a flag in an SQL statement that initiates placing the record in the database.

13. The computer implemented method of claim 9 wherein it is determined to cluster the database attributes based on a historical information for accessing the database

14. A computer-readable article of manufacture comprising:

a database loader for pre-loading an in-memory database to optimize database efficiency by clustering database attributes, wherein clustering database attributes comprises placing data corresponding to a first attribute for a plurality of records together on a first node in the database and placing data corresponding to a second attribute for the plurality of records together on a second node in the database; and

tangible computer recordable media bearing the database loader.

15. The article of manufacture of claim 14 wherein the database loader determines to cluster the database attributes based on a flag in an SQL statement that initiates placing the record in the database.

16. The article of manufacture of claim 14 wherein the database loader determines to cluster the database attributes based on a historical information for accessing the database.

17. The article of manufacture of claim 14 wherein the historical information includes information chosen from the following:

node information, network information and query historical information.

18. The article of manufacture of claim 17 wherein the node information includes node identification, timestamp, current utilization future utilization and availability.

19. The article of manufacture of claim 17 wherein the network information includes network identification, timestamp, current utilization future utilization and availability.

20. The article of manufacture of claim 17 wherein the query information includes query identification, network used, elapsed time, node list and priority.