Storage performance optimization

• US 8,924,357 B2
• Filed: 07/09/2012
• Issued: 12/30/2014
• Est. Priority Date: 05/16/2008
• Status: Active Grant

• Current Assignee: Paraccel Incorporated (Actian Corporation)
• Original Assignee: Paraccel Incorporated (Actian Corporation)
• Inventors: Zane, Barry M., Steinhoff, David E.
• Primary Examiner(s): REYES, MARIELA D
• Application Number:
  US13/544,879
• Publication Number:
  US 20120278285A1
• Time in Patent Office: 904 Days
• Field of Search: 707/649, 707/655, 707/656
• US Class Current: 707/649
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system and method for enhancing data throughput in data warehousing environments by connecting multiple servers having local storages with designated external storage systems, such as, for example, those provided by SANS. The system and method may preserve a full reference copy of the data in a protected environment (e.g., on the external storage system) that is fully available. The system and method may enhance overall I/O potential performance and reliability for efficient and reliable system resource utilization.

Claims

1. A database system comprising a plurality of server nodes, including a given server node, the plurality of server nodes comprising:

one or more computer processors; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the one or more computer processors, the computer program modules comprising instructions, which, when executed, cause the one or more computer processors to;

maintain a local copy of a database comprising at least a portion of the database on a local electronic storage that is local to a given server node, the local electronic storage coupled to the computer processor;

maintain a reference copy of the database, corresponding to a complete copy of the database, stored in an external storage system, the external storage system shared between the plurality of server nodes and storing complete copy of the portions of database stored on two or more of the plurality of server nodes, wherein the complete copy of the database held on the external storage system operates as a backup for the local copy of the database and the complete copy of the database is kept current with respect to the local copy of the database, wherein maintaining the reference copy of the database comprises;

receiving a request to update data stored in the local storage,

responsive to receiving the request to update, identifying data stored on the external storage system corresponding to the data being updated in the local electronic storage, and

performing an update of the data in the local electronic storage and an update of the corresponding data stored on the external storage system as a single operation; and

responsive to receiving a query;

map the query to a first portion of result data stored in the local electronic storage of the given server node;

map the query to the reference copy of the database stored in the shared external storage corresponding to the given server node;

identify a second portion of result data stored in the external storage system, so that the first portion of the result data and the second portion of the result data are distinct;

retrieve, in parallel, to the given server node, the first portion of the result data from the local electronic storage of the given server node and the second portion of the result data from the reference copy of the database stored in the external storage system; and

combine, at the given server node, the first and second portion of the result data to generate a complete result to the query.

2. The database system of claim 1, wherein the computer program modules further comprise instructions that cause the one or more processors to take a snapshot of the reference copy of the database in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

3. The database system of claim 2, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

4. The database system of claim 1, wherein the computer program modules further comprise instructions that cause the one or more processors to take a snapshot of the reference copy of the database stored in the external storage system, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

5. The database system of claim 1, wherein the computer program modules further comprise instructions that cause the one or more processors to:

maintain tables of contents corresponding to snapshots of database, each table of content representing information describing data blocks of a snapshot.

6. The database system of claim 5, wherein the table of content stores storage address of data blocks of the database.

7. The database system of claim 1, wherein the computer program modules further comprise instructions that cause the one or more processors to:

maintain a current table of content representing information describing data blocks;

receive a request for a snapshot; and

responsive to the request, create the snapshot and store the current table of content in association with the snapshot.

8. The database system of claim 7, wherein the current table of contents is a previous table of contents, wherein the instructions further cause the one or more processors to:

create a new table of content and initialize the new table of content based on the previous table of contents; and

modify the new table of content responsive to changes in data blocks of the database.

9. A non-transitory computer readable storage medium including computer executable code adapted to perform steps comprising:

maintaining a local copy of a database comprising at least a portion of the database on a local electronic storage that is local to a given server node of a plurality of server nodes, the local electronic storage coupled to a given computer processor;

maintaining a reference copy of the database, corresponding to a complete copy of the database, stored in an external storage system, the external storage system shared between the plurality of server nodes and storing copy of the portions of database stored on two or more of the plurality of server nodes, wherein the complete copy of the database held on the external storage system operates as a backup for the local copy of the database and the complete copy of the database is kept current with respect to the local copy of the database, wherein maintaining the reference copy of the database comprises;

receiving a request to update data stored in the local electronic storage,

responsive to receiving the request to update, identifying data stored on the external storage system corresponding to the data being updated in the local electronic storage, and

performing an update of the data in the local storage and an update of the corresponding data stored on the external storage system as a single operation; and

responsive to receiving a query;

mapping the query to a first portion of result data stored in the local storage of the given server node;

mapping the query to the reference copy of the database stored in the shared external storage corresponding to the given server node;

identifying a second portion of result data stored in the external storage system, so that the first portion of the result data and the second portion of the result data are distinct;

retrieving, in parallel, by the given server node, the first portion of the result data from the local storage and the second portion of the result data from the reference copy stored in the external storage system; and

combining, at the given server node, the first and second portion of the result data to generate a complete result to the query.

10. The non-transitory computer readable storage medium of claim 9, wherein computer executable code is further adapted to take a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

11. The non-transitory computer readable storage medium of claim 10, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

12. The non-transitory computer readable storage medium of claim 9, wherein computer executable code is further adapted to perform the step of taking a snapshot of the reference copy of the database stored in the external storage system, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

13. A method for maintaining storage of data at a given server node of a plurality of server nodes in a database system, the method comprising:

maintaining a local copy of a database comprising at least a portion of the database on a local electronic storage that is local to a given server node, the local electronic storage coupled to a given computer processor;

maintaining a reference copy of the database, corresponding to a complete copy of the database, stored in an external storage system, the external storage system shared between the plurality of server nodes and storing copy of the portions of database stored on two or more of the plurality of server nodes, wherein the complete copy of the database held on the external storage system operates as a backup for the local copy of the database and the complete copy of the database is kept current with respect to the local copy of the database, wherein maintaining the reference copy of the database comprises;

receiving a request to update data stored in the local storage,

responsive to receiving the request to update, identifying data stored on the external storage system corresponding to the data being updated in the local storage, and

performing an update of the data in the local storage and an update of the corresponding data stored on the external storage system as a single operation; and

responsive to receiving a query;

mapping the query to a first portion of result data stored in the local storage of the given server node;

mapping the query to the reference copy of the database stored in the shared external storage corresponding to the given server node;

identifying a second portion of result data stored in the external storage system, so that the first portion of the result data and the second portion of the result data are distinct;

retrieving, in parallel, by the given server node, the first portion of the result data from the local storage and the second portion of the result data from the reference copy stored in the external storage system; and

combining, at the given server node, the first and second portion of the result data to generate a complete result to the query.

14. The method of claim 13, further comprising, taking a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

15. The method of claim 14, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

16. The method of claim 14, wherein the parameter describing the database comprises an amount of time passed since a previous snapshot was taken.

17. The method of claim 14, wherein the parameter describing the database comprises an amount of information added, deleted, or updated within the database.

18. The method of claim 13, wherein maintaining the reference copy of the database in the external storage system comprises taking a snapshot of the reference copy of the database, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

19. The method of claim 18, wherein the snapshot of the reference copy of the database is taken without isolating the external storage system from the server.

20. A non-transitory computer readable storage medium including computer executable code adapted to perform steps comprising:

maintaining at least a portion of a database on a local electronic storage that is local to a given server node of a plurality of server nodes, the local electronic storage coupled to a given computer processor;

maintaining a reference copy of the database, corresponding to a complete copy of the database, stored in an external storage system, the external storage system shared between the plurality of server nodes and storing copy of the portions of database stored on two or more of the plurality of server nodes;

responsive to receiving a query;

mapping the query to a first portion of result data stored in the local electronic storage of the given server node;

mapping the query to the reference copy of the database stored in the shared external storage corresponding to the given server node;

identifying a second portion of result data stored in the external storage system, so that the first portion of the result data and the second portion of the result data are distinct;

retrieving, in parallel, by the given server node, the first portion of the result data from the local storage and the second portion of the result data from the reference copy stored in the external storage system;

combining, at the given server node, the first and second portion of the result data to generate a complete result to the query; and

storing a first table of contents comprising information describing blocks of data before a commit operation of a transaction; and

storing a second table of contents comprising information describing blocks of data after the commit operation including blocks generated by the transaction and excluding blocks abandoned by the transaction.

21. The non-transitory computer readable storage medium of claim 20, wherein the steps further comprise, taking a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

22. The non-transitory computer readable storage medium of claim 21, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

23. The non-transitory computer readable storage medium of claim 21, wherein the parameter describing the database comprises an amount of information added, deleted, or updated within the database.

24. A method for maintaining storage of data in a database system, the method comprising:

maintaining at least a portion of a database on a local electronic storage that is local to a given server node, the local electronic storage coupled to a given computer processor;

maintaining a reference copy of the database, corresponding to a complete copy of the database, stored in an external storage system, the external storage system shared between a plurality of server nodes and storing copy of the portions of database stored on two or more of the plurality of server nodes;

responsive to receiving a query;

mapping the query to a first portion of result data stored in the local electronic storage of the given server node;

mapping the query to the reference copy of the database stored in the shared external storage corresponding to the given server node;

identifying a second portion of result data stored in the external storage system, so that the first portion of the result data and the second portion of the result data are distinct;

retrieving, in parallel, by the given server node, the first portion of the result data from the local storage and the second portion of the result data from the reference copy stored in the external storage system;

combining, at the given server node, the first and second portion of the result data to generate a complete result to the query; and

storing a first table of contents comprising information describing blocks of data before a commit operation of a transaction; and

storing a second table of contents comprising information describing blocks of data after the commit operation including blocks generated by the transaction and excluding blocks abandoned by the transaction.

25. The method of claim 24, further comprising, taking a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

26. The method of claim 25, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

27. The method of claim 25, wherein the parameter describing the database comprises an amount of information added, deleted, or updated within the database.