Storage Performance Optimization

• US 20120278285A1
• Filed: 07/09/2012
• Published: 11/01/2012
• Est. Priority Date: 05/16/2008
• Status: Active Grant

• Current Assignee: Paraccel Incorporated (Actian Corporation)
• Original Assignee: Paraccel Incorporated (Actian Corporation)
• Inventors: Zane, Barry M., Steinhoff, David
• Granted Patent: US 8,924,357 B2
• Time in Patent Office: Days
• US Class Current: 707/639
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system and method for enhancing data throughput in data warehousing environments by connecting multiple servers having local storages with designated external storage systems, such as, for example, those provided by SANS. The system and method may preserve a full reference copy of the data in a protected environment (e.g., on the external storage system) that is fully available. The system and method may enhance overall I/O potential performance and reliability for efficient and reliable system resource utilization.

Claims

1. A database system comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising a distributed data manager configured to;

maintain at least a first portion of a database on a local electronic storage that is local to a server;

maintain a reference copy of the database stored in an external storage system;

responsive to receiving a request to commit a transaction, update blocks of data stored in the local storage and update corresponding blocks in the reference copy of the database stored in the external storage system in a single operation;

responsive to receiving a query, retrieve a first portion of the requested data from the local storage and a second portion of the requested data from the reference copy of the database stored in the external storage system for returning a result combining the first and second portion of the requested data; and

responsive to a failure of the local storage, obtain the entire result data for a subsequent query from the reference copy of the database stored in the external storage system.

2. The database system of claim 1, wherein the computer program modules further comprise a data imaging module configured to take a snapshot of the reference copy of the database in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

3. The database system of claim 2, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

4. The database system of claim 1, wherein the computer program modules further comprise a data imaging module configured to take a snapshot of the reference copy of the database stored in the external storage system, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

5. The database system of claim 1, wherein the computer program modules further comprise a data imaging module configured to:

store a first table of contents comprising information describing blocks of data before a commit operation of a transaction; and

store a second table of contents comprising information describing blocks of data after the commit operation including blocks generated by the transaction and excluding blocks abandoned by the transaction.

6. A computer program product having a non-transitory computer readable storage medium and including computer executable code adapted to perform steps comprising:

maintaining at least a first portion of a database on a local electronic storage that is local to a server;

maintaining a reference copy of the database stored in an external storage system;

responsive to receiving a request to commit a transaction, updating blocks of data stored in the local storage and updating corresponding blocks in the reference copy of the database stored in the external storage system in a single operation;

responsive to receiving a query, retrieving a first portion of the requested data from the local storage and a second portion of the requested data from the reference copy stored in the external storage system and returning a result combining the first and second portion of the requested data; and

responsive to a failure of the local storage, obtaining the entire result data for a subsequent query from the reference copy of the database stored in the external storage system.

7. The computer program product of claim 6, wherein computer executable code is further adapted to take a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

8. The computer program product of claim 7, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

9. The computer program product of claim 6, wherein computer executable code is further adapted to perform the step of taking a snapshot of the reference copy of the database stored in the external storage system, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

10. The computer program product of claim 6, wherein computer executable code is further adapted to perform the steps of:

storing a first table of contents comprising information describing blocks of data before a commit operation of a transaction; and

storing a second table of contents comprising information describing blocks of data after the commit operation including blocks generated by the transaction and excluding blocks abandoned by the transaction.

11. A method for maintaining storage of database, the method comprising:

maintaining at least a first portion of a database on a local electronic storage that is local to a server;

maintaining a reference copy of the database stored in an external storage system;

responsive to receiving a request to commit a transaction, updating blocks of data stored in the local storage and updating corresponding blocks in the reference copy of the database stored in the external storage system in a single operation;

responsive to receiving a query, retrieving a first portion of the requested data from the local storage and a second portion of the requested data from reference copy of the database stored in the external storage system and returning a result combining the first and second portion of the requested data; and

responsive to a failure of the local storage, obtaining the entire result data for a subsequent query from the reference copy of the database stored in the external storage system.

12. The method of claim 11, further comprising:

responsive the local storage recovering from the failure, restoring a version of the first portion of the database to the local storage using data from reference copy of the database stored in the external storage system.

13. The method of claim 11, further comprising, taking a snapshot of the reference copy of the database stored in the external storage system, wherein the snapshot is taken at a time determined using a parameter describing the database.

14. The method of claim 13, wherein the parameter describing the database comprises a number of querying operations performed on the database since a previous snapshot was taken.

15. The method of claim 13, wherein the parameter describing the database comprises an amount of time passed since a previous snapshot was taken.

16. The method of claim 13, wherein the parameter describing the database comprises an amount of information added, deleted, or updated within the database.

17. The method of claim 11, wherein maintaining the reference copy of the database in the external storage system comprises taking a snapshot of the reference copy of the database, wherein taking the snapshot comprises storing a table of contents describing blocks of data, the table of contents storing storage address of each block and size of each block.

18. The method of claim 17, wherein the snapshot of the reference copy of the database is taken without isolating the external storage system from the server.

19. The method of claim 11, wherein maintaining the copy of the database in the external storage system comprises:

storing a first table of contents comprising information describing blocks of data before a commit operation of a transaction; and

storing a second table of contents comprising information describing blocks of data after the commit operation including blocks generated by the transaction and excluding blocks abandoned by the transaction.

20. The method of claim 11, wherein updating blocks of data stored in the local storage is performed in parallel with updating corresponding blocks stored in the external storage system.