Retrieving point-in-time copies of a source database for creating virtual databases

• US 10,067,952 B2
• Filed: 10/11/2013
• Issued: 09/04/2018
• Est. Priority Date: 10/11/2012
• Status: Active Grant

• Current Assignee: Delphix Corporation (Perforce Software Incorporated)
• Original Assignee: Delphix Corporation (Perforce Software Incorporated)
• Inventors: Ahrens, Matthew Allan, Sinha, Subhadeep, Wilson, George Raymond, Biseda, Jeffrey Karr
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Skhoun, Hicham
• Application Number:
  US14/052,580
• Publication Number:
  US 20140108352A1
• Time in Patent Office: 1,789 Days
• Field of Search: 707639
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/065 Replication mechanisms

Abstract

A storage system maintains copies of a source database to allow virtual databases to be provisioned. Transaction logs received from the source database are applied to database blocks stored in the storage system to update the database blocks. The storage system checks if there are defect in the transaction logs of the source database. If a defect in transaction logs is detected, the storage system obtains a latest copy of the source database. The storage system matches database blocks of the received copy with database blocks already available in the storage system to determine whether a corresponding database block is already available in the storage system. If a received database block is different from database blocks already available at the storage system, the received database block is stored in the storage system. The database blocks may be matched using a hash value of the database blocks.

Claims

1. A method for storing copies of databases in a database storage system, the method comprising:

maintaining, by the database storage system, a plurality of copies of a source database on a storage system, each copy including a plurality of database blocks, wherein a database block is a unit of data used by a database, wherein a portion of the database block stores metadata associated with the database block, wherein one or more database blocks are associated with at least two copies of the source database stored on the database storage system, the maintaining comprising, iteratively;

receiving, by the database storage system, from a backup server maintaining backups of the source database, a full backup of the source database;

restoring the full backup of the source database to obtain a first copy of the source database;

creating, by the database storage system, one or more intermediate copies of the source database, comprising;

receiving transaction logs from the backup server; and

creating an intermediate copy by updating the first copy or a previously created intermediate copy of the source database by applying the received transaction logs;

determining whether a recent copy of the source database can be successfully updated using transaction logs received from the backup server;

responsive to detecting a failure in updating the recent copy using the received transaction logs, receiving, from the backup server, another full backup of the source database;

restoring the other full backup of the source database to obtain a plurality of database blocks; and

determining, by the database storage system, a subset of the plurality of database blocks, the determining comprising, excluding database blocks that match corresponding database blocks of a previously stored copy of the source database;

storing the determined subset of database blocks as a new copy of the source database.

2. The method of claim 1, wherein determining whether the recent copy of the source database can be successfully updated comprises determining whether any transaction logs in the received transaction logs are missing.

3. The method of claim 2, wherein determining whether any transaction logs in the received transaction logs are missing comprises comparing sequence numbers of consecutive transaction logs received.

4. The method of claim 1, wherein determining whether the recent copy of the source database can be successfully updated comprises determining whether any transaction logs in the received transaction logs are corrupted.

5. The method of claim 1, wherein restoring the received transaction logs comprises mounting the database storage system on a database server and sending a request to the database server to restore the received transaction logs.

6. The method of claim 1, wherein restoring the received transaction logs comprises:

determining whether the transaction logs received are in backup format; and

responsive to determining that the transaction logs are in backup format, converting the transaction logs from backup format to transaction log format.

7. The method of claim 1, wherein transaction logs are received from the backup server based on a predefined schedule.

21. The computer-implemented method of claim 1, further comprising:

creating a virtual database comprising database blocks from a plurality of copies of the source database stored on the database storage system, the virtual database sharing database blocks with one or more other virtual databases.

8. A computer program product having a non-transitory computer-readable storage medium storing computer program modules configured to execute on a computer processor, the computer program modules comprising:

a database storage system configured to maintain a plurality of copies of a source database on a database storage system, each copy including a plurality of database blocks, wherein a database block is a unit of data used by a database, wherein a portion of the database block stores metadata associated with the database block, wherein one or more database blocks are associated with at least two copies of the source database stored on the database storage system, the database storage system comprising instructions to;

receive, from a backup server maintaining backups of the source database, a full backup of the source database;

send a request to restore the full backup of the source database to obtain a first copy of the source database;

create one or more intermediate copies of the source database by receiving transaction logs from the backup server and creating an intermediate copy by updating the first copy or a previously created intermediate copy of the source database by applying the received transaction logs;

determine whether a recent copy of the source database can be successfully updated using transaction logs received from the backup server;

responsive to detecting a failure in updating the recent copy using the received information, receive, from the backup server, another full backup of the source database;

send a request to restore the other full backup of the source database to obtain a plurality of database blocks; and

determine a subset of the plurality of database blocks by excluding, the database blocks that match corresponding database blocks of a previously stored copy of the source database;

store the determined subset of database blocks as a new copy of the source database.

9. The computer program product of claim 8, wherein the database storage system is configured to determine whether the recent copy of the source database can be successfully updated by determining whether any transaction logs in the received transaction logs are missing.

10. The computer program product of claim 9, wherein the database storage system is configured to determine whether any transaction logs in the received transaction logs are missing by comparing sequence numbers of consecutive transaction logs received.

11. The computer program product of claim 8, wherein the database storage system is configured to determine whether the recent copy of the source database can be successfully updated by determining whether any transaction logs in the received transaction logs are corrupted.

12. The computer program product of claim 8, wherein the database storage system is configured to restore the received transaction logs by mounting the database storage system on a database server and sending a request to the database server to restore the received transaction logs.

13. The computer program product of claim 8, wherein the database storage system is configured to restore the received transaction logs by determining whether the transaction logs received are in backup format, and if the transaction logs are determined to be in backup format, converting the transaction logs from backup format to transaction log format.

14. The computer program product of claim 8, wherein transaction logs are received from the backup server based on a predefined schedule.

15. A computer-implemented system for storing copies of databases in a database storage system, the system comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising;

a database storage system configured to maintain a plurality of copies of a source database on a storage system, each copy including a plurality of database blocks, wherein a database block is a unit of data used by a database, wherein a portion of the database block stores metadata associated with the database block, wherein one or more database blocks are associated with at least two copies of the source database stored on the database storage system, the database storage system comprising instructions to;

receive, from a backup server maintaining backups of the source database, a full backup of the source database;

send a request to restore the full backup of the source database to obtain a first copy of the source database;

create one or more intermediate copies of the source database by receiving transaction logs from the backup server and creating an intermediate copy by updating the first copy or a previously created intermediate copy of the source database by applying the received transaction logs;

determine whether a recent copy of the source database can be successfully updated using transaction logs received from the backup server;

responsive to detecting a failure in updating the recent copy using the received information, receive, from the backup server, another full backup of the source database;

send a request to restore the other full backup of the source database to obtain a plurality of database blocks; and

determine a subset of the plurality of database blocks by excluding, the database blocks that match corresponding database blocks of a previously stored copy of the source database;

store the determined subset of database blocks as a new copy of the source database.

16. The computer-implemented system of claim 15, wherein the database storage system is configured to determine whether the recent copy of the source database can be successfully updated by determining whether any transaction logs in the received transaction logs are missing.

17. The computer-implemented system of claim 16, wherein the database storage system is configured to determine whether any transaction logs in the received transaction logs are missing by comparing sequence numbers of consecutive transaction logs received.

18. The computer-implemented system of claim 15, wherein the database storage system is configured to determine whether the recent copy of the source database can be successfully updated by determining whether any transaction logs in the received transaction logs are corrupted.

19. The computer-implemented system of claim 15, wherein the database storage system is configured to restore the received transaction logs by mounting the database storage system on a database server and sending a request to the database server to restore the received transaction logs.

20. The computer-implemented system of claim 15, wherein transaction logs are received from the backup server based on a predefined schedule.