RETRIEVING POINT-IN-TIME COPIES OF A SOURCE DATABASE FOR CREATING VIRTUAL DATABASES

• US 20140108348A1
• Filed: 03/13/2013
• Published: 04/17/2014
• Est. Priority Date: 10/11/2012
• Status: Active Grant

• Current Assignee: Delphix Corporation (Perforce Software Incorporated)
• Original Assignee: Delphix Corporation (Perforce Software Incorporated)
• Inventors: Sinha, Subhadeep, Ahrens, Matthew Allen, Wilson, George Raymond, Biseda, Jeffrey Karr
• Granted Patent: US 10,346,369 B2
• Time in Patent Office: Days
• US Class Current: 707/639
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/065 Replication mechanisms

Abstract

A storage system maintains copies of a source database to allow virtual databases to be provisioned. Transaction logs received from the source database are applied to database blocks stored in the storage system to update the database blocks. The storage system checks if there are defect in the transaction logs of the source database. If a defect in transaction logs is detected, the storage system obtains a latest copy of the source database. The storage system matches database blocks of the received copy with database blocks already available in the storage system to determine whether a corresponding database block is already available in the storage system. If a received database block is different from database blocks already available at the storage system, the received database block is stored in the storage system. The database blocks may be matched using a hash value of the database blocks.

Claims

1. A method for storing copies of databases in a storage system, the method comprising:

maintaining a plurality of copies of a source database on a storage system, each copy including a plurality of database blocks, wherein one or more database blocks are associated with at least two copies, the maintaining comprising, iteratively;

checking for defect in the transaction logs of the source database;

responsive to identifying no defect in the transaction logs;

receiving transaction logs of the source database, and

applying the received transaction logs to database blocks of the source database stored in the storage system;

responsive to detecting a defect in the transaction logs;

receiving a latest copy of data from the source database comprising a plurality of database blocks, and

for each database block received in the latest copy;

determining if the database block of the latest copy matches a corresponding database block of a previously stored copy of the source database, and

responsive to determining that the database block from the latest copy does not match the corresponding database block of the previously stored copy of the source database, storing the database block of the latest copy on the storage system; and

creating a snapshot on the storage system representing a copy of the source database at a point in time.

2. The method of claim 1, wherein matching the database block of the latest copy with the database block of the previously stored copy comprises comparing a hash value based on the database block of the latest copy with a hash value of the corresponding database block of the previously stored copy.

3. The method of claim 1, wherein a snapshot comprises a plurality of database blocks representing information stored in the source database at a point in time.

4. The method of claim 3, wherein the plurality of database blocks associated with the snapshot comprises one or more database blocks to which transaction logs of the source database are applied, the applied transaction logs representing updates of the source database executed before the point in time.

5. The method of claim 1, further comprising:

responsive to receiving a request to provision a virtual database corresponding to a point in time, returning a file structure identifying a plurality of database blocks comprising at least a first database block to which a first set of transaction logs is applied and second database block to which a second set of transaction logs is applied, the first and second sets of transaction logs received at different points in time.

6. The method of claim 1, wherein checking for defect in the transaction logs comprises:

sending a request to the source database to determine whether there is a defect in the transaction logs; and

receiving a response from the source database indicating whether there is defect in the transaction logs.

7. The method of claim 1, wherein the plurality of database blocks in the latest copy of the source database includes database blocks of the source database that changed since a previous copy of the source database was obtained as well as database blocks that did not change since the previous copy of the source database was obtained.

8. The method of claim 1, wherein detecting a defect in the transaction logs comprises:

identifying a transaction log with a first log sequence number followed by a transaction log with a second log sequence number that is not consecutive with respect to the first log sequence number.

9. The method of claim 1, further comprising:

responsive to applying the received transaction logs to a copy of the source database, receiving information indicating a failure; and

responsive to detecting a failure applying the transaction logs, receiving a latest copy of data from the source database.

10. The method of claim 1, wherein receiving the latest copy of the source database comprises taking a full backup of the source database and restoring database blocks of the source database from the full backup.

11. The method of claim 1, wherein the transaction logs of the source database are received from a backup server configured to store a copy of the source database and receive transaction logs in a backup format from the source database, the method further comprising:

restoring the transaction logs from the backup format before applying the transaction logs.

12. A computer program product having a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising:

a database storage system configured to maintain a plurality of copies of a source database on a storage system, each copy including a plurality of database blocks, wherein one or more database blocks are associated with at least two copies, the database storage system comprising;

a transaction log manager configured to;

check for defect in the transaction logs of the source database;

responsive to identifying no defect in the transaction logs;

receive transaction logs of the source database, and

apply the received transaction logs to database blocks of the source database stored in the storage system;

a point-in-time copy manager configured to;

responsive to the transaction log manager detecting a defect in the transaction logs, receive a latest copy of data from the source database comprising a plurality of database blocks, and

for each database block received in the latest copy;

determine if the database block of the latest copy matches a corresponding database block of a previously stored copy of the source database, and

responsive to determining that the database block from the latest copy does not match the corresponding database block of the previously stored copy of the source database, store the database block of the latest copy on the storage system; and

a snapshot manager configured to create a snapshot on the storage system representing information stored in the source database at a point in time.

13. The computer program product of claim 12, wherein the point-in-time copy manager is configured to match the database block of the latest copy with the database block of the previously stored copy by comparing a hash value based on the database block of the latest copy with a hash value of the corresponding database block of the previously stored copy.

14. The computer program product of claim 12, wherein the computer program modules further comprise a virtual database manager configured to:

responsive to receiving a request to provision a virtual database corresponding to a point in time, return a file structure identifying a plurality of database blocks comprising at least a first database block to which a first set of transaction logs is applied and second database block to which a second set of transaction logs is applied, the first and second sets of transaction logs received at different points in time.

15. The computer program product of claim 12, wherein the point-in-time copy manager is further configured to:

responsive to applying the received transaction logs to a copy of the source database, receive information indicating a failure; and

responsive to detecting a failure applying the transaction logs, receive a latest copy of data from the source database.

16. The computer program product of claim 12, wherein the transaction log manager is configured to receive the transaction logs of the source database from a backup server configured to store a copy of the source database and receive transaction logs in a backup format from the source database, the transaction log manager further configured to:

restore the transaction logs from the backup format before applying the transaction logs.

17. A computer-implemented system for storing copies of databases in a storage system, the system comprising:

a computer processor; and

a computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising;

a database storage system configured to maintain a plurality of copies of a source database on a storage system, each copy including a plurality of database blocks, wherein one or more database blocks are associated with at least two copies, the database storage system comprising;

a transaction log manager configured to;

check for defect in the transaction logs of the source database;

responsive to identifying no defect in the transaction logs;

receive transaction logs of the source database, and

apply the received transaction logs to database blocks of the source database stored in the storage system;

a point-in-time copy manager configured to;

responsive to the transaction log manager detecting a defect in the transaction logs, receive a latest copy of data from the source database comprising a plurality of database blocks, and

for each database block received in the latest copy;

determine if the database block of the latest copy matches a corresponding database block of a previously stored copy of the source database, and

responsive to determining that the database block from the latest copy does not match the corresponding database block of the previously stored copy of the source database, store the database block of the latest copy on the storage system; and

a snapshot manager configured to create a snapshot on the storage system representing information stored in the source database at a point in time.

18. The computer system of claim 17, wherein the point-in-time copy manager is configured to match the database block of the latest copy with the database block of the previously stored copy by comparing a hash value based on the database block of the latest copy with a hash value of the corresponding database block of the previously stored copy.

19. The computer system of claim 17, wherein the computer program modules further comprise a virtual database manager configured to:

responsive to receiving a request to provision a virtual database corresponding to a point in time, return a file structure identifying a plurality of database blocks comprising at least a first database block to which a first set of transaction logs is applied and second database block to which a second set of transaction logs is applied, the first and second sets of transaction logs received at different points in time.

20. The computer system of claim 17, wherein the point-in-time copy manager is further configured to:

responsive to applying the received transaction logs to a copy of the source database, receive information indicating a failure; and

responsive to detecting a failure applying the transaction logs, receive a latest copy of data from the source database.