Content-independent and database management system-independent synthetic full backup of a database based on snapshot technology

• US 9,904,598 B2
• Filed: 04/21/2015
• Issued: 02/27/2018
• Est. Priority Date: 04/21/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kumarasamy, Paramasivam
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US14/692,357
• Publication Number:
  US 20160314046A1
• Time in Patent Office: 1,043 Days
• Field of Search: 707739, 707745, 707746, 707735, 707747
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

The illustrative storage management system uses snapshot technologies and performs block-level analysis thereof and to synthesize full backups of a database, regardless of the database management system (DBMS) and content of the database and without involving native DBMS utilities. Synthetic full copies are created efficiently and use network transport resources sparingly. Successive snapshots of the database may be hardware snapshots and/or software snapshots. The illustrative storage management system tracks which data blocks in the database change over time after a baseline full copy is saved to secondary storage. Only the changed blocks are subsequently moved to secondary storage as incremental backups and are indexed for future reference. A synthetic full backup of the database, at a given point in time, may be created by synthesizing the original full backup with changed blocks from any number of later incremental backups. The synthetic full backup comprises pointers to appropriate data blocks, ignoring intermediate changes, and pointing to the most current data block in secondary storage relative to the given point in time chosen for the synthetic full backup. The illustrative storage management system may restore the synthetic full backup without native DBMS utilities.

Claims

1. A computer-implemented method for synthesizing a block-level backup copy of a database in a storage management system based on snapshots of the database, the method comprising executing on one or more computing devices the steps of:

generating a first snapshot of the database, which includes associated metadata, wherein the first snapshot is stored to a first storage device,

wherein a database management system (DBMS) that executes on a client computing device component of the storage management system performs operations that change data in the database;

storing, by a media agent component of the storage management system, a copy of the first snapshot to a second storage device;

after the first snapshot is generated, generating a plurality of successive snapshots of the database;

after each snapshot in the plurality of successive snapshots is generated,

identifying, by a data agent that is associated with the database management system, a respective set of data blocks which changed in the database relative to corresponding data blocks in an immediately preceding snapshot, which set of identified data blocks are designated changed data blocks for the respective snapshot, and

maintaining, by the data agent, a respective list of the changed data blocks for the respective snapshot;

extracting, by the media agent, the identified changed data blocks from the respective snapshot;

storing and indexing to the second storage device, by the media agent, a plurality of incremental backups of the database corresponding to respective snapshots in the plurality of successive snapshots,

wherein a respective incremental backup comprises the changed data blocks extracted from the respective snapshot;

generating, based on the indexing, by the media agent, a synthetic full block-level backup of the database represented at a given point in time,

wherein the synthetic full block-level backup is independent of database content and DBMS type, and comprises pointers to most-current data blocks relative to the given point in time, including pointers to changed data blocks and skipping over intermediate changed data blocks, as found by the media agent in the second storage device from;

(a) the copy of the first snapshot and (b) the plurality of incremental backups,

thereby resulting in the synthetic full block-level backup which represents the database at the given point in time;

storing, by the media agent to the second storage device, the synthetic full backup of the database;

reporting, by the media agent, completion of the synthetic full block-level backup of the database to a storage manager that maintains records thereof; and

restoring the database from the synthetic full block-level backup in the second storage device without using a native restore feature of the DBMS.

2. The method of claim 1 wherein the copy of the first snapshot stored by the media agent to the second storage device comprises all data blocks in the first snapshot.

3. The method of claim 1

wherein the restored database comprises all data blocks of the database at the time the most recent snapshot was generated in the plurality of successive snapshots;

and

wherein the restoring of the database is performed at least in part by the media agent.

4. The method of claim 1 wherein the generating of the plurality of successive snapshots of the database is performed by the first storage device.

5. The method of claim 1 wherein the generating of the plurality of successive snapshots of the database is performed by the client computing device.

6. The method of claim 1 wherein the snapshots are hardware snapshots created by the first storage device.

7. The method of claim 1 wherein the snapshots are software snapshots created by the client computing device.

8. A data storage management system useful for synthesizing a block-level backup copy of a database based on snapshots of the database, the system comprising:

a client computing device that (i) executes a database management system (DBMS), which operates upon a database that includes associated metadata, and (ii) also executes a data agent associated with the DBMS;

a storage array in communication with the client computing device, wherein the storage array stores the database and a plurality of successive snapshots of the database;

a secondary storage computing device that executes a media agent;

a secondary storage device in communication with the media agent, wherein the secondary storage device comprises a full backup of a first snapshot of the database taken at a first time, wherein the full backup and the first snapshot each comprises a first set of data blocks;

wherein the client computing device, when executing the data agent, is programmed to identify, after each snapshot is taken, data blocks that changed in the database relative to corresponding data blocks in the respective snapshot; and

wherein the secondary storage computing device, when executing the media agent, is programmed to;

index, into a first index, storage locations in the secondary storage device of the first set of data blocks of the full backup,

store a plurality of incremental backups of the database to the secondary storage device, wherein each respective incremental backup comprises a corresponding set of changed data blocks extracted by the media agent from a respective snapshot in the plurality of successive snapshots on the storage array,

wherein the extracted data blocks are data blocks that changed relative to an immediately preceding snapshot of the database, and

wherein the changed data blocks are identified by the data agent into a respective list maintained by the data agent for the respective incremental backup,

generate, for each incremental backup, a corresponding index of the storage locations in the secondary storage device of the data blocks of the respective incremental backup,

generate, based on one or more corresponding index, a synthetic full block-level backup of the database, which represents the database at the time of the most recent snapshot in the plurality of successive snapshots,

wherein the synthetic full block-level backup is independent of database content and DBMS type, and comprises pointers to most most-current data blocks relative to the time of the most recent snapshot, including pointers to changed data blocks and skipping over intermediate changed data blocks, as found by the media agent in;

(a) the full backup of the database, and (b) the plurality of incremental backups;

store the synthetic full backup of the database to the secondary storage device;

report completion of the synthetic full block-level backup of the database to a storage manager that maintains records thereof; and

restore the database from the synthetic full block-level backup in the secondary storage device without using a native restore feature of the DBMS.

9. The storage management system of claim 8 wherein the synthetic full block-level backup is based on the indexes corresponding to the incremental backups.

10. The storage management system of claim 8

wherein the restored database comprises all data blocks of the database at the time of the most recent snapshot in the plurality of successive snapshots.

11. A non-transitory computer-readable medium storing instructions that, when executed by at least one secondary storage computing device, which comprises one or more processors and computer memory, cause the secondary storage computing device to perform operations comprising:

storing a copy of a first snapshot of a database, which includes associated metadata, to a secondary storage device, which copy is designated a full backup of the database, and wherein a database management system (DBMS) operates upon the database;

generating a plurality of successive snapshots of the database and identifying a respective set of data blocks which changed in the database relative to corresponding data blocks in an immediately preceding snapshot, which set of identified data blocks are designated changed data blocks for the respective snapshot;

receiving identifiers of a first set of changed data blocks in the database relative to the first snapshot;

extracting the identified first set of changed data blocks from a second snapshot of the database which follows the first snapshot;

storing a first incremental backup of the database to the secondary storage device, wherein the first incremental backup comprises the first set of changed data blocks extracted from the second snapshot and maintain a respective list of the changed data blocks for the respective snapshot;

generating, for the first incremental backup, a corresponding first index of storage locations in the secondary storage device of the first set of changed data blocks;

generating, based at least in part on the first index, a first synthetic full block-level backup of the database, representing the database at the time the second snapshot was taken,

wherein the first synthetic full block-level backup is independent of database content and DBMS type, and comprises a set of pointers to most-current data blocks and skipping over intermediate changed data blocks relative to the time the second snapshot was taken, as found in;

(i) the full backup of the database and (ii) the first incremental backup of the database;

storing the first synthetic full backup of the database to the secondary storage device;

reporting completion of the first synthetic full block-level backup of the database to a storage manager that maintains records thereof; and

restoring the database from the first synthetic full block-level backup in the second storage device without using a native restore feature of the DBMS.

12. The non-transitory computer-readable medium of claim 11, wherein the copy of the first snapshot of the database stored by the secondary storage computing device to the secondary storage device comprises all data blocks in the first snapshot.

13. The non-transitory computer-readable medium of claim 11, wherein

the database is restored from the first synthetic full backup to a location in a primary storage device, and

wherein the database restored from the first synthetic full block-level backup comprises all data blocks of the database at the time the second snapshot was taken.

14. The non-transitory computer-readable medium of claim 11, wherein the operations further comprise:

receiving identifiers of a second set of changed data blocks in the database relative to the second snapshot;

extracting the identified second set of changed data blocks from a third snapshot of the database which follows the second snapshot;

storing a second incremental backup of the database to the secondary storage device, wherein the second incremental backup comprises the second set of changed data blocks extracted from the third snapshot; and

generating, for the second incremental backup, a corresponding second index of storage locations in the secondary storage device of the second set of changed data blocks;

generating, based at least in part on the second index, a second synthetic full block-level backup of the database, representing the database at the time the third snapshot was taken,

wherein the second synthetic full block-level backup is independent of database content and DBMS type, and comprises a set of pointers to most-current data blocks and skipping over intermediate changed data blocks relative to the time the second snapshot was taken, as found in;

(i) the full backup of the database, (ii) the first incremental backup of the database, and (iii) the second incremental backup of the database;

reporting completion of the second synthetic full block-level backup of the database to a storage manager that maintains records thereof; and

restoring the database from the second synthetic full block-level backup in the second storage device without using a native restore feature of the DBMS.

15. The non-transitory computer-readable medium of claim 11,

wherein the database restored from the second synthetic full block-level backup comprises all the data blocks of the database at the time the third snapshot was taken.