Efficient deduplication database validation

• US 10,572,348 B2
• Filed: 06/14/2018
• Issued: 02/25/2020
• Est. Priority Date: 04/14/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Maranna, Chandrashekar, Vijayan, Manoj Kumar, Narulkar, Abhishek, Attarde, Deepak Raghunath, Agrawal, Saurabh
• Primary Examiner(s): Tsui, Daniel D
• Application Number:
  US16/008,591
• Publication Number:
  US 20180357132A1
• Time in Patent Office: 621 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/83 the solution involving signatures
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

According to certain aspects, a method can include receiving an indication that a restoration of a deduplication database using a secondary copy of a file associated with a secondary copy job is complete; retrieving a first data fingerprint from a data storage database, wherein the first data fingerprint is associated with the secondary copy job used to restore the deduplication database; retrieving a second data fingerprint from a deduplication database media agent, wherein the second data fingerprint is associated with the secondary copy job used to restore the deduplication database; comparing the first data fingerprint with the second data fingerprint to determine whether the first data fingerprint and the second data fingerprint match; and transmitting an instruction to the deduplication database media agent to rebuild the restored deduplication database in response to a determination that the first data fingerprint and the second data fingerprint do not match.

Claims

1. A system configured to validate a database including deduplication information, the system comprising:

one or more computing devices each having one or more hardware processors and configured to;

determine that a current version of a first database is invalid, the first database including information about a set of deduplication data blocks, wherein the deduplication data blocks are used to create deduplicated secondary copies on one or more secondary storage devices;

obtain a previous version of the first database;

retrieve a first data fingerprint from a second database, the second database associated with a storage management computer, the first data fingerprint corresponding to a secondary copy job which resulted in generation of a secondary copy of a file, the first data fingerprint comprising at least one of;

(i) an indication of a first count of unique data blocks stored in the secondary copy of the file;

(ii) an indication of a first count of references stored in the secondary copy of the file that reference data blocks stored in other secondary copies;

(iii) an indication of a size of the unique data blocks stored in a secondary copy of a file; and

(iv) an indication of a size of the references stored in the secondary copy of the file;

compare the first data fingerprint with a second data fingerprint, wherein the second data fingerprint corresponds to the obtained previous version of the first database; and

in response to a determination that the first data fingerprint and the second data fingerprint do not match, rebuild the first database using the secondary copy of the file.

2. The system of claim 1, wherein the one or more computing devices are further configured to generate, at a time that the previous version of the first database is created, the first data fingerprint.

3. The system of claim 1, wherein the one or more computing devices are further configured to:

receive an indication that a secondary copy of a file is created;

determine a number of unique data blocks in the secondary copy;

determine a number of reference data blocks in the secondary copy;

determine a size of the unique data blocks in the secondary copy;

determine a size of the reference data blocks in the secondary copy;

store, as data fingerprint, the determined number of unique data blocks, the determined number of reference data blocks, the determined size of the unique data blocks, and the determined size of the reference data blocks, in the storage manager database in an entry associated with the secondary copy;

receive a request for the data fingerprint; and

transmit the data fingerprint.

4. The system of claim 1, wherein the second data fingerprint comprises an indication of a second count of unique data blocks stored in the secondary copy of the file, and wherein the one or more computing devices are further configured to:

compare the first count of the unique data blocks stored in the secondary copy of the file with the second count of the unique data blocks stored in the secondary copy of the file; and

rebuild the first database in response to a determination that the first count of the unique data blocks stored in the secondary copy of the file and the second count of the unique data blocks stored in the secondary copy of the file do not match.

5. The system of claim 1, wherein the one or more computing devices are configured to obtain the previous version of the first database by restoring a backup copy of the first database.

6. The system of claim 1, wherein the one or more computing devices are further configured to determine that the current version of the first database is invalid based on a flag provided by the first database.

7. A computer-implemented method for validating a database including deduplication information, the computer-implemented method comprising:

determining that a current version of a first database is invalid, the first database including information about a set of deduplication data blocks, wherein the deduplication data blocks are used to create deduplicated secondary copies on one or more secondary storage devices;

obtaining a previous version of the first database;

retrieving a first data fingerprint from a second database, the second database associated with a storage management computer, the first data fingerprint corresponding to a secondary copy job which resulted in the generation of a secondary copy of a file, the first data fingerprint comprising at least one of;

(i) an indication of a first count of unique data blocks stored in the secondary copy of the file;

(ii) an indication of a first count of references stored in the secondary copy of the file that reference data blocks stored in other secondary copies;

(iii) an indication of a size of the unique data blocks stored in a secondary copy of a file; and

(iv) an indication of a size of the references stored in the secondary copy of the file;

comparing the first data fingerprint with a second data fingerprint, wherein the second data fingerprint corresponds to the obtained previous version of the first database; and

in response to a determination that the first data fingerprint and the second data fingerprint do not match, rebuilding the first database using the secondary copy of the file.

8. The computer-implemented method of claim 7, further configured to:

receive an indication that a secondary copy of a file is created;

determine a number of unique data blocks in the secondary copy;

determine a number of reference data blocks in the secondary copy;

determine a size of the unique data blocks in the secondary copy;

determine a size of the reference data blocks in the secondary copy;

store, as data fingerprint, the determined number of unique data blocks, the determined number of reference data blocks, the determined size of the unique data blocks, and the determined size of the reference data blocks, in the storage manager database in an entry associated with the secondary copy;

receive a request for the data fingerprint; and

transmit the data fingerprint.

9. The computer-implemented method of claim 7, wherein said comparing the first data fingerprint with the second data fingerprint comprises:

comparing the first count of the unique data blocks stored in the secondary copy of the file with the second count of the unique data blocks stored in the secondary copy of the file; and

rebuilding the previous version of the first database in response to a determination that the first count of the unique data blocks stored in the secondary copy of the file and the second count of the unique data blocks stored in the secondary copy of the file do not match.

10. The computer-implemented method of claim 7, wherein said obtaining the previous version of the first database comprises restoring a backup copy of the first database.

11. The computer-implemented method of claim 7, wherein the first database comprises information about a set of deduplication data blocks, and wherein the deduplication data blocks are used to create deduplicated secondary copies on one or more secondary storage devices.

12. The computer-implemented method of claim 7, wherein determining that a current version of a first database is invalid further comprises determining that the current version of the first database is invalid based on a flag provided by the first database.

13. A system configured to validate a database including deduplication information, the system comprising:

one or more computing devices each having one or more hardware processors and configured to;

receive an indication that a restoration of a previous version of a first database is complete, the first database including deduplication information;

obtain a first data fingerprint of a plurality of data fingerprints, wherein the first data fingerprint is associated with a secondary copy job which resulted in generation of a first secondary copy of a file, wherein the first data fingerprint comprises an indication of a first count of unique data blocks stored in the first secondary copy and an indication of a first count of references stored in the first secondary copy that reference data blocks stored in other secondary copies;

obtain a second data fingerprint, wherein the second data fingerprint is derived from the restored previous version of the first database;

compare the first data fingerprint with the second data fingerprint to determine whether the first data fingerprint and the second data fingerprint match; and

in response to a determination that the first data fingerprint and the second data fingerprint do not match, transmit an instruction to rebuild the first database using the secondary copy of the file.

14. The system of claim 13, wherein the first data fingerprint further comprises at least one of an identification of a total size of the unique data blocks or an identification of a total size of the references.

15. The system of claim 13, wherein the first data fingerprint was generated at a time that the secondary copy job is being performed.

16. The system of claim 13, the one or more computing devices are further configured to:

receive an indication that a secondary copy of a file is created;

determine a number of unique data blocks in the secondary copy;

determine a number of reference data blocks in the secondary copy;

determine a size of the unique data blocks in the secondary copy;

determine a size of the reference data blocks in the secondary copy;

store, as data fingerprint, the determined number of unique data blocks, the determined number of reference data blocks, the determined size of the unique data blocks, and the determined size of the reference data blocks, in the storage manager database in an entry associated with the secondary copy;

receive a request for the data fingerprint; and

transmit the data fingerprint.

17. The system of claim 13, wherein the second data fingerprint comprises an indication of a second count of unique data blocks stored in the first secondary copy of the first file, and wherein the one or more computing devices are configured to:

compare the first count of the unique data blocks with the second count of the unique data blocks; and

transmit the instruction to rebuild the first database in response to a determination that the first count of the unique data blocks and the second count of the unique data blocks do not match.

18. The system of claim 13, wherein the one or more computing devices are configured to:

process entries in the restored previous version of the first database in response to completion of the restoration; and

generate the second data fingerprint based on the processing of the entries in the restored previous version of the first database.

19. The system of claim 18, wherein each entry in the first database associated with the secondary copy job corresponds to a data block.