Maintaining a deduplication database

• US 9,633,056 B2
• Filed: 03/17/2014
• Issued: 04/25/2017
• Est. Priority Date: 03/17/2014
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Attarde, Deepak Raghunath, Vijayan, Manoj Kumar
• Primary Examiner(s): TORGRIMSON, TYLER J
• Application Number:
  US14/216,703
• Publication Number:
  US 20150261792A1
• Time in Patent Office: 1,135 Days
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors

Abstract

An information management system can modify working copies of database entries residing in a local database of a secondary storage computing device in response to instructions to modify the database entries residing in a deduplication database. If the working copy does not already reside in the local database, a copy of the database entry, or portion thereof, from the deduplication database can be used to generate the working copy. Based on a desired policy, the working copies in the local database can be merged with the actual database entries in the deduplication database.

Claims

1. A method for managing a deduplication database in a deduplication storage system, wherein the deduplication database stores database entries comprising deduplication signatures corresponding to deduplication data blocks stored in secondary storage, the method comprising:

by a secondary storage computing device comprising computer hardware, determining whether a working copy of a first database entry of a plurality of database entries of a deduplication database exists in a local database residing in main memory of the secondary storage computing device, wherein the deduplication database resides in one or more storage devices external to the secondary storage computing device;

based at least on a determination that the working copy of the first database entry does not reside in the local database, querying the deduplication database for the first database entry and generating the working copy of the first database entry based at least on information received from the deduplication database in response to the querying;

modifying the working copy of the first database entry based on a storage operation involving a stored deduplication data block corresponding to the first database entry;

detecting a triggering condition indicating that a plurality of working copies corresponding to a respective plurality of database entries in the deduplication database should be merged with the deduplication database, the plurality of working copies including the working copy of the first database entry contained in the local database; and

based at least on said detecting the triggering condition, updating the deduplication database by merging at least some contents of the plurality of working copies with the deduplication database.

2. The method of claim 1, wherein said detecting the triggering condition comprises one or more of detecting expiration of a time threshold since a previous merge and detecting a size threshold associated the local database being exceeded.

3. The method of claim 1, further comprising, following a failure associated with the secondary storage computing device, reconstructing the plurality of working copies using a log file generated prior to the failure.

4. The method of claim 1, wherein said modifying comprises at least one of incrementing a deduplication signature reference count and decrementing a deduplication signature reference count associated with the first database entry.

5. The method of claim 1, wherein the main memory of the secondary storage computing device comprises volatile memory.

6. The method of claim 1, wherein said querying the deduplication database results in receipt by the secondary storage computing device of contents of the first database entry, and wherein generating the working copy of the first database entry comprises generating the working copy of the first database entry based at least in part on the contents of the first database entry received by the secondary storage computing device.

7. The method of claim 1, wherein said querying the deduplication database comprises receiving an indication that the first database entry does not exist in the deduplication database, and wherein the working copy of the first database entry is generated using information provided to the secondary storage computing device by a primary storage subsystem as part of the storage operation.

8. The method of claim 1, wherein said updating comprises forwarding the plurality of working copies to the deduplication database and flushing the working copies from the local database.

9. The method of claim 1, wherein said updating the deduplication database comprises adding at least one database entry to the deduplication database.

10. The method of claim 1, further comprising, before beginning said updating, setting the local database as inactive and setting a second local database to be active.

11. The method of claim 10, wherein said setting the local database as inactive includes removing write-access to the local database.

12. A system for managing a deduplication database, comprising:

a deduplication database residing in one or more storage devices in a secondary storage subsystem and comprising database entries which include deduplication signatures corresponding to deduplication data blocks stored in the secondary storage subsystem; and

a secondary storage computing device comprising computer hardware, the computer hardware configured to;

maintain a local database residing in main memory of the secondary storage computing device;

determine whether a working copy of a first database entry in the deduplication database resides in the local database;

based at least on a determination that the working copy of the first database entry does not reside in the local database, query the deduplication database for the first database entry and generate the working copy of the first database entry based at least on a response from the deduplication database;

modify the working copy of the first database entry based on a storage operation involving a deduplication data block stored in the secondary storage subsystem that corresponds to the first database entry;

detect a triggering condition indicating that a plurality of working copies corresponding to a respective plurality of database entries in the deduplication database should be merged with the deduplication database, wherein the plurality of working copies includes the working copy of the first database entry; and

based at least on the detection of the triggering condition, updating the deduplication database by merging at least some contents of the plurality of working copies residing in the local database with the deduplication database.

13. The system of claim 12, wherein the secondary storage computing device is configured to detect the triggering condition upon one or more expiration of a time threshold since a previous merge and a size threshold associated with the local database being exceeded.

14. The system of claim 12, wherein the first database entry does not exist in the deduplication database prior to the query, and the secondary storage computing device generates the working copy of the first database entry based on data received from the primary storage subsystem as part of the storage operation.

15. The system of claim 12, wherein the first database entry exists in the deduplication database prior to the query, and the secondary storage computing device generates the working copy of the first database entry based at least on contents from the first database entry received from the deduplication database.

16. The system of claim 12, wherein the secondary storage computing device is configured to increment a deduplication signature reference count as part of the modification of the working copy of the first database entry.

17. The system of claim 12, wherein the secondary storage computing device is configured to forward the plurality of working copies to the deduplication database and flush the working copies from the local database in response to detection of the triggering condition.

18. The system of claim 17, wherein the secondary storage computing device is configured to add the first database entry to the deduplication database as part of the update, based on the working copy of the first database entry received from the secondary storage computing device.