High availability distributed deduplicated storage system

• US 10,229,133 B2
• Filed: 03/30/2017
• Issued: 03/12/2019
• Est. Priority Date: 01/11/2013
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Kochunni, Jaidev Oppath, Agrawal, Saurabh, Narulkar, Abhishek
• Primary Examiner(s): Torgrimson, Tyler J
• Application Number:
  US15/474,730
• Publication Number:
  US 20170206219A1
• Time in Patent Office: 712 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A high availability distributed, deduplicated storage system according to certain embodiments is arranged to include multiple deduplication database media agents. The deduplication database media agents store signatures of data blocks stored in secondary storage. In addition, the deduplication database media agents are configured as failover deduplication database media agents in the event that one of the deduplication database media agents becomes unavailable.

Claims

1. A method of performing a storage operation in a distributed, deduplicated storage system, comprising:

receiving, at a first secondary storage computing device, a request from a client computing device to perform a storage operation corresponding to a first data block,

wherein a plurality of deduplication database computing devices are communicatively coupled to the first secondary storage computing device, wherein the plurality of deduplication database computing devices comprises a first deduplication database computing device and a second deduplication database computing device, wherein the first deduplication database computing device is configured to store a first subset of signature blocks and is designated as a failover deduplication database computing device for the second deduplication database computing device, and wherein the second deduplication database computing device is configured to store a second subset of signature blocks that is different than the first subset of signature blocks;

in response to the request and using one or more processors, identifying that the second deduplication database computing device is assigned to store a first signature corresponding to the first data block;

determining that the first signature is not stored in the second deduplication database computing device due to an unavailability of the second deduplication database computing device and is instead stored in the first deduplication database computing device,

wherein the first subset of signature blocks comprises a first signature block, and

wherein the first signature block comprises the first signature, an indication of a location of the first data block in a secondary storage device, and a value representing a number of references to the first data block in the secondary storage device;

wherein said determining includes querying a failover index using the first signature, and receiving an indication, from the failover index, that the first signature is stored in the first deduplication database computing device; and

in response to determining that the first signature is not stored in the second deduplication database computing device and is instead stored in the first deduplication database computing device, querying the first deduplication database computing device instead of the second deduplication database computing device for the first signature and the location of the first data block in the secondary storage device, wherein querying the first deduplication database computing device for the first signature causes the first deduplication database computing device to increment the value representing the number of references to the first data block in the secondary storage device.

2. The method of claim 1, wherein the first signature is stored in the first deduplication database computing device instead of the second deduplication database computing device because the second deduplication database computing device was unavailable when the first signature block was stored in the first deduplication database computing device.

3. The method of claim 1, wherein the storage operation comprises a pruning operation.

4. The method of claim 3, wherein querying the first deduplication database computing device for the first signature causes the first deduplication database computing device to decrement the value representing the number of references to the first data block in the secondary storage device.

5. The method of claim 4, further comprising deleting the first data block from the secondary storage device in response to a determination that the decremented value is zero.

6. The method of claim 1, wherein identifying that the second deduplication database computing device is assigned to store the first signature corresponding to the first data block further comprises:

determining a second signature of the first data block;

performing a modulo operation on the second signature; and

identifying that the second deduplication database computing device is assigned to store the first data block based on a result of the performed modulo operation.

7. The method of claim 1, wherein each deduplication database computing device of the plurality of deduplication database computing devices is identified as a failover deduplication database computing device to another one of the plurality of deduplication database computing devices.

8. The method of claim 1, wherein a third deduplication database computing device in the plurality of deduplication database computing devices is designated as a failover deduplication database computing device for the first deduplication database computing device, and wherein the second deduplication database computing device is designated as a failover deduplication database computing device for the third deduplication database computing device.

9. A distributed deduplicated storage system, comprising:

a first deduplication database computing device configured to store a first subset of signature blocks;

a second deduplication database computing device configured to store a second subset of signature blocks,

wherein the first deduplication database computing device is designated as a failover deduplication database computing device for the second deduplication database computing device, and wherein the second subset of signature blocks is different than the first subset of signature blocks; and

a secondary storage computing device communicatively coupled to the first deduplication database computing device and the second deduplication database computing device, the secondary storage computing device comprising one or more processors and storage, wherein the secondary storage computing device is configured to;

receive a request to perform a storage operation corresponding to a first data block,

identify that the second deduplication database computing device is assigned to store a first signature corresponding to the first data block,

determine that the first signature is not stored in the second deduplication database computing device due to an unavailability of the second deduplication database computing device and is instead stored in the first deduplication database computing device, wherein the first subset of signature blocks comprises a first signature block, and wherein the first signature block comprises the first signature, an indication of a location of the first data block in a secondary storage device, and a value representing a number of references to the first data block in the secondary storage device, wherein said determining includes querying a failover index using the first signature, and receiving an indication, from the failover index, that the first signature is stored in the first deduplication database computing device; and

in response to the determination that the first signature is not stored in the second deduplication database computing device and is instead stored in the first deduplication database computing device, query the first deduplication database computing device instead of the second deduplication database computing device for the first signature and the location of the first data block in the secondary storage device, wherein querying the first deduplication database computing device for the first signature causes the first deduplication database computing device to increment or decrement the value representing the number of references to the first data block in the secondary storage device.

10. The system of claim 9, wherein the first signature is stored in the first deduplication database computing device instead of the second deduplication database computing device because the second deduplication database computing device was unavailable when the first signature block was stored in the first deduplication database computing device.

11. The system of claim 9, wherein the storage operation comprises a pruning operation.

12. The system of claim 11, wherein the first deduplication database computing device is further configured to decrement the value representing the number of references to the first data block in the secondary storage device.

13. The system of claim 12, wherein the secondary storage computing device is further configured to delete the first data block from the secondary storage device in response to a determination that the decremented value is zero.

14. The system of claim 9, wherein the secondary storage computing device is further configured to:

determine a second signature of the first data block;

perform a modulo operation on the second signature; and

identify that the second deduplication database computing device is assigned to store the first data block based on a result of the performed modulo operation.

15. The system of claim 9, further comprising a third deduplication database computing device configured to store a third subset of signature blocks.

16. The system of claim 15, wherein the third deduplication database computing device is designated as a failover deduplication database computing device for the first deduplication database computing device, and wherein the second deduplication database computing device is designated as a failover deduplication database computing device for the third deduplication database computing device.