HIGH AVAILABILITY DISTRIBUTED DEDUPLICATED STORAGE SYSTEM

• US 20140201171A1
• Filed: 01/10/2014
• Published: 07/17/2014
• Est. Priority Date: 01/11/2013
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar, Kochunni, Jaidev Oppath, Agrawal, Saurabh, Narulkar, Abhishek
• Granted Patent: US 9,665,591 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A high availability distributed, deduplicated storage system according to certain embodiments is arranged to include multiple deduplication database media agents. The deduplication database media agents store signatures of data blocks stored in secondary storage. In addition, the deduplication database media agents are configured as failover deduplication database media agents in the event that one of the deduplication database media agents becomes unavailable.

Claims

1. A computer-implemented method of performing a storage operation in a distributed, deduplicated storage system, comprising:

during a period of availability of a first deduplication database server of a plurality of deduplication database servers and during a period of availability of a second deduplication database server of the plurality of deduplication database servers, the first deduplication database server designated as a failover deduplication database server for the second deduplication database server, the plurality of deduplication database servers storing signatures of data blocks stored in secondary storage which correspond to data blocks received from primary storage;

receiving at one or more secondary storage computing devices and as part of a first secondary copy operation a first set of one or more signatures corresponding to one or more data blocks stored in primary storage;

identifying the second deduplication database server as the deduplication database server assigned to store the first set of signatures;

determining that the first set of signatures match a second set of signatures comprising one or more signatures that were stored in the first deduplication database server during a previous period of unavailability of the second deduplication database server; and

with the one or more secondary storage computing devices, querying one of the first deduplication database server and the second deduplication database server for the first set of signatures based at least on a storage policy.

2. The method of claim 1, wherein following the period of unavailability, the method further comprises:

determining that the second deduplication database server is available; and

in response to determining that the second deduplication database server is available, copying the second set of signatures from the first deduplication server to the second deduplication database server,

wherein said querying comprises querying the second deduplication database server for the first set of signatures.

3. The method of claim 1, wherein following the period of unavailability, the method further comprises:

determining that the second deduplication database server is available;

in response to determining that the second deduplication database server is available, retaining the first set of signatures at the first deduplication database server,

wherein said querying comprises querying the first deduplication database server for the first set of signatures.

4. The method of claim 1, wherein each deduplication database server is assigned to store a different group of unique signatures.

5. The method of claim 1, wherein, during periods of availability of the first deduplication database server, the first deduplication database server is assigned to store a set of signatures that is different from a set of signatures stored by the second deduplication database.

6. The method of claim 1, wherein each deduplication database server of the plurality of deduplication database servers is identified as a failover deduplication database server to at least another one of the plurality of deduplication database servers.

7. The method of claim 1, further comprising identifying a third deduplication database server as the failover deduplication database server for the first deduplication database server and identifying the second deduplication database server as the failover deduplication database server for the third deduplication database server.

8. A distributed deduplicated storage system, comprising:

a plurality of deduplication database servers each configured to store signatures of a corresponding subset of a plurality of data blocks stored in a plurality of secondary storage devices, the plurality of data blocks corresponding to data blocks received from primary storage, wherein a first deduplication database server of the plurality of deduplication database servers is identified as a failover deduplication database server for a second deduplication database server of the plurality of deduplication database servers;

a plurality of secondary storage computing devices each comprising one or more processors and storage, wherein following a period of unavailability of the second deduplication database server, at least one secondary storage computing device of the plurality of secondary storage computing devices is configured to;

receive a first set of one or more signatures corresponding to one or more data blocks stored in the primary storage,

identify the second deduplication database server as the deduplication database server assigned to store the first set of signatures,

determine that the first set of signatures match a second set of signatures comprising one or more signatures that were stored in the first deduplication database server during the period of unavailability of the second deduplication database server, and

query one of the first deduplication database server and the second deduplication database server for the first set of signatures based at least on a storage policy.

9. The system of claim 8, wherein following the period of unavailability, the at least one secondary storage computing device is further configured to:

determine that the second deduplication database server is available;

copy the second set of signatures from the first deduplication database server to the second deduplication database server,

wherein, based at least on the storage policy, the second deduplication database server is queried for the first set of signatures.

10. The system of claim 8, wherein following the period of unavailability, the at least one secondary storage computing device is further configured to:

determine that the second deduplication database server is available;

retain the first set of signatures at the first deduplication database server,

wherein, based at least on the storage policy, the first deduplication database server is queried for the first set of signatures.

11. The system of claim 8, wherein each deduplication database server is assigned to store a different group of unique signatures.

12. The system of claim 8, wherein during periods of availability of the first deduplication database server the first deduplication database server is assigned to store a set of signatures that is different from a set of signatures stored by the second deduplication database.

13. The system of claim 8, wherein each deduplication database server of the plurality of deduplication database servers is identified as a failover deduplication database server to at least another one of the plurality of deduplication database servers.

14. The system of claim 8, further comprising a third deduplication database server identified as the failover deduplication database server for the first deduplication database server, wherein the second deduplication database server is identified as the failover deduplication database server for the third deduplication database server.

15. The system of claim 8, wherein the at least one of the plurality of secondary storage computing devices is further configured to store in the plurality off secondary storage devices the location of a copy of the particular data block upon determining that the signature of the particular data block resides in the first deduplication database server.

16. The system of claim 8, wherein the at least one of the plurality of secondary storage computing devices is further configured to store a copy of the particular data block in the plurality of secondary storage devices and store the signature of the particular data block and the location of the copy of the particular data block in the first deduplication database server upon determining that the signature of the particular data block does not exist at the first deduplication database server.

17. A computer-implemented method of performing a storage operation in a distributed, deduplicated storage system, comprising:

designating a first deduplication database server of a plurality of deduplication database servers as a failover deduplication database server for a second deduplication database server of the plurality of the plurality of deduplication database servers, wherein the plurality of deduplication database servers store signatures of data blocks stored in secondary storage, wherein the data blocks stored in secondary storage correspond to data blocks received from primary storage during one or more previous secondary copy operations;

during a period of unavailability of the second deduplication database server and as part of a first secondary copy operation;

receiving at one or more secondary storage computing devices a first set of signatures comprising one or more signatures which correspond to one or more data blocks stored in primary storage;

with the one or more secondary storage computing devices, determining that the second deduplication database server is unavailable; and

storing the first set of signatures in the first deduplication database server;

after the period of unavailability of the second deduplication database server;

determining that the second deduplication database server is available;

receiving at the one or more secondary storage computing devices and as part of a second secondary copy operation a second set of one or more signatures that match the first set of signatures that were stored in the first deduplication database server during the period of unavailability of the second deduplication database server;

based at least on a storage policy, selecting one of the first deduplication database server and the second deduplication database server to query for the second set of signatures, and

with the one or more secondary storage computing devices, querying the selected deduplication database server for the second set of signatures.

18. The method of claim 17, wherein following the period of unavailability, the method further comprises:

copying the first set of signatures to the second deduplication database server,

wherein said selecting comprises selecting the second deduplication database server to query for the second set of signatures.

19. The method of claim 17, wherein following the period of unavailability, the method further comprises:

retaining the first set of signatures at the first deduplication database server; and

wherein said selecting comprises selecting the first deduplication database server to query for the second set of signatures.