REDUNDANT AND ROBUST DISTRIBUTED DEDUPLICATION DATA STORAGE SYSTEM

• US 20170193003A1
• Filed: 10/20/2016
• Published: 07/06/2017
• Est. Priority Date: 12/30/2015
• Status: Abandoned Application

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: ANKIREDDYPALLE, Ramachandra Reddy, VIJAYAN, Manoj Kumar, ATTARDE, Deepak Raghunath, KOCHUNNI, Jaidev Oppath
• Application Number:
  US15/299,299
• Publication Number:
  US 20170193003A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 : Plurality of storage devices
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods are provided herein for automatically configuring newly installed secondary storage computing devices and managing secondary storage computing devices when one or more become unavailable. For example, a storage manager can then detect the computing resources available to the newly installed secondary storage computing device, assign a role to the newly installed secondary storage computing device based on the detected computing resources, configure the newly installed secondary storage computing device with deduplication and storage policies used by the other secondary storage computing devices, re-partition secondary storage devices to allocate memory for the newly installed secondary storage computing device, and instruct other secondary storage computing devices to replicate their managed data such that the newly installed secondary storage computing device has access to the replicated data.

Claims

1. A networked information management system configured to automatically configure installed data storage computers, the networked information management system comprising:

a first data storage computer comprising computer hardware, wherein the first data storage computer is a first control node configured to manage first deduplication information stored in a first deduplication database and direct read and write requests to secondary nodes, and wherein the first data storage computer manages first data in a secondary storage device;

a second data storage computer comprising computer hardware, wherein the second data storage computer is installed in the networked information management system after the first data storage computer; and

a storage manager comprising computer hardware configured to;

detect that the second data storage computer is installed in the networked information management system,

determine whether the second data storage computer is a second control node or a first secondary node based on computing resources available to the second data storage computer,

partition the secondary storage device such that a first portion of the secondary storage device is assigned to the first data storage computer and a second portion of the secondary storage device is assigned to the second data storage computer, and

instruct the first data storage computer to replicate the first data and transmit the replicated first data to the second data storage computer for storage in the second portion of the secondary storage device.

2. The networked information management system of claim 1, wherein the first data storage computer is configured with a deduplication policy and a storage policy, and wherein the storage manager is further configured to configure the second data storage computer with the deduplication policy and the storage policy.

3. The networked information management system of claim 1, wherein the second data storage computer is the second control node, and wherein the storage manager is further configured to instruct the first data storage computer to replicate the first deduplication information and transmit the replicated first deduplication information to the second data storage computer for storage in a second deduplication database.

4. The networked information management system of claim 3, wherein the second deduplication database comprises a third portion allocated to the first data storage computer and a fourth portion allocated to the second data storage computer, and wherein the replicated first deduplication information is stored in the third portion of the second deduplication database.

5. The networked information management system of claim 3, wherein the replicated first deduplication information comprises at least one of a data block signature,

a storage location of a data block, or a count of a number of times the data block is used.

6. The networked information management system of claim 1, further comprising a third data storage computer comprising computer hardware, wherein the third data storage computer is a second secondary node, wherein a third portion of the secondary storage device is assigned to the third data storage computer, and wherein the third portion comprises the replicated first data and replicated second data corresponding to the second data storage computer.

7. The networked information management system of claim 6, wherein the second data storage computer is the first secondary node, wherein the second data storage computer is unavailable, and wherein the storage manager is further configured to:

receive a read request intended for the second data storage computer; and

transmit the read request to the first data storage computer, wherein the first data storage computer routes the read request to the third data storage computer instead of the second data storage computer such that the third data storage computer can retrieve a portion of the replicated second data that corresponds with the read request.

8. The networked information management system of claim 1, wherein the second portion of the secondary storage device comprises a third portion allocated to the first data storage computer and a fourth portion allocated to the second data storage computer, and wherein the replicated first data is stored in the third portion.

9. A computer-implemented method for automatically configuring installed data storage computers, the computer-implemented method comprising:

detecting that a first data storage computer is installed in a networked information management system, wherein the first data storage computer is a first control node configured to manage first deduplication information stored in a first deduplication database and direct read and write requests to secondary nodes, and wherein the first data storage computer manages first data in a secondary storage device;

detecting that a second data storage computer is installed in the networked information management system, wherein the second data storage computer is installed in the networked information management system after the first data storage computer;

determining whether the second data storage computer is a second control node or a first secondary node based on computing resources available to the second data storage computer;

partitioning the secondary storage device such that a first portion of the secondary storage device is assigned to the first data storage computer and a second portion of the secondary storage device is assigned to the second data storage computer; and

instructing the first data storage computer to replicate the first data and transmit the replicated first data to the second data storage computer for storage in the second portion of the secondary storage device.

10. The computer-implemented method of claim 9, wherein the first data storage computer is configured with a deduplication policy and a storage policy, and wherein the computer-implemented method further comprises configuring the second data storage computer with the deduplication policy and the storage policy.

11. The computer-implemented method of claim 9, wherein the second data storage computer is the second control node, and wherein the computer-implemented method further comprises instructing the first data storage computer to replicate the first deduplication information and transmit the replicated first deduplication information to the second data storage computer for storage in a second deduplication database.

12. The computer-implemented method of claim 11, wherein the second deduplication database comprises a third portion allocated to the first data storage computer and a fourth portion allocated to the second data storage computer, and wherein the replicated first deduplication information is stored in the third portion of the second deduplication database.

13. The computer-implemented method of claim 11, wherein the replicated first deduplication information comprises at least one of a data block signature, a storage location of a data block, or a count of a number of times the data block is used.

14. The computer-implemented method of claim 9, wherein a third data storage computer is a second secondary node, wherein a third portion of the secondary storage device is assigned to the third data storage computer, and wherein the third portion comprises the replicated first data and replicated second data corresponding to the second data storage computer.

15. The computer-implemented method of claim 14, wherein the second data storage computer is the first secondary node, wherein the second data storage computer is unavailable, and wherein the computer-implemented method further comprises:

receiving a read request intended for the second data storage computer; and

transmitting the read request to the first data storage computer, wherein the first data storage computer routes the read request to the third data storage computer instead of the second data storage computer such that the third data storage computer can retrieve a portion of the replicated second data that corresponds with the read request.

16. The computer-implemented method of claim 9, wherein the second portion of the secondary storage device comprises a third portion allocated to the first data storage computer and a fourth portion allocated to the second data storage computer, and wherein the replicated first data is stored in the third portion.

17. A networked information management system configured to automatically configure installed data storage computers, the networked information management system comprising:

a first data storage computer comprising computer hardware, wherein the first data storage computer is a first control node configured to manage first deduplication information stored in a first deduplication database and direct read and write requests to secondary nodes, and wherein the first data storage computer manages first data in a secondary storage device;

a second data storage computer comprising computer hardware; and

a storage manager comprising computer hardware configured to;

detect that the second data storage computer is installed in the networked information management system,

determine that the second data storage computer is a second control node based on computing resources available to the second data storage computer,

partition the secondary storage device such that a first portion of the secondary storage device is assigned to the first data storage computer and a second portion of the secondary storage device is assigned to the second data storage computer,

instruct the first data storage computer to replicate the first data,

receive the replicated first data, and

transmit the replicated first data to the second data storage computer for storage in the second portion of the secondary storage device.

18. The networked information management system of claim 17, wherein the first data storage computer is configured with a deduplication policy and a storage policy, and wherein the storage manager is further configured to configure the second data storage computer with the deduplication policy and the storage policy.

19. The networked information management system of claim 17, wherein the storage manager is further configured to:

instruct the first data storage computer to replicate the first deduplication information;

receive the replicated first deduplication information; and

transmit the replicated first deduplication information to the second data storage computer for storage in a second deduplication database.

20. The networked information management system of claim 19, wherein the second deduplication database comprises a third portion allocated to the first data storage computer and a fourth portion allocated to the second data storage computer, and wherein the replicated first deduplication information is stored in the third portion of the second deduplication database.